Software Testing

“Software testers do not make software; they only make them better.” – Anonymous

In today’s world of complex programming, the importance of testing has increased multifold. If software gets released without formal testing, it can be a disaster and can cause irreparable loss to organizations’ reputation with their user and client base.

When it comes to Software testing, majority of the people associated with software industry contemplate it as necessary evil. The common thoughts shared by them are:

  • Why waste Money? Let’s focus on Software Development!
  • Why waste time on going through a testing cycle, when we could be live in production
  • My developers are good enough and would have tested their own code
  • “Testers”. Those guys just create problems. I have enough to worry about, why add another

But the ground reality has been appropriately summarized in the below mentioned school of thought:

“The bitterness of poor quality remains long after the sweetness of meeting the schedule has been forgotten.” – Anonymous

The reality is that the importance of Software Testing should not be undermined. It is absolutely necessary to ensure that the software being delivered meets its specifications and will work as expected for the end user.

Intersoft Data Labs has a mature Software Testing Practice that has been able to provide Software Testing Services to Fortune 100 clientele including CitiFinancial, MGM Resorts among others and to a large number of ISV’s.

Intersoft Data Labs has successfully helped organizations get past some of the key challenges that are faced in setting up a Software Testing Practice by providing cost effective and highly reliable testing services. We have worked extensively with ISVs and have helped them by taking complete ownership of the Software Testing Processes and delivering quality software on time.

Intersoft Data Labs Testing services include:

  • Functional and Integrated Systems Testing
  • Regression and Day in life testing
  • Functional Automation Testing
  • Performance Testing
  • Compatibility Testing
  • Usability Testing
  • Test Process Consulting
  • Testing Center of Excellence

Importance of Software Testing:

Cost of bugs in the modern business often is so high that you don’t have the privilege of committing mistakes. Software testing reins the code and keeps a tab on the developmental activities and confirms reliability of the software. In today’s world of complex programming, the importance of testing has increased many folds. If software gets released without formal testing, it can be a disaster and can cause irreparable loss to the users of software. Few of the important aspects of software testing are listed below:

  • Return-on-Investment – Early investment in testing ensure handsome returns in terms of cost savings, increased efficiency, increased quality and timely delivery of the software. The cost of bug fixing is considerably low when caught earlier. Please see the image below to understand how early detection of bugs prevent overheads:
  • Damage to Business & Reputation – If an organization is unable to provide service to their customers due to undetected bugs or untested software, its customers would eventually lose confidence in the organization and would probably opt for other available options in the market
  • Quality Product Assurance – Software testing ascertains the delivery of a quality product that can stand in the market for a longer duration
  • Risk Analysis – It helps to determine the areas in software which are prone to risks and needs to be analyzed before the software is released in the market
  • Documentation – An important practice involved in software testing is efficient and accurate documentation. It provides an organization with a reference material for the product that can help people to gain knowledge on the product
  • Identify areas of weakness and enhancements – As software testing involves thorough analysis of the business related to a product, the areas of weakness and enhancements gets identified at a very early stage
  • Enhanced Performance – Software testing helps the organization in enhancing the performance and productivity of a software by bringing forward all the aspects of the software
  • Compatibility Verifications – At times software behaves in an unintended manner on different machines that does not have similar configurations, software testing ensures the appropriate functioning of the product in different configurations which the end users more likely use while using the software
  • Verification of the accuracy of logic – Complete understanding and in depth knowledge of the business is a result of software testing process that helps in verification of the logic implemented behind the development of the product



Challenges of Software Testing Domain:

Eventually, the need of Software Testing has been realized by majority of the people in Software Industry. But, similar to other domains, Software Testing domain has its own challenges some of which are listed below:

  • Formation of good team – For setting up of a testing organization a good team comprising of skillful tester’s having adequate analytic and logical abilities is very important
  • Experienced resources – Lack of experienced resources in a testing organization restricts the organizations goal of achieving a quality product in limited time. Experienced resource’s comes up with useful suggestions that help in maintaining decent level of quality and standards. Some products needs compliance from the govt. organization, an expert’s involvement helps in completion of such compliances smoothly
  • Systematic Approach – The approach to be followed for testing of a product needs to be selected by keeping in view various aspects of the product such as timelines, target audience etc. Selection of an inappropriate approach can lead to disastrous results
  • Planning and management – Software Testing Life Cycle is spread over many stages, therefore, proper planning and management of all the activities should be done efficiently in a testing organization. Right decisions at right time save lots of chaos and ambiguity and ensure good quality and robust software. Planning and Management also include right composition of team structure, ability to take right decision while selecting the tools to be used while testing, defining the entry and exit criteria and an efficient review mechanism
  • Expertise in Automation tools – Testing of an application is a time consuming task, performing regression tests can be even more time consuming and tedious. Here, Automation testing can be of great help and a team having expertise in various automation tools can solve the purpose
  • Meeting the deadlines – For the success of any of the project, meeting the deadlines is very important and it also becomes a major hurdle if the management and planning is not done correctly
  • Lack in vision and prioritizing tasks – For the success of a testing organization it is very important to differentiate among “What to do” and “When to do”, miss-management of these two aspects can prove very costly
  • Compatibility Verifications – At times software behaves in an unintended manner on different machines that does not have similar configurations, software testing ensures the appropriate functioning of the product in different configurations which the end users more likely use while using the software
  • Verification of the accuracy of logic – Complete understanding and in depth knowledge of the business is a result of software testing process that helps in verification of the logic implemented behind the development of the product

Intersoft Data Lab’s QA ownership and Testing Processes:

Intersoft Data Labs has developed a dedicated practice for QA testing processes, specialized tools, frameworks, methodologies and consulting expertise to enhance testing efficiency and precision, reduce costs and decrease cycle times. Our testing practice offers a comprehensive suite of software testing ranging from BFSI domain to ISV products. Our strong competency in documentation and analysis complement our capability to be productive in a short time.

We use a risk-based prioritization approach and mainly focus on defect prevention by designing test strategies around the entire development life cycle and implementing optimized test plans. We have a responsive and agile mindset that provides a focused "On Time, Every Time" approach to each engagement resulting in customer satisfaction and repeat business. We have a proven track record of delivering on-time and on-budget fixed bid projects.

With the of use an established testing methodology and employment of a wide range of industry-standard testing tools that leverage organization with superior software quality, Intersoft Data Labs has enhanced business readiness and real competitive advantage. A typical representation of Intersoft Data Lab’s QA ownership is as below:

  • Intersoft Testing Ownership includes:
    • Gap Analysis from BR to FS to TS
    • Strategy including Entrance & Exit criteria, Risk & Mitigation planning
    • Enhancement Coverage
    • Test ware (Plans/Scenarios/Scripts) creation & Execution
    • Test Bed (Data Requirements)
    • Environment Setup from Technical Specifications
    • Version Control & Release Management from Developers
    • Defect Analyzers - Coordinators and Experienced BAs directing defects to Environment, Other Apps, Enhancements

Intersoft Data Lab’s Testing process and practices can be well defined through the points and diagram below:

  • Intersoft Testing Process and Practices includes:
    • Review Requirements
    • Trace Requirements
    • Estimate Work, Schedule
    • Review System Spec And Design
    • Develop Test Strategy
    • Develop Test Ware
    • Automate Test Ware
    • Execute Test Ware
    • Track Defects
    • Analyze Results
    • Prepare Metrics
    • Publish Results
    • Exit Reports
    • Knowledge Management



Testing Methodologies:

With time many testing methodologies have been developed in the Software industry. Some of these are quite successful and result oriented where as some has become obsolete due to frequent changes in the Software industry. Intersoft Data Labs practices two methodologies and the decision to use a specific methodology is taken after considering the requirements of the software and nature of the project. The two methodologies practiced by Intersoft Data Labs are:

  • V Model
  • Testing Application Model
  • Agile Model

V Model: It is considered as an extension of the Water fall. Instead of following a linear approach, as was followed in water fall model, in V Model the processes are bent upwards to form a typical V shape. The V Model depicts the relationship between each phase of development with each phase of testing. There are two phases of this model i.e. Verification and Validation. The diagram below captures the flows of a V Model:

Verification:

  • Requirement Analysis
  • System Design
  • Architecture Design
  • Module Design

Validation:

  • Unit Testing
  • Integration Testing
  • System Testing
  • User Acceptance Testing
  • Release Testing



Testing Process for Current (evolving) System

Intersoft follows the “Application Testing Life Cycle” model of testing to ensure quality testing of the Application (software) under test and make the application highly reliable before its final delivery.

Application Testing Life Cycle:

  • Requirements Phase
    • Test Requirements are studied and understood from the Requirement Specification documents, Functional Specification document, Design Specification documents (if any) and Use case Documents (if any) provided by the client
    • Based on the understanding gathered from the requirement documents a clarification sheet (having clarification, if any) is sent to the client
    • The Test Trace-ability Matrix for identifying Test Coverage is prepared
    • Test phases to be automated and % of automation (if automation is in the test scope)


  • Test Planning Phase
    • The Test Scope is decided
    • The Test Environment is thought of and set
    • Different Test phase and Test Methodologies are discussed and decided
    • Manual and Automation Testing plans are written
    • Low level Test Cases are prepared
    • Way and means for Defect Management, Configuration Management, and Risk Management etc. are identified
    • Evaluation & identification – Test, Defect tracking tools are chosen


  • Test Design Phase
    • Test Traceability Matrix and Test coverage is prepared/completed
    • Test Scenarios Identification & Test Case preparation done
    • Test data and Test scripts preparation is done
    • Test case reviews are done and approval is ensured
    • Base lining under Configuration Management is done


  • Test Environment Setup Phase
    • Test Bed installation and configuration is done
    • Network connectivity’s is ensured
    • All the Software/ tools Installation and configuration to test the project are ensured.
    • Coordination with Vendors (if required)


  • Test Automation (if automation is intended for the project) Phase
    • Automation requirement identification
    • Tool Evaluation and Identification
    • Designing or identifying Framework and scripting
    • Script Integration, Review and Approval
    • Base lining under Configuration Management


  • Test Execution and Defect Tracking Phase
    • Execution of Test cases
    • Testing Test Scripts
    • Capture, review and analyze Test Results
    • Raised the defects and tracking for its closure


  • Test Reports and Acceptance Phase
    • Test summary reports is prepared
    • Test Metrics is prepared and process improvements suggestions are made
    • Build release is done to production
    • Receiving acceptance
    • Exit Report is written and sent across to the client/ managers for the acceptance


  • Why do we prefer the Application Testing Life Cycle model?
    Intersoft prefers the Application Test life cycle model because:
    • It ensures High Quality of software testing
    • It goes through rigorous formal phases that ensure that the test assets (test plan, traceability, report etc.) are properly documented and maintained so that the same can be reused (if required) with required modification if application goes through any enhancement phase in future
    • Reduced testing time (actually the better are the base documents the better is the time management because coverage of each and every test activity can be tracked backward as well forward)
    • Consistent test procedures therefore reduces training cost/time
    • It helps in repeating the test iterations therefore improving the application quality
    • It Reduces QA costs in the long run
    • It improves testing productivity
    • It ensures improved product quality as there are little chances of anything go unnoticed because of proper and quality documentation
    • It is highly appreciated by BIG names like CitiFinancial, Inc. and Ripple Communication, Inc. USA


Testing process for existing system:

  • Creating Knowledge Base for Testing:
    The Knowledge Base will be created from
    • Any Functional Specification Documents (Where in business requirements are elaborated with their functional point of view).
    • Any documented documents regarding existing functionality that might contain the raw description of the functionality
    • Study the system that will be used to gather maximum functional information and document the gathered knowledge
    • Any QA/Testing documents assets (Test plan / Test scripts), that would be a result of any kind of testing efforts done in the past


  • Creating Regression Suite:
    • Why is there a need for creating a Regression Suite at all?


The purpose of creating a regression suite is to have a set of key findings that are part and parcel of the application and if in future, the system undergoes any changes/enhancements (including the platform Migration/addition of security features) as per the changing business needs the regression functionality must not be broken.

Agile Methodology: The term “Agile” is self-explanatory. True to its meaning agile methodology provides flexibility in the entire Software Development Life Cycle (SDLC). Testing starts as early as possible and as often as code becomes available and stable enough from module/unit level testing. It is a methodology that promotes continuous iteration of development and testing throughout the software development life cycle of the project. Software is developed and tested in incremental rapid cycles called iterations or sprints. This results in small incremental releases with each release building on previous functionality. Each release is thoroughly tested to ensure software quality is not compromised. The diagram below explains the basic agile process.



Types of Agile Testing Methodologies:

There are two prominent types of Agile Testing Methodologies: Intersoft Data Labs use both of these types depending on the requirements and nature of the project:

Scrum Testing
Scrum as a process is incremental. Each increment is called sprint, where each sprint lasts for four weeks. Prior to the sprint, there is a sprint planning meeting where it is determined what features should be developed and followed in the next sprint. During the sprint, the team meets daily at the same time at a short meeting called a scrum or the daily stand-up meeting. Each sprint in a scrum process lasts for 4 weeks.

The phases of scrum testing agile methodology are:

  • Test Design
    • User stories are used for development of test cases. User stories are short description of functionalities of the System under Test
  • Test Execution
    • Test cases written in the test design phase are executed in this phase of scrum testing
    • Defects are logged in Defect Management tool which are tracked on a daily basis. Defects can be discussed during scrum meeting
    • Defects are retested as soon as it is fixed and deployed for testing
  • Managing
    • Sprint(Cycle) meeting is held at the beginning to decide the scope of sprint, understand the requirements for the sprint , task estimates and assignments
    • Daily 15 minutes meeting are held where participants answer the following three questions:
      • Last 24 hours tasks
      • Plan for next 24 hours tasks
      • What are all obstructions of the tasks?


Extreme Programming (XP)
In this agile testing methodology development and testing happen in parallel. Business Requirements are gathered in terms of user stories. All these stories are stored in a repository. In this agile methodology, releases are based on the shorter cycles called Iterations. Each cycle/iteration lasts for 14 days and includes phases like coding, unit testing and system testing where at each phase some minor or major functionality will be built in the application.

The phases of eXtreme Programming testing methodology are described below:

  • Analysis
    • Capturing of User Stories
    • Prioritize user stories
    • Define the time duration of an iteration
    • Resource planning for both Development and QA teams


  • Design
    • Break down of tasks
    • Test Scenario preparation for each task based on the captured user stories


  • Execution
    • Execution of Manual test scenarios
    • Defect Reporting using a defect management system
    • Retesting the defects as soon as they are fixed and deployed


  • Wrapping
    • Small Releases
    • Regression Testing
    • Demos and reviews
    • Develop new stories based on the need for the next iteration


Testing Services provided by Intersoft Data Labs:

Functional Testing: Functional testing refers to testing the software application or website to validate that it confirms to functional specification and perform all the functions in an expected manner. In the functional testing output from the application, for a given input is analyzed. It involves both positive and negative testing.

Challenges: Functional testing has its own set of challenges, few are mentioned below:

  • Ambiguity in Requirements
  • Creation and selection of test data
  • Poor domain knowledge
  • Lack of Training to resources
  • Uncertainty in code coverage in test cases
  • Lack of skilled resources

Intersoft’s Taking: By virtue of its rich experience in Testing domain, Intersoft has overcome these challenges. The reason of Intersoft’s advantage over other’s in this field can be defined as:

  • Intersoft’s onsite team continuously interacts with business in order to get clarity on requirements
  • Calls are scheduled daily between onsite & offsite teams where, onsite team clarifies the queries of its offsite counterparts
  • Neat and appropriate test data is created by experienced testers
  • Training is provided to the resources on regular basis. Training can be associated with project or it can be on new technologies
  • Intersoft has implemented a robust review mechanism wherein, the test scenarios and test cases are thoroughly reviewed by Managers or Leads
  • Intersoft keeps updating its resource pool by hiring people having knowledge on latest technologies

Integrated Systems Testing: Integration Systems Testing is very important. After independent components of the system are tested, there is a need of testing these components together to confirm that they are interacting properly or not. Test team validates the interaction by calling values from different modules and analyzing their results.

Challenges: Integration testing is quite crucial and it faces different challenges. Few of which described below:

  • Involvement of different other systems
  • Involves lots of permutations and combinations
  • Impact analysis due to changed requirements
  • Sound business knowledge is required

Intersoft’s Taking: Intersoft understands these challenges and appropriate actions are taken on regular basis to update its resources:

  • The testing teams in Intersoft works hand in hand with developers which helps in suggesting measures to avoid bugs and Dev teams helps the test teams by letting them know different implementations.
  • A lot of brain storming is done before finalizing anything important like defining various permutations and combinations and analyzing the impacts on other pieces of code due to change in requirements
  • Resources undergo rigorous trainings before they are on boarded
  • Separate sessions are held to provide business knowledge to the resources
  • Test Environment is created after careful analysis of the requirements

Regression Testing: It is done to test the revised code after bug fixation or enhancements, to see whether the enhancements or revision of code has impacted other areas of the system.

Challenges: Following are the most common challenges faced while performing regression testing:

  • Perspective of Impact analysis. Generally, the impact is analyzed from the perspective of either business angle or technical angle
  • Test cases are not updated to sync with changes in the software
  • Prioritization of tests
  • Tedious and dull nature of these tests and testers loose interest

Intersoft’s Taking: Intersoft Data Lab has devised strategies to take up these challenges and has been successful in getting desired results:

  • While performing regression testing it is the prime concern of the leadership in test teams that both technical and business angles are considered equally as with ignorance of one aspect, the results cannot be relied upon
  • The test leads and managers ensure that test cases are updated in parallel to the implementation of changes by the Dev teams
  • Test cases are designed keeping in view the important functionalities. However, each and every functionality is covered in the test cases, but the test scenarios are prioritized based on the importance of different functionalities
  • Morale of the resources is kept high by awarding them with awards of different categories
  • Occasional parties, outings and picnics are arranged to refresh the team members

Day in Life Testing: The Day in Life Testing can be described as the testing in which the application or software is tested under the same conditions as when the application goes live and real time users would use it. It is performed before the application goes live. Real data, real flows, real transactions and real authorizations are base to such testing.

Challenges: This is quite a challenging testing because of the reasons mentioned below:

  • This type of testing require trained people as they have to replicate the real scenarios and play with real data
  • Sometimes it is not possible to cover the entire flow of application in a single day. Moreover, all business days are not same so this aspect has to be calculated very carefully
  • The application is to be run in unison with other systems

Intersoft’s Taking: Intersoft has undergone such scenarios a number of times hence, can aptly chalk out whatever is required:

  • Testers are trained in line with the real business requirements
  • Plans are prepared in advance after discussing the timelines with clients and the users from client’s side
  • The requirement of other systems being up and running is discussed with the client which helps in deciding the time when such tests can be conducted
  • Testers are specifically asked to mingle with the real users in order to have a close idea of their business routine

Functional Automation Testing: The scope of functional automation testing is the same as of the functional manual testing. The difference is that while in manual functional testing the test cases are run by human resources and in functional automation testing the test cases are run by automation tool, where manual intervention is required for a considerably less time.

Challenges: Automation functional testing makes the life of a tester easy by running the test scripts through a tool, but it too has its own set of challenges:

  • Availability of automation resources
  • Keeping pace with different tools coming to market
  • Unavailability of solutions to different issues in automation testing
  • Constant update of scripts as a result of change requests is a hectic task

Intersoft’s Taking: Intersoft has well qualified automation resources and these challenges are dealt with effectively:

  • The HR department keeps on searching best automation resources
  • Intersoft has a wide knowledge base with information of automation tools and practices
  • Scripts are updated on a regular basis so that there is no pile up of pending work
  • Intersoft provides its resources with the opportunities to do R&D on new tools entering the market hence, Intersoft is always abreast with latest in market

Performance Testing: It is also called non-functional testing. This is done to test non- functional aspects of a software application like the load taking capacity of the application, speed of the application or we can say that it is to test the system’s response and stability which is different from its functionality.

Challenges: There are many challenges in performance testing domain:

  • Performance testers are very expensive to afford for a small company and that’s why most of the times performance testing is outsourced to other vendors
  • Procurement of expensive tool licenses
  • Compatibility issues with performance tools
  • Data Management for iterating users

Intersoft’s Taking: Intersoft Data Labs has been instrumental in dealing with these challenges:

  • Intersoft has its own team of performance testers and don’t rely on outsourcing performance testing
  • Intersoft has handsome repository of performance tools like Load Runner, JMeter etc.
  • Tools are updated with latest versions from time to time
  • Experienced and dedicated resources are entrusted with creation and management of data

Compatibility Testing: Compatibility testing is conducted to evaluate the application’s compatibility with different browsers, hardware, operating systems and databases.

Challenges: The challenges faced under this type of testing are:

  • Availability of different software and hardware
  • Working knowledge of different types of computing elements
  • Mapping all the requirements with different computing elements and creating a traceability matrix

Intersoft’s Taking: By virtue of its presence in the industry for long time, Intersoft can pride on its richness of resources:

  • Intersoft has a vast repository of different types of computing elements
  • Test cases are created and executed to unearth defects arising out because of difference in software and hardware configurations

Usability Testing: Usability testing is to evaluating software by representatives of real users. During this test the observers keep on observing, listening and taking notes of the user actions in order to identify usability issues and collect quantitative and qualitative data.

Challenges: Following are the challenges faced while performing usability testing:

  • Highly skillful observers are required to analyze and collect information from the users
  • Creating a robust test plan

Intersoft’s Taking: Following points go in favor of Intersoft while talking about Usability Testing:

  • Intersoft has such skilled observers who have done such testing over and over many times
  • Test planning is other strong area of Intersoft’s test team

Test Process Consultancy:
Intersoft helps other test organizations to optimize their testing and QA process by evaluating them with the benchmark standards of the software industry. We analyze the organizations test structure and identify the gaps which cause failure of testing and QA processes. After analyzing these gaps, we provide technical and non-technical assistance to the organizations and guiding them towards a more robust testing and QA structure. In the process following are the areas that are generally covered:

  • Analysis of existing QA and Testing teams
  • Methodologies and processes followed by the QA and Test teams
  • Guidance on test plans and strategies
  • Guidance on creation and execution of test scripts

Testing Center of Excellence:
Intersoft’s Testing Center of Excellence helps other test organizations. The framework of Intersoft’s Testing Center of Excellence enables other organizations to establish robust Testing and QA processes which are tried and trusted over a period of time. Intersoft is committed to provide quality products and help other organizations to be able to achieve the same. As we have discussed earlier the importance and role of testing domain in software industry, it is very important for a testing organization to have excellent, dedicated and experienced resources. It is also very important to have access to latest test tools and technologies for increased productivity. Intersoft as a Testing Center of Excellence offers following services:

  • A strong testing methodology with well-defined phases
  • Tested and tried experienced testing resources
  • Specialized industry focused testing solutions
  • A spectrum of automation and test management tools
  • Ready to deploy frameworks to enable you reap results quickly and efficiently
  • End-to-End test management and ownership models
  • Matrices of different types required to trace requirements
  • Various testing templates to have unambiguous direction
  • Effective knowledge management processes

Why Automate Testing?

Test automation can accelerate the testing cycle and promote software quality as well as improve the development process of a software product in many cases. Automating regression tests and other repetitive tasks helps to release QA personnel(s) to expand their quality efforts.

In other words, ‘Automation Testing’ is a fast execution process which accommodates ever-changing software systems and requirements whenever necessary. The use of automation approach and frameworks further increases testing efficiency.

At Intersoft data labs, test automation is assisted by a dedicated automation test team, for automation solutions built on new tools and technologies. We believe in continuous improvement, innovation and enhancement of core competencies across our service offerings. Intersoft testing center of excellence (TCoE) believes in bringing together the strongest expertise, processes and technology.



Enhancement of Speed, Efficiency, Quality and Cost Effectiveness

Software development and testing cycles are time bound processes. Primary objective is to ensure timely release with productive and better quality product.

Automated regression tests ensure the continuous system stability and functionality after changes made on the exiting software product. Test automation leads to shorter development cycles combined with better quality software and thus the benefits of automated testing quickly outgain the initial costs.



Proof of Concepts (POC)

A POC may always help in minimizing the risk of picking up a wrong “Tool of the choice” and hence saving the wastage of Efforts, Time and Cost. Intersoft test automation performs POC with a test automation tool(s) before giving it a go ahead for the “tool of the choice”. Tool should be able to identify all of the objects/ controls/ operations (native/ customized) used in the Application/ system under test.

Motivation and Efficiency

Manual testing can be monotonous, error-prone and therefore become annoying. Test automation relieves tester’s hindrances and allows the test execution without user interaction while assuring repeatability and accuracy. Instead testers can now concentrate on more complex and critical test scenarios. Test candidates for the automation is identified based on the Return On Investment (ROI)



Calculating ROI:

It is equally significant to have a reasonable idea about the ROI, even though there is often no easy formula available to calculate ROI to determine upfront what type of automation, what level of skills, what automation tools will be required.

However, a classic method to calculate ROI is:

Test Automation Cost = Cost of Hardware + Cost of Test Automation Software + Automation suite Development Cost + Automation suite Maintenance Cost + Suite Execution Cost

Manual Testing Cost = Manual Test Development Cost + Manual Test Maintenance Cost + Manual Test Execution Cost
ROI = (Manual Testing Cost – Test Automation Cost)/ Test Automation Cost

Functional Automation Testing Tools:

Automated testing tools are used as a part of the testing phase within the software development lifecycle (SDLC) to automate certain test cases identified tasks, improve testing efficiency, and discover issues that might be difficult to find using manual review alone.

Intersoft functional test automation team has expertise on both, the open source and proprietary tools.
Open Source Tool

  • Selenium
  • Cucumber
  • Soapui
  • Watir
  • Sahi

Proprietary/Commercial Tools

  • QuickTest Pro (QTP)/ Unified Functional Testing (UFT)
  • Microsoft Visual Studio Coded UI
  • IBM Rational Functional Tester
  • Test Complete
  • Silk Test

Non Functional Testing:

Non Functional/ Performance test is the way to determine how fast some aspect of a system performs under the identified workload. Scalability, Reliability and Resource usage are the few key quality attributes determined during the test.

Performance testing supports different purposes:

  • Demonstrate that the system meets performance criteria defined under Service Level Agreement(SLA)
  • Compare two systems to find which performs better
  • Measure system’s performance under a certain workload



Performance Test Types

Intersoft performance testing team conduct several test types in order to guarantee that performance goals are reaching expected results.

Also, different types of test are conducted by the team to ensure the application under test is capable of handling anticipated load over a period of time.

Load Tests primary objective of test is to determine the response time for various time critical transactions and business processes and that they are within documented expectations (or Service Level Agreements – SLAs).

Fail-over Tests verify the redundancy mechanism while application is under load. For example, such testing determines what will happen if multiple web servers are being used under peak anticipated load and one of them dies.

Reliability Tests also known as ‘Endurance Test’, ‘Soak Test’, ‘Long Running Test’ is running a system at high levels of load for prolonged periods of time.

Stress Tests determine the load under which a system fails, and how it fails. This is in contrast to Load Testing, which attempts to simulate anticipated load. It is important to know in advance if a ‘stress’ situation will result in a catastrophic system failure, or if everything just “goes really slow”.

Baseline Tests are tests that determine end to end timing (benchmarking) of various time critical business processes and transactions, while the system is under low load, but with a production sized database.

Network Sensitivity Test tests are tests that set up scenarios of varying types of network activity (traffic, error rates…), and then measure the impact of that traffic on various applications that are bandwidth dependent.

Rendezvous Tests is a test on which we can instruct multiple users to perform tasks simultaneously by using rendezvous points. A rendezvous point creates intense user load on the server and measure server performance under load.

Volume Tests are often most appropriate to Messaging, Batch and Conversion processing type situations. In a Volume Test, there is often no such measure as Response time.

Test Results:

During run and result analysis several statists are collected and further analyzed in order to generate a report, guarantee the major performance goals (Speed, Scalability and Stability) and to identify possible performance problems.

Examples of statistics collected under test are:

  • Transactions Response Times.
  • Hits Per Second
  • Throughput
  • Transaction per second
  • CPU% Utilization
  • Memory Utilization
  • Disk Utilization

Non Functional Testing Tools
Intersoft non-functional testing automation team has expertise on both, the open source and proprietary tools.

Open Source Tool:

  • Apache JMeter
  • OpenSTA
  • Selenium
  • Httperf
  • http_load

Proprietary/Commercial Tools

  • HP Loadrunner
  • VSTS Load Test
  • IBM Rational Performance Test
  • Silk Performer
  • Load Impact