Automation Tool Selection Recommendation

• Overview
• Information Gathering
• Tools and Vendors
• Evaluation Criteria
• Tools Evaluation
• Matrix
• Conclusion
• Overview

“Automated Testing” means automating the manual testing process currently in use. This requires that a formalized “manual testing process” currently exists in the company or organization. Minimally, such a process includes:

–        Detailed test cases, including predictable “expected results”, which have been developed from Business Functional Specifications and Design documentation.

–        A standalone Test Environment, including a Test Database that is restorable to a known constant, such that the test cases are able to be repeated each time there are modifications made to the application.

Information Gathering

Following are sample questions asked to tester who have been using some the testing tools:

How long have you been using this tool and are you basically happy with it?

How many copies/licenses do you have and what hardware and software platforms are you using?

How did you evaluate and decide on this tool and which other tools did you consider before purchasing this tool?

How does the tool perform and are there any bottlenecks?

What is your impression of the vendor (commercial professionalism, on-going level of support, documentation and training)?

Tools and Vendors

• Robot – Rational Software
• WinRunner 7 – Mercury
• QA Run 4.7 – Compuware
• Visual Test – Rational Software
• Silk Test – Segue
• QA Wizard – Seapine Software

Tools Overview

Robot – Rational Software

–        IBM Rational Robot v2003 automates regression, functional and configuration testing for e-commerce, client/server and ERP Applications. It’s used to test applications constructed in a wide variety of IDEs and languages, and ships with IBM Rational TestManager. Rational TestManager provides desktop management of all testing activities for all types of testing.

WinRunner 7 – Mercury

–        Mercury WinRunner is a powerful tool for enterprise wide functional and regression testing.

–        WinRunner captures, verifies, and replays user interactions automatically to identify defects and ensure that business processes work flawlessly upon deployment and remain reliable.

–        WinRunner allows you to reduce testing time by automating repetitive tasks and optimize testing efforts by covering diverse environments with a single testing tool.

QA Run 4.7 – Compuware

–        With QA Run, programmers get the automation capabilities they need to quickly and productively create and execute test scripts, verify tests and analyze test results.

–        Uses an object-oriented approach to automate test script generation, which can significantly increase the accuracy of testing in the time you have available.

Visual Test 6.5 – Rational Software

–        Based on the BASIC language and used to simulate user actions on a User Interface.

–        Is a powerful language providing support for pointers, remote procedure calls, working with advanced data types such as linked lists, open-ended hash tables, callback functions, and much more.

–        Is a host of utilities for querying an application to determine how to access it with Visual Test, screen capture/comparison, script executor, and scenario recorder.

Silk Test – Segue

–        Is an automated tool for testing the functionality of enterprise applications in any environment.

–        Designed for ease of use, Silk Test includes a host of productivity-boosting features that let both novice and expert users create functional tests quickly, execute them automatically and analyze results accurately.

–        In addition to validating the full functionality of an application prior to its initial release, users can easily evaluate the impact of new enhancements on existing functionality by simply reusing existing test casts.

QA Wizard – Seapine Software

–        Completely automates the functional regression testing of your applications and Web sites.

–        It’s an intelligent object-based solution that provides data-driven testing support for multiple data sources.

–        Uses scripting language that includes all of the features of a modern structured language, including flow control, subroutines, constants, conditionals, variables, assignment statements, functions, and more.

Evaluation Criteria

Record and Playback         Object Mapping
Web Testing Object              Identity Tool
Environment Support        Extensible Language
Cost                                            Integration
Ease of Use                             Image Testing
Database Tests                     Test/Error Recovery
Data Functions                    Object Tests
Support

3 = Basic  2 = Good  1 = Excellent

Tool Selection Recommendation

Tool evaluation and selection is a project in its own right.

It can take between 2 and 6 weeks. It will need team members, a budget, goals and timescales.
There will also be people issues i.e. “politics”.

Start by looking at your current situation
– Identify your problems
– Explore alternative solutions
– Realistic expectations from tool solutions
– Are you ready for tools?

Make a business case for the tool

–What are your current and future manual testing costs?
–What are initial and future automated testing costs?
–What return will you get on investment and when?

Identify candidate tools

– Identify constraints (economic, environmental, commercial, quality, political)
– Classify tool features into mandatory & desirable
– Evaluate features by asking questions to tool vendors
– Investigate tool experience by asking questions to other tool users Plan and schedule in-house demonstration by vendors
– Make the decision

Choose a test tool that best fits the testing requirements of your organization or company.

An “Automated Testing Handbook” is available from the Software Testing Institute (www.ondaweb.com/sti), which covers all of the major considerations involved in choosing the right test tool for your purposes.

Thanks For Reading This Blog. You Know More About: Automation Tool

HexaFLOW GRABBER

Introduction: In the third generation of ERP Applications and Web Based Application are the proficient solution in the Automation arena, the automation process is being accelerated by the BPT Accelerator Framework, which speeds up test design phase since it acts as a ready reckoner. However a gap analysis on the existing Accelerator Framework to make the Accelerator more efficient was required.
Being the pioneers as the BPT Accelerator solution providers across the globe for almost a decade, there was a need to perform a gap analysis, the gap analysis highlights, that the effort distributed on the BPT Accelerator workflow is still biased towards the Test Design Phase and can be fine-tuned.

2.Abstract:In the new era of automation, the focuses of the organizations are mainly to minimize script maintenance effort thereby increasing the Return on Investment (ROI) and to look for a framework where a Functional Subject Matter Experts (SMEs) or Business Analysts will able to build the Automation Test Scripts. This paper promotes an optimal approach to accelerate the automation, in order to leverage the Efforts spent specifically on the test design Phase. Through a simplified and swift test design process.

With a groundbreaking Solution Via a new framework which will transcribe the manual interactions to Automation script. The Solution is intended to be adopted for various ERP applications, Windows Application and Web-Based Applications and to reduce the gap that exists between the Functional Knowledge and Technical feasibility.

3.Challenges Faced:The BPT Accelerator framework provides an exclusive and easy solution for the automation testing, leaving behind many areas of improvements and main challenges to be address by an evolution of a concept or framework.

There are many challenges that have to be addressed on the solution that has to be provided to increase the efficiency of the BPT Accelerators in a drastic manner.
3.1High dependency on Manual mapping of the correct components against the Business process flow:
BPT Accelerators has a ready to go or a Jump start kit which covers nearly 40% of the Test Design Phase in any Automation Project starting from scratch. The Accelerators comes along with three solutions that lay a framework on top of the Traditional automation to drive the complete Test Design Phase. By using the highly reusable Jump Start Kit and the Accelerator solutions the Test Design Phase could be automated. However there is a major dependency on the Accelerator Experts to possess monotonous knowledge on the Business Components namely sequencing the Business Components as per the Business Flow, Parameterization and creation of data table for iterative execution.
3.2Non-existence of Application Independent solution:
The non-availability on single acceleration solution that will handle business flows across various ERPs and web applications simultaneously in spite of the BPT Accelerator which could leverage the Automation script development for multiple applications, and this indeed creates a necessity for a solution which could be used in designing Test Scripts across different Applications (ERP and Web) for Test Script Design using the BPT Framework.
3.3Need of a One Stop package Solution
An End to End Acceleration solution which will package all aspects of automations Testing and render quality at its best by performing Verification, validation and synchronization of the scripts to serve the customer’s Requirements’ is the need of the hour.
3.4 JIT (Just In Time)Considering the role played by the Automation scripting inside the testing delivery, there is high dependency of a Freezing of Requirements and stable instance, which makes positioning of the automation critical and thus calls swift and speedy delivery .Customers expectations are also more toward the above call ensuring the same quality.
3.5Erroneous Test Design-The Test design involves critical decision making of replication of the business flow into Automation flow, human errors, assumptions and misconceptions often proves very costly in Time, effort and Money .This pushes the industry to evolve with a solution that self handles and transcripts the Business flow all by itself and reduces the human intervention even during the Test design phase
3.6 Application & Version Compatibilities:Though the challenges are on the expectation of a solution, there would be an additional challenge that need to be taken into account is the solution need to be compatible to any application and any versions.
4.Proposed Solution(s):The following section will throw more light over a solution proposed and the approach that could be adopted for finding a remedy to various challenges that are faced by the current acceleration industry
4.1Application & Version Compatibilities-The solution provided for the challenges faced should be capable for capturing the flow on the application (ERP applications and the Web Based Applications). Thus a solution requires a medium that could capture the events or steps replicated in GUI for ERP and Web Based Applications.
4.1.1HP Quick Test Professional-On the analysis over the applications under test a tool that can capture all the steps happening on the GUI and interpret to an expected BPT Test. The Best solution for capturing the test step could be providing by “HP Quick Test Professional”.
•The solution would be apt to serve the purpose of capturing the manual flow, since the BPT Accelerator has a major prerequisite as HP Quick Test Professional. This way we can have a feature added to the solution to understand manual flow performed with nil investment of the first and major stage of the mission.
•The solution can be used to capture the flow on any version and any ERP applications since the required Add-ins would be installed in the QTP for recognizing the Test Objects in the AUT for the execution of the designed automation scripts
4.2Manual Interventions-As the initial challenge of understanding and capturing the business flow could be handled through the HP QTP the functional Automation tool. The next we are in need of a solution to override the manual intervention on
•Manual Execution to confirm
o whether application is stable
o whether the data and flow is valid
• Creation of the Business Process Test in HP Quality Center
• Stitching the Business Components in the BPT as per the business flow
• Synchronization and Proof of Execution
These challenges need to be addressed on the venture of the solution.
4.2.1HP Quality Center Automation:-As the first challenge in the manual intervention could be handled far better using the HP QuickTest Professional in our first solution, let us concentrate on getting a solution that can interpret the captured flow and understand which component has to be placed and in the sequence.
The solution for eliminating the manual intervention on the second and the third point could be accomplished by automating the HP Quality Center to create the Business Process Test and to stitch the business components in sequence; here the interpretation plays a major role. Thus by the interpretation the challenges of stitching the business components could be addressed.

4.3Synchronization and Proof of Execution:
The key component for a testing either a manual or automation would be the Proof of execution. This will provide an evidence of both capturing the defect and also to confirm that the application is working as expected. There could be a scalability included to the solution. Inclusion of the Capture Screen Business Component in the manual flow as and when required with the screen name, during the manual flow through the control bars having the controls.
The synchronization is a key solution that is really expected in an automation script. The inclusion of the feature in the solution would be more added edge. The Business Component for the synchronization could be added in the flow along with the specific time taken for the screen to be loaded or an action to be completed.
4.4Tangible ROI – Business Benefits:
An analysis has be done on the current system and the system where the Flow Transcriber could be included in the workflow and the script can be designed and the ROI study also done on the same. Considering the effort distribution on a regular BPT Accelerator Project and the Project once the Flow Transcriber is included, there would be a drastic change in the effort spent on the Test Design Phase of the overall BPT Accelerator workflow. The table and the graph below will state the difference in the effort spent to design a medium complexity scenario in real time.
There would be a drastic reduction on the Test Design Phase of the Workflow of the BPT Accelerators using this solution in the Business in real time.

Accelerator Activities – Flow Transcriber Implementation Using Accelerator(Efforts in Minutes) Using Flow Transcriber (Efforts in Minutes)
Review of Manual Test case

- Data Identification
- Manual Execution 30 30
Test Design Phase

- Creation of Business Process Test
- Stitching of Business Process Test using Pre-defined
- Parameterization of the Components in the Workflow
- Creation of the Data Table 170 60
Consolidation 15 15
Execution
- Debug and Execute in test plan
- Execute in test lab 45 45
Report Log validation 10 10

Table: Accelerator Effort Distribution Vs Flow Transcriber Implemented Accelerator Effort Distribution

Graph: Comparative Effort Distribution Graph – Test Design Phase
5 Business Benefit
• Extensive Reduction of Test Design Phase.
• Provision of Ready to Run scripts in negligible time.
• Reduction of Manual Intervention in the BPT Scripts Creation Phase.
• Instant parameterization and Datatable creation.
• Provision to include synchronization and screenshot in midst of the Design Phase through User Interactive controls.
• Cuts across boundaries to support various ERPs and Web Applications
o A Single solution for various ERPs and Web Application

Figure: Compatibility of HFG across various Applications
6 Future Direction / Long-Term Focus
• Inclusion of UI Scanner – In the current feature of the concept ,the UI Scanner Tool has not been included and in the Long-Term Focus the UI Scanner is being planned to be included in the flow.

• Verification & Validations – A Testing of an application gets fulfilled only when the Verification and Validations are provided. In the Flow Grabber there is a good scope and the enhancement to include the same.

7 Conclusion
A complete solution for the Test Design Phase of any accelerator project irrespective of any ERP applications is the necessity for optimizing the accelerator, this solution is being taken as the requirement and an analysis and development is being done to evolve the solution into a tool named Hexa Flow Transcriber in real time. It has an expectation to migrate into a complete package for the Test Design Phase in an accelerator Project.

Tagged as: BPT Accelerators, ERP applications, HP Quick Test Professional, HP QuickTest, QTP

Some myths and challenges faced by BPT Methodology

Is designing test for an entire business scenario a time-consuming process?
Business Process (BP) – a BP should be well-documented written at a level that can be script. The BP should be understandable by people not familiar with business. It should be sufficiently detailed. If possible we can have Mercury Screen Recordings (MSR), Navigational flow etc.

Data – The input data required for executing the BP should be reusable. If it is not reusable, sufficient set of data should provided for automating the BP. For most of the finance modules data are not reusable.

Business Process should be complete, correct and accurate. Verifying BP completeness and correctness takes a lot of time, better to start off with these steps in mind as verification time should be used to ensure the quality of the scripts instead of verifying the quality of the BP. Don’t assume anything, design the test cases so that any person can run it or execute it. Design the test cases so that any person can understand what steps are to be validated.

Generally speaking, the more user involvement a BP requires the less desirable it becomes for automation. Though scripts build during development process are executed successfully, they are liable to fail during regression execution. The common errors and solutions can be summarized as:

Wrong Data is being used: Most of the time the script fails due to a mismatch between the data displayed in application and the data stored in Data Table. The solution for this is to change the incorrect value in the Data Table.

Script Flow has changed due to a new build or a new data: if this is the case we need to modify the existing script to add/remove the necessary steps. This is not acceptable because we are automating scripts with the same process. This may represent delays in our delivery date.

The script is running with a different user: The solution is to use the same user that we were using during the development phase or give the new user the same privileges that the user utilized during development had.

Data is burned: If this is the case we need to get or create new data in order to execute the script.

Are your scripts backward compatible i.e., can they run in lower versions?
Scripts execution in lower version of QTP: The other major problem faced in scripts are not backward compatable.ie.., scripts build in upgraded version of quick test professional like 9.2,cannot be executed in 9.0 or lower versions. This problem can be overcome by using BCIE (Bushiness component Import and Export) tool. Using this tool we can import the reusable component from higher version of QC and export it to lower version of QC. Scripts are then compatible with lower versions.

Does Subject Matter Experts require technical knowledge?
Even though creating Business Processes Test Script using accelerators might seem like a simple task, there are multiple factors and caveats that need to be taken into account.
1.The initial Components base might prove complete for some BP’s, but there will surely be a need to create new components in order to complete all the scripts.
2.Though the UI Scanner automatically generates the required components, it might be required to manually modify the QTP code, or even to manually create a whole component.
3.VB Script knowledge is welcomed, since sometimes, there might even be the need to create new functions in the libraries, or modify the existing ones. It is always better to have a technical architect in a project dedicated to handle Libraries, tool installation, debug and trouble shooting. A technical architect should have considerable knowledge of library architecture and descriptive programming understanding, should understand relationship between Accelerators libraries and Business Process Testing. The responsibilities of the technical architect would be to provide the team with wrappers and

Are accelerators only for ERP applications?
Accelerator projects are not restricted to only ERP applications like SAP, PeopleSoft,Seibel and Oracle. It can be customized for application like Metavance, web based application like SAP web portal by our technically expertise development team.