Ensuring Accuracy in Performance Testing

Most of the time applications may face performance issues even after rigorous performance validation. This is primarily because of improper performance test environment setup and model. It is a common issue across the industry that the testing tool might not have performed correctly during the load testing. So it is always a best practice and a mandate to validate that the testing tool simulates the network traffic as expected genuinely and to ensure the test environment is also accurate. Here is an idea, how Queuing theory Laws can be applied in validating the Performance test accuracy and to ensure that the application has a smooth accessibility in production without performance issues.

Little’s Law
The long-term average number of customers in a stable system L is equal to the long-term average effective arrival rate, λ, multiplied by the average time a customer spends in the system, W and it is expressed algebraically,
L = λW


Applying Little’s Law in Performance Testing
The Average number of (virtual) users N in the system (server) at any instance is equal to the product of average throughput X and average response time Z. It is expressed algebraically,

N= X * (Z + R), where R=think time
Demonstration of Little’s Law to ensure Performance Testing
From the results obtained from the performance testing tool, we can find how many actual users have been generated to test the application using Little’s Law. A sample load test done on a sample application with 10 users has obtained following test results.

Average Transactions/sec=1.7, Average transaction response time=0.5 sec, Average Think time=5sec


By Little’s Law, Number of virtual users emulated by the performance testing tool is, N=X*(Z+R) =1.7*(0.5+5)
N=9.35≈10 virtual users have been emulated during load test

If the actual virtual users used in the system is equal to the Little’s Law result, then neither the tool nor the server has undergone any problem. If the Little’s Law result is less than the actual virtual users, then it means remaining users were idle throughout the test.

* It is understood that the throughput data above has been extracted from the tool but it is always preferred and a best practice to use the throughput data from the server.

ARIS FOR SAP NETWEAVER

Business process management (BPM) allows you to continuously adapt your business processes to new business strategies – modeling processes with different users performing different roles and tasks, and optimizing communication between process owners and IT experts. SAP and IDS Scheer offer a comprehensive BPM solution – SAP NetWeaver and ARIS for SAP NetWeaver. This joint application provides essential elements of a closed-loop BPM solution, from design and configuration, to implementation and execution, to evaluation of the overall process.
There are lots of BPM tools available in market, then why should we choose ARIS for modeling our business processes. Here are some of the features and benefits of ARIS which make this tool different from others.

• ARIS Platform is Highly Scalable and hence complete solution for the Entire Business Process Management – From Web Based Description, analysis, and optimization of Business processes and software engineering to SAP Netweaver integration and continuous process controlling.
• ARIS is not just for SAP NetWeaver. ARIS can be used for all SAP initiatives — to design, implement, and monitor SAP, non-SAP and manual processes in an organization.
• The system architecture of the ARIS Platform allows globally active companies to set up distributed scenarios for designing, analyzing and optimizing process, IT and software architectures.
• Business Process Management with SAP NetWeaver and ARIS for SAP NetWeaver provides procedure models, methods, technologies and reference content for modeling, configuring, executing and monitoring these business processes.
• With ARIS Implementation Platform, gap between business and IT is closed. ARIS for SAP Netweaver helps design the process architecture for SAP Solutions that have been optimally adapted to the company’s business process.
• Adds functions to SAP NetWeaver for graphically modeling processes at various levels. At the highest level (Process Architecture Model), the process architecture of a company is built from a purely business perspective, that is, without technical details.
• 
  

User-Friendly Functionality:

ARIS Business Architect and ARIS Business Designer Web Based Solution enable shared, company wide modeling, analysis, and optimization of business processes, as well as IT Architecture set up via the Internet. With its user-friendly functionality that allows even unskilled occasional users to perform modeling. ARIS Business designer quickly paves the path to professional BPM.

Synchronization between ARIS for SAP Netweaver and SAP Solution Manager:

You can use ARIS for SAP NetWeaver as a modeling environment to import predefined process models as reference content from SAP Solution Manager. You can enhance these models and synchronize them with SAP Solution Manager.
Process models and other configuration elements can be exchanged (synchronized) between ARIS for SAP NetWeaver and SAP Solution Manager. In addition to scope information, structural information is also transferred.
The SAP Solution Manager provides you with the relevant SAP reference models. You can create an implementation project using ARIS for SAP NetWeaver and then synchronize it using the SAP Solution Manager. This enables you, for example, to make SAP reference models available in ARIS for SAP NetWeaver, and adapt and enhance them as required. Finally, you can make the implementation project available again in the SAP Solution Manager, where you can then configure the processes and adapt them to your specific system landscape. The SAP Solution Manager also provides you with an extensive range of functions for monitoring your solutions.

Upload of ARIS Processes to SAP XI:

With the ARIS BPEL export, ARIS BPEL models can be transferred to SAP XI. The elements to be exported must be exported from ARIS Business architect for SAP Netweaver. ARIS creates ZIP files, which can be imported into SAP XI.

Interfaces and development of Add-ons:

ARIS UML Designer is included in the Implementation Platform as a tool for;
Developing add-on functions and interfaces during the implementation. With the addition of classic UML methodology to ARIS, a consistent and uniform visual representation of the requirements of the business world can be obtained. The UML models created in ARIS can be sent via XML to other development tools integrated in Eclipse, which are then available as source code or as J2EE software components from ARIS. This means that software developers can work with (UML) data from ARIS directly in the development environment.

Globally Distributed Process Design with ARIS:

The system architecture of the ARIS platform allows globally active companies to set up distributed scenarios for designing, analyzing and optimizing process, IT and software architectures.
Web-based products such as ARIS Business Architect, ARIS Business Designer and ARIS UML Designer access a centrally set up and managed ARIS Business Server from locations around the world via three-tier architecture. These products are designed for use beyond firewall limits, with a very low bandwidth (e.g. ISDN).

Next Generation of BPM: ARIS embedded in SAP NetWeaver:

In the next SAP NetweaverTM release, the design, modeling and model based configuration will take place in a technically integrated solution, a unified modeling environment as part of the ESR in SAP NetWeaver. There, on the basis of a unified metamodel, users will be able to perform various different role-specific tasks in a unified modeling environment.
SAP AG and IDS Scheer AG leverage the strength of their technologies together to cover the whole business process lifecycle. SAP NetWeaver today offers a comprehensive BPM solution that will be enhanced in future releases providing a unified modeling environment and a powerful combination of Business Process Management and Business Activity Monitoring.

ARIS for NetWeaver contains a consistent description of the process architecture-from enterprise process models to implementation of the processes by SAP Solution Manager, the integration of executable processes in SAP XI, and the applications with SAP Business Workflow.

Some of the key benefits of using ARIS at a glance would be :-

• Get a clear understanding of your process architecture as a starting point for SAP ESA.
• Alignment of business, configuration and service process in one common repository.
• Intuitive and user friendly user interface
• Decentralized Design For centralized optimization
• Available 24 hours a day, 7 days a week and in all locations due to the Web Based Front end.
• Proven , extendable methods for various areas of application
• Highly scalable architecture.

Test Consulting: How to Improve a Quality Assurance Area

Is your client having difficulty to measure QA performance? Is your client undecided on what test strategy to follow for new application implementation? Is your client looking for testing tool recommendations or need to improve the usage of the existing ones?

Hexaware has a dedicated Strategic Consulting practice where testing experts are involved in providing test consulting services to clients. During the consulting engagements, Hexaware assesses and evaluates the Approach, People and Technology and fill in the gaps by bringing in our domain experience, best practices, frameworks, tools experience, etc. The consulting services also include tools selection, tools optimization and TCoE Creation and/or optimization.

If you want to learn more about test strategy, the next information will help you to execute a test strategy engagement.

Test Strategy Approach
The first step is to identify the problem(s) that the client is facing and define the strategy objectives. After that, I recommend to follow this approach to execute a test strategy project:

Assessment Areas
As part of the information gathering phase, we leverage its proprietary ATPTM (Approach, People, and Technology) methodology to focus on the right areas and meet the strategy objectives. Hexaware’s APT™ Methodology is the foundation of all of our QA service offerings, below is the description of each component:
• The Approach component is designed to lay the foundation for the processes that each client use as part of testing
• People is the component of the IT organization with focus on testing, Hexaware analyze the groups, roles and responsibilities involved in QA and testing
• The Technology component includes the use of QA and automation testing tools for efficiency to optimize technology and lower costs.


At the end of the project, we provide to the client the following component:
•Current State: An analysis of the current state with regard to the testing objectives
•Gaps: The gaps found between the current state and the best practices and the desired state
•Recommendations: Our recommendations to close the gaps and meet the organization objectives
•Implementation Road Map: A recommended path to follow in order to implement the recommendations.
As a result of the analysis phase, we show to our clients the current state in a quantitative graph. This graph evaluates all relevant aspects of an IT organization and prioritizes each category according to the testing objectives. One example of this graph is showed below.



This was an assessment provided as part of a test strategy we created for a leading bank in Mexico for a T24 product implementation. The benefits showed in this strategy was to reduce testing cycles by 30%, automate at least 50% of the manual test cases and have a defect free implementation using robust and repeatable testing processes.
Other examples of metrics commonly used as part of the strategy objectives are the increment of automation coverage by 30%, increase productivity by 25%, reduce overall testing cost by 15%, etc.

A test consulting practice is an area full of innovation, industry best practices and shared experiences. Now with Hexaware blogs all of us will be able to formally share our experience and our colleagues can leverage them for future assignments.

To know More: Visit Quality Assurance Area 

Good Attributes to be kept in mind for designing Automated Test Cases

Programming remains the biggest & most critical component of test case automation. Hence designing & coding of test cases is extremely important, for their execution and maintenance to be effective.

Some fundamental attributes of good automated test cases which can be followed,

• Simple
• Modular
• Robust
• Reusability
• Maintainable
• Documented
• Independant

1) Simplicity: The test case should have a single objective. Multi-objective test cases are difficult to understand and design. There should not be more than 10-15 test steps per test case, May be depending on the process steps may increase. However 10 to 15 steps note a good clarity test case. Multipurpose test cases are likely to break or give misleading results. If the execution of a complex test leads to a system failure, it is difficult to isolate the cause of the failure.

2) Modularity: Each test case should have a setup and cleanup phase before and after the execution test steps, respectively. The setup phase ensures that the initial conditions are met before the start of the test steps. Similarly, the cleanup phase puts the system back in the initial state, that is, the state prior to setup. Each test step should be small and precise. However we can’t expect all test cases to have set up and cleanup process, it may vary accordingly.. The test steps are building blocks from reusable libraries that are put together to form multi-step test cases.

3) Robustness and Reliability: A test case verdict (pass or fail) should be assigned in such a way that it should be unambiguous and understandable. Robust test cases can ignore trivial failures such as one pixel mismatch in a graphical display. Care should be taken so that false test results are minimized. The test cases must have built-in mechanisms to detect and recover from errors. For example, a test case need not wait indefinitely if the software under test has crashed. Rather, it can wait for a while and terminate an indefinite wait by using a timer mechanism.

4) Reusability: The test steps are built to be configurable, that is, variables should not be hard coded. They can take values from a single configurable file(Data Tables). Attention should be given while coding test steps to ensure that a single global variable is used, instead of multiple, decentralized, hard-coded variables. Test steps are made as independent of test environments as possible. The automated test cases are categorized into different groups so that subsets of test steps and test cases can be extracted to be reused for other platforms and/or configurations. Finally, in GUI automation hard-coded screen locations must be avoided.

5) Maintainability: Any changes to the software under test will have an impact on the automated test cases and may require necessary changes to be done to the affected test cases. Therefore, it is required to conduct an assessment of the test cases that need to be modified before an approval of the project to change the system. The test suite should be organized and categorized in such a way that the affected test cases are easily identified. If a particular test case is data driven, it is recommended that the input test data be stored separately from the test case and accessed by the test procedure as needed. The test cases must comply with coding standard formats. Finally, all the test cases should be controlled with a version control system.

6) Documented: The test cases and the test steps must be well documented. Each test case gets a unique identifier, and the test purpose is clear and understandable. Author name, date of creation, and the last time it was modified must be documented. There should be traceability matrix to the features and requirements being checked by the test case. The situation under which the test case cannot be used is clearly described. The environment requirements are clearly stated with the source of input test data (if applicable). Finally, the result, that is, pass or fail, evaluation criteria are clearly described.

7) Independent and Self-sufficient: Each test case is designed as a cohesive entity, and test cases should be largely independent of each other. Each test case consists of test steps, which are naturally linked together. The predecessor and successor of a test step within a test case should be clearly understood.

Know More About: Automated Test Cases

Manual Testing (Vs) Automation Testing

As the world is moving toward a new era in Technology and business with respect to Testing, we will just have a walkthrough on Manual Testing and Automation testing.

1. Manual Testing is Boring
• No one wants to keep filling the same forms
• There is nothing new to learn when one tests manually
• People tend to neglect running manual tests
• No one maintains a list of the tests required to be run if they are manual tests

Automated tests on the other hand are Code
• They are fun and challenging to write
• One has to carefully think of design for reusability and coverage
• They require analytical and reasoning skills
• They represent contribution that is usable in the future

2. Manual Testing is not reusable
• The effort required is the same each time
• One cannot reuse a Manual Test

Automated Test are completely reusable
• Once written the Automated Tests form a part of the codebase
• They can be reused without any additional effort for the lifetime of the Project

3. Manual Testing has a high risk of missing out on something
• Each time a developer runs manual tests it is likely he will miss out on an important test case
• New developers may have no clue about the battery of tests to be run
Automated Tests have zero risk of missing out a pre-decided test
• Once a Test becomes a part of Continuous Integration – it will run without someone having to remember to run it

4. Manual Tests do not provide a safety-net
• Manual tests are run post-facto and hence only drive bug-patching
Automated Tests provide a safety-net for refactoring / additions
• Even New developers who have never touched the code can be confident about making changes

5. Manual Tests have no training value
• Manual Test are a mechanical process and does not have defined training values
Automated Tests act as documentation
• Reading a set of Unit Tests clarifies the purpose of a codebase
• They provide a clear contract and define the requirement
• They provide visibility into different use cases and expected results
• A new developer can understand a piece of code much more by looking at Unit Tests than by looking at the code
• Unit Tests define the expected behavior of the code

Read More: About Manual & Automation Testing