Skip to main content

Performance Testing

This guides includes general information about performance testing and a brief description of different types of performance testing.

Performance is a quality (non-functional) attribute that should be defined in the requirement specification. Other quality attributes that may be important to test are usability, maintainability, efficiency, and reliability.

Intended for​

Software engineers, scrum masters, and test engineers.

About performance testing​

Performance testing is similar to requirements-based testing, i.e., it evaluates the requirements at different test levels (mainly at the component level, which is therefore considered part of component testing). Hence, an essential input to performance testing is the performance requirements. The expected output cannot be stated when designing the test cases without requirements. Different types of performance testing exist, and in IEC 61508, performance testing is divided into stress testing, response timings, memory constraints, and performance requirements.

Performance requirements​

The purpose of testing against performance requirements is to design test cases that demonstrate whether the requirements are fulfilled or not. Examples of performance requirements are:

  • A download to the controller with a small project should not take more than x minutes in 95% of the cases.
  • In a standard workload, CPU usage should be less than 50%.
  • Online upgrade for a normal-size project should not take more than x minutes (excluding user communication) in 95% of the cases.

Stress testing​

The purpose of stress testing is to break the system, i.e., find circumstances under which the system will crash. While determining the system's limits, testing provides evidence that the system will work under normal conditions. Stress testing reveals faults like wasted memory space, errors in timing sequence, and deadlocks. It is preferable to use a tool when stress testing. A load generator can simulate many users of the system, many processes, much data input to the system, and so forth.

Response timings and memory constraints​

Response timings are how long it takes the system to respond to a request for information. A user, another system, or another process could initiate the request. Memory constraints define how much memory is allowed to be used. The response timings and memory constraints should be tested for average and worst-case conditions. Examples of response timings are:

  • The time to download a project to the controller.
  • The time to open a new project.
  • The time it takes the safety module to answer a specific request from the program unit.
Owner: Software Development Team