System Test Entry-Exit Criteria
Entry-Exit criteria is a set of conditions that permits a task to perform, or in absence of any of these conditions, the task cannot be performed. Entry Criteria give the prerequisite items that must be completed before testing can begin in the respective phase. Exit Criteria define the items that must be completed before testing can be concluded in the respective phase.
System Integration Test (SIT) phase
The System Integration Test (SIT) integrates deliverables from the Development Streams, builds configurations, develops test applications, and creates test cases incrementally. The purpose is to give feedback to the Development Streams on quality issues found, verifying the integration of functionality across the products.
Entry and Exit criteria
| Entry Criteria | Exit Criteria |
|---|---|
| Demonstration of the scope features shall be provided to the system test team. | All products are integrated into the system successfully, and core functionalities are verified. |
| SIT at the Product testing (PT) level shall be completed. Each devStream shall confirm that integration of the product and its dependencies work without any impact. | No Critical or blocker shall be open. In case some blockers cannot be fixed under some circumstances, an easily implementable workaround shall exist. |
| Each devStream shall provide version notes on delivered builds mentioning implemented features/epics. Also, it shall contain the known bugs with a workaround in case there are any. | Installation and Configuration of individual product shall be possible. Workarounds are acceptable. |
Pre-System Test (Pre-ST) phase
During the pre-ST phase, the different configurations and applications are built up step by step. The Pre-ST phase shall judge the quality level before entering the ST phase. Use cases can be performed if the quality level of the system-wide function is good. System capacity requirements such as alarms, subscriptions, etc are tested.
Entry and Exit criteria
| Entry Criteria | Exit Criteria |
|---|---|
| Software shall be delivered as part of the Baseline. | All the scope system epics are tested at least once (except some specific test cases which can be decided by the test lead in agreement with the release owner). |
| Demonstration of the scope, system epics, and features shall be provided to the system test team. | “Conclusion of test” shall be performed for all the participating test configurations. |
| A reviewed draft version of documents (IP, DoF, etc.) for scope system epics and features shall be provided to the system test team. | No Critical or blocker shall be open. In case some blockers cannot be fixed under some circumstances, an easily implementable workaround shall exist. |
| Each devStream shall ensure basic functionality is not broken and the build is ready to deploy in larger test configurations. | All the test cases required to execute testing are written, reviewed, and in a Ready state for the next test phase |
| Each devStream shall confirm that integration of the product and its dependencies work without any impact. | Required test configuration (application, hardware, third-party software versions, etc.) is ready for formal test execution and reviewed and approved. |
| Each devStream shall provide version notes on delivered builds, mentioning implemented functions and epics. Also, it shall contain the known bugs with a workaround in case there are any. | No project stopper/ Critical bugs without a workaround shall be active. |
System Test (ST) phase
During the pre-ST phase, the test environments, applications, and test cases were developed incrementally and are now ready for the final run. The major difference from pre-ST (where alfa versions are used) is that in the ST phase, the functionality is completed and only beta candidates (BC) are used in tests.
Entry and Exit Criteria
| Entry Criteria | Exit Criteria |
|---|---|
| The “Start of STT” checklist shall be passed with inputs from all the devStream. | No High and above severity bug shall remain unconcluded. |
| All the scope features shall be in a Resolved state. | All the scope epics and features shall be tested completely, and all the planned test cases are executed in all the participating test configurations (except the long term tests). |
| A reviewed draft version of all the user documents shall be available for the ST phase. | The conclusion of tests is performed for all the participating test configurations. |
| Every build shall pass through the quality doors and smoke tests shall be performed in all the participating dev stream test setups to ensure basic functionality is not broken and the build is ready to deploy in larger test configurations. | All the bugs found during the previous phase and this phase shall be validated except for a few medium bugs. Such bugs which are not planned to fix in a particular system version shall be concluded and discussed with product management and dev. stream product owner (System Product Owner shall also be consulted for conclusion wherever required). |
| Each dev stream shall confirm that the product and its dependencies will work without any impact. | A decision shall be made on deferred bugs for entering in release notes documents and a draft version of release notes shall be verified. |
| Each build shall be provided with version notes on delivered epics & features and information regarding known bugs with a workaround and corrected problems in case there are any. | Bugs, where ST and devStream have different opinions, shall be discussed with product management and decision from the meeting shall be recorded in a formal document. |
| In case pre-ST is not executed in a specific project, entry and exit criteria for pre-ST shall also be referred to and confirmed. |
Release Acceptance Test (RAT) phase
RAT is the final Release Acceptance Test. During this period, a subset of tests is re-executed based on the influence of late changes in the builds. The purpose is to check if the core system is intact and not influenced by late changes in the software.
Entry and Exit criteria
| Entry Criteria | Exit Criteria |
|---|---|
| Build shall be installed in some test con-figurations and verified that installation and basic functionality are not broken. | All the scope test cases shall be executed and concluded. |
| Start of RAT checklist shall be passed with all the dev stream inputs. | All the bugs shall be concluded. |
| All the epics are in Resolved state and the Features shall be in a Closed state. | IVA checklist shall be executed and passed. |
| System release notes shall be ready with known problem information. | All the Scope System Requirements/ Epics/ Features shall be ready for the conclusion, the System test release summary shall be prepared and shared with System stakeholders, and a recommendation from ST shall be ready. |