How-to Create Test Plans and Test Suites

The purpose of this guide is to provide hands-on support to the roles involved in setting up a test plan and test suites in Azure DevOps (ADO), demonstrating the product's test coverage.

Intended for

Release owners, product owners, scrum masters, software engineers, hardware engineers, and test engineers

Activities

Create a test plan

Note: Before creating a test plan, it is important to define your backlog of requirements to facilitate the creation of requirement based suites. For these suites, you can add one or more clauses to filter work items using queries.

1. Create a test plan for creating and maintaining test cases:

   • System package release testcase pool test plan. The name of the test plan should contain "System package release name_Test team name_Testcase Pool test plan" for easy identification, e.g.:

     • DCS SPR 6.0_STT_Testcase Pool test plan
     • DCS SPR 6.0_PTT_Testcase Pool test plan

   • Product release testcase pool test plan. The name of the test plan should contain "Product release name_Test team name_Testcase Pool test plan" for easy identification, e.g.:

     • DCS PR 2.4 SP2_PTT_Testcase Pool test plan

   Note: Test cases are stored in this test plan, and should not be used for test execution.

2. For each release, create another test plan for execution:

   • System package release test execution test plan. The name of the test plan should contain "System package release name_Test team name_Test Execution test plan" for easy identification, e.g.:

     • DCS SPR 6.0_STT_Test Execution test plan
     • DCS SPR 6.0_PTT_Test Execution test plan

   • Product release test execution test plan. The name of the test plan should contain "Product release name_Test team name_Test Execution test plan" for easy identification, e.g.:

   • DCS PR 2.4 SP2_PTT_Test Execution test plan

Note: Test cases are executed in this test plan, which shouldn’t be used for storing or maintaining test cases.

Note The STT team’s test plan should be distinct from the agile team’s test plan. It is recommended to create a unified test plan for each release that encompasses all agile teams.

Provide the following details when creating a test plan in ADO:

• Name. Enter the name of the test plan.
• Area path. Enter the area path of the test plan.
• Iteration. Enter the iteration of the test plan.

Enter the test plan details in ADO and click the “Create” button to create the new test plan.

Create a test suite

There are three types of test suites available in ADO - static test suites, requirement based test suites, and query based test suites. The three sections below describe how to create test suite of the different types.

Create a static test suite

1. Click on the purple test plan icon in the left column and enter the name of the test plan in the "Filter by title" field. The test plan will then appear.

   

2. Click on the test plan to open it.

   

3. Right-click the root test suite.

   

   Note: The root test suite is created automatically once the test plan is created. The name of the root test suite is the same as the name of the test plan. The root test suite ID number is the test plan ID number +1, e.g.:

   • Name of the test plan: QMS Test suite Creation (ID:321984).
   • Name of the root test suite: QMS Test suite Creation (ID:321985).

   

   

4. Select "Root Test Suite" > "New Suite" and click on "Static suite" to create a static test suite.

   

5. The name of the just created static test suite is "New Suite".

   

6. Select the new suite and click on "More options", or right-click on the new suite, and then select "Rename" to change the name of the test suite.

   

7. Change the name manually from "New Suite" to "Static test suite".

   

8. Create any type of test suite (static, requirement based, or query based) under the static test suite based on testing expectations.

   

9. In a static test suite, add a test case by clicking "New Test Case" and select "Add existing test cases" or "Add test cases using grid".

   

Example when an existing test case is added to a static test suite:

Create a requirement based test suite

1. Click on the purple test plan icon in the left column to open the test plan.

   

2. Select "Test suite" > "New Suite" > "Requirement based suite" to create a requirement based test suite.

   

3. Create a query based on expectation.

   

4. Click "Run query" to list down the work item, and then click "Create suite".

   

5. A new requirement based test suite is created, e.g.:

   

Create a query based test suite

1. Click on the purple test plan icon in the left column to open the test plan.

   

2. Select "Test suite" > "New Suite" > "Query based suite" to create a query based test suite.

   

3. Create a query based on expectation.

   

4. Click "Run query" to list down the work item, and then click "Create suite".

   

5. A new query based test suite is created, e.g.:

   

Plan the test cases

1. Add test cases from existing test plan

   Test cases are added to the test suites based on system epics, epics, features, and bugs. Add existing test cases to a test suite with the following actions:

   • Select a test suite. From the "New Test Case" menu, select "Add existing test cases".
   • Add search clauses, as needed, and then select "Run query".
   • When you find the test cases you want, highlight them and select "Add test cases".

2. Copy the test cases to maintain multiple revisions

   Existing test cases can be reused in different test plans/suites, an option is also to copy the test cases. A copy creates a new baseline, changes to these new test cases don’t affect your previous test plans.

View report

The progress report consists of the following sections:

• Summary. This section provides a consolidated view of the selected test plans.

• Outcome trend. This graph renders a daily snapshot which provides an execution and status trend line. It can show data for 14 days (default), 30 days, or a custom range of choice.

• Details. This section enables drill down by each test plan to get a summary of each test suite. It also allows navigation to a test plan or test suite by double-clicking on it.

  

Details

About test suite types

Static test suites are used either as containers to group other test suites or to group a specific set of test cases. They can contain any type of test suite - just like folders. For instance, a test automation static test suite is used to group a specific set of automated test cases.

Drag test suites to group them in a test plan. Drag and drop tests to reorder them.

Requirement based test suites are the simplest and most traceable, as they include all the test cases associated with a specific requirement. They can be created for a single requirement at a time.

These suites can be created for individual requirements, and each test case added is automatically linked to the corresponding backlog item. You can also add clauses to filter work items by the iteration path for the sprint.

Group your test cases using requirement based suites to track the testing status of backlog items.

Query based test suites gather a collection of tests from the project, regardless of the requirements to which the test cases are linked.

Use a query to group test cases with specific characteristics, such as all tests with "Priority=1". The suite will automatically include every test case that matches the criteria defined in the query.

From the list of work items returned by the query, select the backlog items you want to test in this sprint. Then, click "Create Suites" to generate a suite for each query.

Additional information related to test plans and test suites

For each system release or product release, create a test plan and divide it into different test suites, adding test cases to each suite. Reuse the existing test cases in various test plans or suites. Additionally, you have the option to copy or clone test cases. Creating a copy establishes a new baseline, ensuring that changes to these new test cases do not impact your previous test plans.

Each sprint can include requirements and features that are developed during their respective sprints, with each feature encompassing all necessary test cases. Since an epic might be developed across multiple sprints, the features could be distributed among different sprints.

If a feature or functionality that was verified in a previous sprint is found to be broken, or if the completed sprint has test cases with open bugs, then the related suite will be re-run by including it in the regression test suite of the new sprint.

A regression test suite should include all previously failed test cases, as well as test cases from bug validation and existing features that might be impacted. It is good practice to conduct an impact analysis and incorporate all relevant test cases.

The image below shows recommended structure of a test plan and its test suites.

Work item traceability

Every work item can have many children but only one parent. Bugs and test cases are connected through the proper link in ADO ("Affected by", "Tested by"):

• Epics and features are tested by test cases.
• Test cases are affected by bugs.

References

• How-to Write and Review a Test Case in ADO
• Azure DevOps Test Case creation
• Test document templates
• Security Testing Guideline