How to: Analyze Test Runs Using Microsoft Test Manager

If you used Microsoft Test Manager to run automated tests or if you ran them from the command line using tcm.exe, you can analyze your automated test runs to determine why tests have failed or examine any other issues associated with the test run.

As you analyze the test results for each test, you can select a failure type and resolution for each failed test, or add a comment about your analysis to this point. To help with your analysis, you can open the test result for each test to see all previous runs of that test.

If you determine that there is an issue, you can create a bug and associate the data from the test result for this test with the bug. If you determine that the failed test is caused by a known issue, you can link that test to an existing bug. When this bug is fixed, the tester who verifies the bug can then see all the test cases that have to be rerun to verify that the bug is fixed.

If you want to add data to the test run, you can add attachments or comments to the test run itself.

To help with analysis, you might want to run your automated test using the manual test steps in the associated test case. For more information, see How to: Run a Test Case with Associated Automation as a Manual Test.

The following illustration shows how to select the run that you want to analyze and view the details.

To analyze your test runs using Microsoft Test Manager

1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

2. To view the test results, click the down-arrow on the center group switcher and then click Testing Center.

3. On the center group menu bar, click Test and then click Analyze Test Runs.

   The Analyze Test Runs activity is displayed. It shows any automated test runs for this test plan.

4. (Optional) To display manual test runs, click Show manual runs.

   The manual test runs are also displayed in the list of test runs.

5. Double-click a test run to open it and view the details.

   The test run details are displayed.

6. (Optional) To update the title of your test run to be more meaningful, type the new name in Title.

7. (Optional) To add comments to the test run, type your comments in Comments.

8. For any failed tests, you can update the reason for the failure. Select the test in the list of tests, click the drop-down arrow next to Failure type and select the reason for the failure from the list.

9. (Optional) For any failed tests, you can update the resolution of the failure. Select the test in the list of tests, click the drop-down arrow next to Resolution and select the resolution from the list.

10. To create a bug if this failed test is caused by a new issue, click Create bug in the toolbar.

    The New Bug activity is displayed. All the test results data is automatically added to the bug. The bug is associated with the test case, so that when the bug is fixed the tester knows which test case to rerun. Type the necessary data in the bug fields and click Save and Close. For more information about how to add data to the bug, see How to: Submit a Bug Using Microsoft Test Manager.

11. To link this test to an existing bug, follow these steps:

    1. Select the test and click Link to bug.

       The Link Test Result to Existing Bug dialog box is displayed.

    2. Edit the query as required.

    3. Click Run to display the bugs from the query.

    4. Select a bug from the list, and then click Link bug.

12. To add notes for a test result in the test run, follow these steps:

    1. Select the test and click the Notes icon.

       The Notes for test result dialog box is displayed.

    2. Type in the notes and click Save notes.

13. (Optional) To view the details of the test results for an individual test, double-click the test.

    The test result is displayed that shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run. For more information about this view, see How to: View Test Results Using Microsoft Test Manager.

    Note

    After you examine the details and previous result history, you might determine that there is an issue. From this view, you can then create a bug that is associated with this test.

14. (Optional) To add an attachment to the test run, follow these steps:

    1. Click the drop-down arrow next to Attachments

    2. Click Add.

       The Add Attachment dialog box is displayed.

    3. Click Browse to locate the attachment.

    4. Type a description of the attachment in Comment.

    5. Click OK to add the attachment to the test run.

15. To save any changes for the test run, click Save in the toolbar.

See Also

Tasks

How to: Run Automated Tests from a Test Plan Using Microsoft Test Manager

Concepts

How to: Run Test Cases with Automation from the Command Line Using Tcm

Running Automated Tests