Test Results Reported

The test runtime engine produces test results of various kinds, depending on the types of tests you have run.

All test types share the following behavior:

• Basic results. Running a test produces a set of basic test results, whether you ran tests locally or remotely. For more information, see Basic Test Results. The Test Results window displays these results, which include outcomes such as Passed and Failed, and any error messages that were generated. You can customize the display of information in this window; for more information, see How to: Adjust Test Results Views.

• Test results file. Running tests generates a test results file and saves it to disk automatically. This XML file, which has the extension .trx, contains information about the test and the test run. Because a test run remains in memory until you close the solution, you can save it to disk at a specified location and with a specified name by clicking Export Test Run Results. Similarly, you can load test results from a .trx file on disk. For more information, see How to: Export Test Results.

  Note

  Visual Studio writes the automatically generated test results file to a Test Results folder within your solution folder hierarchy. If you used the New Project dialog box to create your project and accepted the default folder structure, the solution folder is in the folder hierarchy under My Documents. If you opened a solution from Team Foundation version control, the solution is within your workspace. For more information about the location of the Test Results folder, see Test Deployment Overview.

• Additional results. You can find more results for an individual test in the Test Results Details page, and for an entire test run in the Test Run Details page. The results that are displayed in both of these pages depend on the type of test.

  Note

  After you have run tests and viewed their results, you might want to change the tests and run them again, especially if some tests failed. To re-create a test run, possibly after changing one or more of its tests, click Rerun on the Test Results window toolbar. For more information, see How to: Rerun a Test.

General Reporting of Test Results

For all test types, the test name and a test result are displayed as soon as the test run starts. The result, such as, Pending, In Progress, Inconclusive, Pass, and Fail, is updated in real time throughout the test run. For certain test types, Web and load tests in particular, more extensive data is reported as the test is running; see Results of Specific Test Types.

You can pause, stop, and resume a test run using buttons on the Test Results window. As you do this, the test result is updated to reflect the current state of the test run. For more information, see How to: Pause, Resume, and Stop a Test Run.

When the test run is completed, its results are written to disk. For more information about identifying and using the file in which they are stored, see How to: Import Test Results.

Test Results Details Page

For all test types except Web and load, the test Results Details page displays all the information shown in the Test Results window, plus additional in-depth test results about an individual test. To do this, in the Test Results window, click the test result and press Enter, double-click the test result, or right-click the test result and then select View Test Results Details.

This window also displays certain errors that can cause a test to fail, such as the following:

• A container test, specifically a load test or an ordered test, is empty and therefore could not be run.

• One or more per-test deployment items could not be found and deployed.

Test Run Details Page

After you run tests, the overall status of the test run is displayed in the status bar of the Test Results window. This status is for the entire test run, not for the individual tests that it contains.

To see in-depth results for the entire test run, open the Test Run Details Page. To do this, click the status link on the status bar. The wording of the status link reflects the status of the test run: Test Run Error, Test Run Inconclusive, Test Run Failed, or Test Run Completed.

The Test Run Details Page displays general information about the test run, including the test run configuration that was active when it was run. It also states how many of the tests in the run passed, failed, or were skipped, and it describes any errors that were encountered, such as failure to instrument binary files.

This window also displays certain errors that can cause a test run to fail, such as the following:

• Deployment files specified in the run configuration could not be deployed.

• Code coverage failed because assemblies could not be instrumented.

• Code coverage failed because information could not be gathered.

• After a remote test run, files could not be copied from the controller.

Results of Specific Test Types

The different built-in test types produce different test results. For more information about test results, see Basic Test Results.

• Unit Tests. A unit test will pass unless it times out, an exception is thrown and there is no corresponding ExpectedException attribute, or an Assert statement causes it to fail or to produce a result of Inconclusive. For more information, see Anatomy of a Unit Test. Code-coverage results can also be obtained when unit tests are run; for more information, see How to: Obtain Code Coverage Data.

  For unit tests, the Test Results Details page also displays the test duration, the name of the computer on which the test ran, and the start and end times of the test. It also displays any error message that was generated, and, under Error Stack Trace, the location at which the error occurred.

• Manual Tests. The tester running the manual tests assigns its result and, optionally, types comments. When you request in-depth details as described in Test Results Details page, a window is displayed that shows the text of the manual test, any comments that the tester typed, and the test result.

• Web Tests. The in-depth test results for Web tests are more extensive than they are for most test types. They include, for example, the HTTP Requests that were issued and the HTTP Responses that were returned. A Web Browser tab displays an image of each Web page that was returned. For more information, see How to: Run a Web Test.

• Load Tests. Load test results are extensive and include, for example, a graph that shows the instantaneous values of counters over the duration of the test. You can view load test results as they are being generated, and you can view results from completed load test runs. For more information, see About the Load Test Results Repository.

• Generic Tests. The results for a generic test include a value of Passed or Failed and a display of any error messages that were generated. A generic test passes if the executable it wraps returns a value of 0, and fails if any other value is returned. A generic test can also create a detailed XML results file that can report more than just Passed or Failed but other values such as Inconclusive or Timed Out, in addition to more in-depth information. For more information, see Generic Tests Overview and Using a Summary Results File with a Generic Test.

• Ordered Tests. An ordered test passes if all the tests it contains also pass. The Test Results Details page contains a table that lists all the tests that are contained in the ordered test. You can double-click a row in this table to see detailed results for an individual test. For more information, see Ordered Tests Overview.

See Also

Tasks

How to: Adjust Test Results Views

Concepts

Basic Test Results

Other Resources

Saving and Publishing Test Results