How to: Run Automated Tests In a Lab Environment Using Microsoft Test Manager
When you run automated tests in Microsoft Test Manager, you must run them in a lab environment. You can run automated tests in SCVMM environments and standard environments. The process is the same for both types of lab environments. To run automated tests in a lab environment, you must associate your automated tests with test cases, and then add the test cases to a test suite in a test plan.
Requirements
- Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional
Note
We do not recommend that you use SCVMM environments to run load or stress tests. Instead, use physical computers in a standard environment.
Use the following procedures to set up your test plan and run the tests using the appropriate environment and test settings:
Build and Install Your Application in a Lab Environment
Run the Automated Test Using Microsoft Test Manager
View and update the test results
Prerequisites
Before you can run automated tests using Microsoft Test Manager, you must follow these steps:
Configure a test controller for your lab environment. See Setting Up Test Controllers in Lab Environments with Microsoft Test Manager.
Create a lab environment that includes a machine for each role that is required in your tests. See Creating Lab Environments.
Associate your automated tests with test cases and add them to a test suite in a test plan. See How to: Associate an Automated Test with a Test Case or How to: Create Test Cases from an Assembly of Automated Tests Using tcm.exe.
Build and install your application in a lab environment
Note
If you are not using Team Foundation Build to build your application and tests, you can still run automated tests using Microsoft Test Manager. You must create a build definition that just has a share location added to it. The share location is where your assemblies for your tests are located.
To create a build instance that can be associated with the test plan, the build definition must be triggered at least once. If you don't have the Visual Studio 2012 Team Foundation Server build infrastructure set up, then you must create a fake build for the build definition. For more information, see Jason Prickett’s Blog: Creating Fake Builds in TFS Build 2010. You can create fake builds from the command line using the following parameters:
TfsCreateBuild.exe /collection:http://tfsservername:8080/tfs/collectionname /project: projectname /builddefinition:"Nightly Build" /buildnumber:"Application_Nightly_1.0"
To build and install your application in a lab environment
Start your lab environment. To start an SCVMM environment, select the environment from the Environments view and choose Start. To start a standard environment, select each machine in the Environments view and choose Start. See Operating and Modifying Lab Environments.
Verify that the machines in your environment have started and that their status is Ready.
Note
If you are running tests that interact with the desktop, you must use the Environment Viewer to log on to the machine that is used for that role. This will enable the testing capability for the test agent.
(Recommended) To prevent other users from accidentally connecting to your environment when you run your tests, choose the drop-down arrow next to Mark 'In Use' in the toolbar. Add a comment about who is using the environment and choose Mark 'In Use'. If another user tries to connect to the environment that is being used, a warning with the comments that you entered is displayed in the Connect to Environment dialog box.
Note
You can also mark an environment In Use from the Environment Viewer.
Install your application on the machines in your lab environment. You can use a build, deploy, and test workflow to do this automatically. For more information about how to do this, see How to: Create a Build-Deploy-Test Workflow for an SCVMM Environment. Or, you can install your application manually.
Note
If you are using an SCVMM Environment, we recommend that you take a snapshot of each virtual machine after you install your application so that you can easily return to this clean state with the application installed.
Run the automated test using Microsoft Test Manager
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Choose Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.
To run the automated test, choose the down-arrow on the center group switcher and then choose Testing Center.
On the center group menu bar, choose Test.
(Optional) To override the build, test settings, or lab environment that is selected in the test plan, right-click the test and then choose Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might choose a different environment. From the Run options dialog box, you can change these settings, and then choose Run to run the selected test.
Note
If you select a different lab environment, it must have the same roles as the environment that was created in the test settings that you use.
To run the automated test without changing any options, right-click the test and then choose Run.
The Analyze Test Runs activity is displayed that shows the progress of the test run that contains this test.
Note
You can run multiple automated tests by selecting multiple tests, or you can run a whole suite of tests. To run a suite of tests, choose the shortcut menu for the test suite and then choose Run.
View and Update the Test Results
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.
To view the test results, choose the down-arrow on the center group switcher and then choose Testing Center.
On the center group menu bar, choose Test and then choose Analyze Test Runs.
The Analyze Test Runs activity is displayed. It shows any test runs for this test plan.
Double-click a test run to open it and view the details.
The test run details are displayed.
(Optional) To update the title of your test run to be more meaningful, type the new name in Title.
(Optional) If your test failed, you can update the reason for the failure. Choose Resolution and select the reason for the failure from the list.
(Optional) To add comments to the test result, choose the Comments icon. Type your comments and then choose Save comments.
(Optional) To view the details of the individual test, double-click the test.
The test result is displayed that shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run.
Note
If, from your analysis, you determine that there is a bug, you can create a bug from this view.
To save these changes for this test run, choose Save in the toolbar.