How to: Create Test Cases from an Assembly of Automated Tests Using tcm.exe
If you have created automated tests using Visual Studio, you might want to run these tests as part of a test plan. By adding them to a test plan, you can measure your testing progress. You can view the results from these automated tests and any other tests in your test plan together and track the progress that you are making. You can also link these test cases to requirements, so that you can view the status of automated tests for the requirement. For more information about how to do this, see Link Existing Test Cases to a Requirement.
To run these tests as part of a test plan, you can use a command line tool tcm.exe to create test cases for all your test methods in an assembly or some of your test methods based on test method properties or the test category. If you want to add a test to an existing test case, you can associate a test method directly with that test case. For more information, see How to: Associate an Automated Test with a Test Case.
If you have existing automated tests from earlier versions of Microsoft Visual Studio 2010, you can associate these tests with a test case after you upgrade the test project that contains the automated tests to Visual Studio 2010.
After you have imported your test methods and created test cases, you have to add these test cases into your test plan. To run these test cases from your test plans, you must use a physical or a virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment.
You must check in the test project that contains the automated test methods and make sure that the test project is part of your build definition. You must make sure that a build created by using this build definition is selected in your test plan. The information from the build is used by Microsoft Test Manager to locate the correct test method assembly and use this to run the automated tests.
For more information about how to check in your test project and team build, see Add Files to Version Control and Building the Application.
You can also run automated tests from the command line using test settings and a test environment. For more information, see How to: Run Test Cases with Automation from the Command Line Using Tcm.
Use the following procedures to create test cases from an assembly of automated tests that you can then run as part of your test plan:
Create the test cases from your assembly of automated tests and add them to your plan
Set up your test plan to use your team build
Create your test settings and environment to run your tests
Run the automated test using Microsoft Test Manager
View and update the test results
Common Parameters for the Procedures
Parameter |
Description |
|---|---|
/collection:CollectionURL |
Specifies the URI of the team project collection. The format for the URI is as follows: http://ServerName:Port/VirtualDirectoryName/CollectionName If no virtual directory is used, then the format for the URI is the following: http://ServerName:Port/CollectionName If you do not know the correct path, contact your Team Foundation system administrator. |
/teamproject:TeamProjectName |
The team project that you want to use. This team project must be defined in the project collection that is specified by the /collection parameter. |
/login:UserName,Password |
Optional. Specifies the name and password of a user who is logged on to the application-tier server for Team Foundation Server and who has permissions to run the command. You would use this option if your Windows credentials do not have the appropriate permissions, or you are using basic authentication, or you are not connected to a domain. |
/? or help |
Displays help about the command in the Command Prompt window. |
Procedures
Create Test Cases from an Assembly of Automated Tests
To create test cases from an assembly of automated tests
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
Change directory to the folder that contains the assembly built from your test project.
To do this, first change directory to your solution folder. Then change directory to the folder for your test project by typing the following command at the command prompt:
cd <project name>\bin\Debug
To import all the test methods from your assembly, type the following at the command prompt:
tcm testcase /collection:<CollectionURL> /teamproject:<Team Project name> /import /storage:<name of your assembly>.dll
You will see results similar to these:
Modification Title
---------------- ------------------
Added <Name of test method>
Added <Name of test method>
Summary
------------
Added 2
Updated 0
Total 2
Note
The test cases that are created will have the same names as the test methods that were imported.
(Optional) You can limit the test methods that are imported from the assembly based on test categories. For more information about test categories, see Defining Test Categories to Group Your Tests. For example, the following command shows you how to import tests that are in both the ShoppingCart and SmokeTest categories.
tcm testcase /collection:<CollectionURL> /teamproject:<Team Project name> /import /storage:<name of your assembly>.dll /category:"ShoppingCart&SmokeTest"
(Optional) You can limit the test methods that are imported from the assembly based on a test method property. For example, the following command shows how to import all test methods that have priority set to 0 or 1.
tcm testcase /collection:<CollectionURL> /teamproject:<Team Project name> /import /storage:<name of your assembly>.dll/maxpriority:1
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.
To select a test plan, click the down-arrow on the center group switcher and then click Testing Center.
On the center group menu bar, click Plan.
(Optional) To select a different test plan, click the test plan name hyperlink in the upper-right corner, or click the home icon.
You can add the test cases to a new test suite, or an existing test suite. To add the test cases into your plan, click Contents and then select the test suite in the test suite hierarchy or add a new test suite. Then click Add in the test suite details pane.
The Add Test Cases to Suite dialog box is displayed.
Note For more information about test suites in test plans, see Organizing Test Cases Using Test Suites.
Select the test cases that were created. These test cases will have the same names as the test methods in the test assembly. Then click Add Test Cases.
The test cases are shown in the list of tests for the test suite that you selected.
Note
You can also import your test cases directly into a static test suite. For more information about how to do this, see tcm: Importing Automated Tests into Test Cases.
Set Up Your Test Plan To Use Your Team Build
To set up your test plan to run the automated test that you have created, you must choose the correct build definition used to build your automated test methods. You must do this so that the automated test assembly can be found in the share location for your build definition and then it can be run from Microsoft Test Manager.
To set up your test plan to use your team build
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.
To select a test plan, click the down-arrow on the center group switcher and then click Testing Center.
On the center group menu bar, click Plan.
To set up your test plan to run the automated tests, click Properties and then click Filter for builds.
The Filter completed builds dialog box is displayed.
To select the build definition that is used to build your automated tests, click Build definition.
Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, click Build quality.
Note
For more information about build definitions and build quality, see Define Your Build Process.
To save your changes, click Set build filter.
To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first click Save to save the plan and then click Modify.
The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements.
To close the Assign Build activity and return to the test plan properties, click the Close icon.
To save these changes for this test plan, click Save in the toolbar.
Create Your Test Settings and Environment to Run Your Tests
To run your automated tests, you must use a physical or virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment.
You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data.
Note
If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop.
To create your test settings and environment to run your tests
To create a physical environment, follow the steps in this topic: Creating a Physical Environment to Use for Testing.
If you are using Visual Studio Lab Management, you can create a virtual environment. To create a virtual environment, follow the steps in this topic: How to: Create an Environment from Virtual Machines or Templates. If you want more information about how to create virtual environments, see Virtual Environments Concepts and Guidelines.
To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests as Part of a Test Plan.
Run the Automated Test Using Microsoft Test Manager
To run the automated test using Microsoft Test Manager
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.
To run the automated test, click the down-arrow on the center group switcher and then click Testing Center.
In the center group menu bar, click Test.
(Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then click Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might choose a different environment. From the Run options dialog box, you can change these settings, and then click Run to run the selected test.
Note
If you select a different environment, it must have the same roles as the environment that was created in the test settings that you use.
To run the automated test without changing any options, right-click the test and then click Run.
The Analyze Test Runs activity is displayed that shows the progress of the test run that contains this test.
Note
You can run multiple automated tests by selecting multiple tests, or you can select to run a whole suite of tests. To run a suite, right-click the test suite and then click Run.
View and Update the Test Results
To view and update the test results
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.
To view the test results, click the down-arrow on the center group switcher and then click Testing Center.
On the center group menu bar, click Test and then click Analyze Test Runs.
The Analyze Test Runs activity is displayed. It shows any test runs for this test plan.
Double-click a test run to open it and view the details.
The test run details are displayed.
(Optional) To update the title of your test run to be more meaningful, type the new name in Title.
(Optional) If your test failed, you can update the reason for the failure. Click Resolution and select the reason for the failure from the list.
(Optional) To add comments to the test result, click the Comments icon. Type your comments and then click Save comments.
(Optional) To view the details of an individual test, double-click the test.
The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run.
Note
If you determine that there is a bug from your analysis, you can create a bug from this view.
To save these changes for this test run, click Save in the toolbar.
See Also
Tasks
How to: Associate an Automated Test with a Test Case
Reference
tcm: Importing Automated Tests into Test Cases
Concepts
How to: Run Test Cases with Automation from the Command Line Using Tcm