Repeat a test with different data

  • [アーティクル]

When you write a manual test, you often want to specify that the test should be repeated several times with different test data. For example, if your users can add different quantities of a product to a shopping cart, then you want to check that a quantity of 200 works just as well as a quantity of 1.

To do this, you insert parameters in your test steps. Along with the test steps, you provide a table of parameter values.

Using Team Web Access with Team Foundation Server 2013 Update 2 or Visual Studio Online, you can also share parameters and their data between test cases. That way you can run multiple test cases with the same data.

Add parameters to a test case

Create a parameter by typing a name preceded by "@" in the actions and expected results of your test steps.

Create parameters in actions and results.

Underneath the list of steps, add combinations of parameter values. You might need to scroll down to see them.

Share parameters between test cases

Convert existing parameters to shared parameters, so that you can use them and the associated data in other test cases.

In the Parameter Values section, click Convert

After you have created a shared parameter set, open another test case and add the shared parameter set to that test case. You can search for the shared parameter set by name.

From Parameter Values, click Add

The shared parameter set is displayed in the Parameter values section after you add it. You can now use these parameters in your test case steps.

If the test case already has different parameter names for these shared parameters, map the shared parameter to the local parameter to use the shared parameter data.

Use drop-down to select shared parameter to map

When they are correctly mapped the data associated with the shared parameter will be displayed.

Add, edit and rename your shared parameter sets from the parameters tab. View the test cases that reference them in the test cases pane.

From the test hub, choose Parameters

Each shared parameter set is a work item. From the Properties tab, you can view or make changes to this work item. For example, you can assign owners and track changes.

Note

From Microsoft Test Manager using Test Professional 2013 Update 2 or Visual Studio Ultimate 2013 Update 2, you can view shared parameters for a test case but you cannot update them. To update shared parameters, you must use Team Web Access. There is a hyperlink in Microsoft Test Manager that takes you to the correct location to edit the shared parameters.

Run a test case with parameters

When you run a test case with parameters, test runner shows the first row of parameter values.

Test run iterations with parameter values.

When you’ve completed the steps, mark the test passed or failed.

Pass or fail the test iteration and go on to next

Then go on to the next iteration of the test, which uses the next row of parameter values.

Next test iteration has different parameter values

Some of the parameter values are wrong. Can I fix them without canceling the test?

Yes. Choose Edit from the context menu of the step.

Review the test results

If you marked any test iteration as failed, then the outcome of the whole test is shown as failed.

Any failed test iteration shows as a failed test.

If you inspect the test results in Microsoft Test Manager, you can see the different results for each iteration:

View Test Results For a Test With Iterations

Speed up test iterations by using record/playback

It can be error-prone and tedious to work through a long table of parameter combinations. To speed things up, create an action recording when you run the test with the first set of parameter values, and then play it back for the other sets.

Here’s how:

  1. Use Microsoft Test Manager to run the test.

  2. Select Create action recording before you press the Start button.

  3. Complete the first test iteration and move on to the next.

    Mark each step as passed or failed as you work. Enter parameter values in the application exactly as they are displayed in the test script.

  4. Choose Play to run the test with the next set of parameter values.

    Your actions will be played back automatically, but you still have to verify the results.

Record/playback doesn’t work with all applications. For details, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings.

Q and A

  • Q: Are parameters the best way to specify that the test should be run on Windows 7 and Windows 8? And with different browsers, databases, and so on?
    A: It’s better to use Test Configurations for that. With test case parameters, you run the different parameter values one after another, which makes it difficult to switch from one platform to another.

  • Q: Can I use parameters in shared steps?
    A: Yes. You set the parameter values in the test cases where you use the shared steps.

  • Q: Can I import parameter values from an Excel spreadsheet to my shared parameter sets?
    A: Yes. Copy the data from your Excel spreadsheet and paste it into your shared parameters grid. You can also copy the data from your grid back into Excel if you need to.