Managing Test Controllers and Test Agents

If you want to run tests remotely, distribute your tests across multiple computers, run load tests, or run tests from a test plan using an environment with Microsoft Test Manager, you must install a test controller and test agents. You can use physical machines, virtual machines, or a combination of both for this. One machine can have a test controller installed and running on it. Each of the other machines has a test agent installed and running on it. The test controller manages the agents and communicates what each agent needs to do. An agent may run tests or run diagnostic data adapters when the test is run.

For more information about how to install and configure test agents and test controllers, see Installing and Configuring Visual Studio Agents and Test and Build Controllers and Test Controller and Test Agent Requirements.

You can administer both the test agents and the test controller. If a test controller is registered with a team project, you can configure and monitor it and any registered test agents using the Test Controller Manager in the Lab Center for Microsoft Test Manager. Otherwise, to configure and monitor the test controller and any registered agents, click Test in Microsoft Visual Studio 2010 and point to Manage Test Controllers.

Managing Test Agents for a Test controller

After you have completed the installation and configuration of both your test controller and test agents, use the following procedures to manage the test agents.

• Add a test agent to a test controller

• Remove a test agent from a test controller

• Change the settings for a test agent

• Configure a test controller

• Manage agents when you run your tests

• Determine the test controller used to run tests

• Loading tests from delay-signed test assemblies

Adding a Test Agent to a Test Controller

You might want to add a test agent to a different test controller or you may have to add a test agent to a test controller that you have just installed.

To add a test agent to a test controller

1. Click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Visual Studio Test Agent 2010 Configuration Tool.

   The Configure Test Agent dialog box is displayed.

   Note

   You must have a test agent already installed to add it to a test controller. For more information about how to install a test agent, see Installing and Configuring Visual Studio Agents and Test and Build Controllers.

2. If you want to change the way that the test agent is run, click Run Options.

   You are presented with two options for how the test agent is going to be run:

   Service   If you do not have to run automated tests that interact with the desktop, such as coded UI tests or creating a video recording when your test runs, under Run the test agent as, select Service. The test agent will be started as a service. Click Next.

   You can now enter the details about the user when the test agent starts as a service.

   1. Type the name in User name.

   2. Type the password in Password.

      Important user account information
      Null passwords are not supported for user accounts.
      If you want to use the IntelliTrace collector or the network emulation, the user account must be a member of the Administrators group.
      If the agent user name is not in the agent service it will try to add it, which requires permissions on the test controller.
      The user who is trying to use the test controller must be in the test controller's Users account or they will be unable to run the tests against the controller.

   Interactive Process   If you want to run automated tests that must interact with the desktop, such as coded UI tests or creating a video recording when your test runs, select Interactive Process. The test agent will be started as an interactive process instead of a service. Click Next.

   You can now enter the details about the user when the test agent starts as a process, and other options.

   1. Type the name in User name.

   2. Type the password in Password.

      Note

      If you configure the test agent to run as an interactive process with a different user who is not the currently active user, you must restart the computer and logon as this different user to be able to start the agent. In addition, null passwords are not supported for user accounts. If you want to use the IntelliTrace collector or the network emulation, the user account must be a member of the Administrators group.

      Important user account information
      Null passwords are not supported for user accounts.
      If you want to use the IntelliTrace or the network emulation data and diagnostic adapter, the user account must be a member of the Administrators group. If the machine that is running the test agent is using Windows Vista or later versions, or any OS that has Least-Privileged User Account, you have to run it as an administrator also (elevated).
      If the agent user name is not in the agent service it will try to add it, which requires permissions on the test controller.
      The user who is trying to use the test controller must be in the test controller's Users account or they will be unable to run the tests against the controller.

   3. To make sure that a computer that has a test agent can run tests after it restarts, you can set up the computer to log on automatically as the test agent use. Select Log on automatically. This will store the user name and password in an encrypted form in the registry.

   4. To make sure that the screen saver is disabled because this might interfere with any automated tests that must interact with the desktop, select Ensure screen saver is disabled.

      Warning

      There are security risks if you log on automatically or disable the screen saver. By enabling automatic log on, you enable other users to start this computer and to be able to use the account that automatically logs on. If you disable the screen saver, the computer might not prompt for a user to log on to unlock the computer. This lets anyone access the machine if they have physical access to the computer. If you enable these features on a computer, you should make sure that these computers are physically secure. For example, these computers are located in a physically secure lab. (If you clear Ensure screen saver is disabled, this does not enable your screen saver.)

3. To register this agent with a different test controller, select Register with test controller. Type the name of your test controller followed by : and the port number that you are using in Register the test agent with the following test controller. For example, type agent1:6901.

   Note

   The default port number is 6901.

4. To save your changes, click Apply Settings.

   A Configuration summary dialog box is displayed that shows the status of each of the steps to configure your test agent.

5. To close the Configuration summary dialog box, click Close. Then click Close, to close the Test Agent Configuration Tool.

   Warning

   If the agent is currently configured to run on another test controller, you must remove the test agent from that controller. If the test controller is registered with a team project, you can remove the test agent using the Test Controller Manager in the Lab Center for Microsoft Test Manager. Otherwise, to remove the test agent, click Test in Microsoft Visual Studio 2010 and point to Manage Test Controllers.

Removing a Test Agent from a Test Controller

A test agent must be set to the offline state before it can be removed.

To remove a test agent from a test controller

1. If the test controller is not registered with a team project, follow these steps.

   1. On the Test menu in Visual Studio, click Administer Test Controllers.

      The Administer Test Controller dialog box is displayed.

   2. In the Controller drop-down list, type the name of the computer on which you have set up the test controller. If you have administered a specific test controller previously, you can select the name from the list.

   3. In the Agents pane, select the test agent name. If the agent is still online, click Offline. To remove it, click Remove.

      Note

      Removing a test agent just disassociates it from the test controller. To completely uninstall the test agent, use Add or Remove Programs from Control Panel on the test agent computer.

2. If the test controller is registered with a team project, follow these steps.

   1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

   1. Click the down-arrow on the center group switcher and then click Lab Center. Click Controllers.

      The Test Controller Manager view is displayed.

   2. Select your test controller from the list of test controllers.

      Any test agents registered with this test controller should now be displayed in the test agents list.

   Note

   If a test agent is set up to run as a process that can interact with the desktop, the test agent is displayed in the list with an asterisk preceding the name.

   1. Select the agent that you want to remove. If the agent is still online, click Offline. To remove it, click the Remove selected test agent icon.

      Note

      Removing a test agent just disassociates it from the test controller. To completely uninstall the test agent, use Add or Remove Programs from Control Panel on the test agent computer.

   You cannot move a test agent to another test controller by using the steps in this procedure. To move a test agent to another test controller, remove the test agent from its current test controller. Then you can follow the steps in the procedure to add a test agent to a test controller.

Changing the Settings for a Test Agent

The status of the test agent can be any one of the following values:

• You can change the status and other settings for a test agent using the following procedures.

If a test controller is registered with a team project, you can configure and monitor it and any registered test agents using the Test Controller Manager in the Lab Center for Microsoft Test Manager. Otherwise, to configure and monitor the test controller and any registered agents, click Test in Microsoft Visual Studio 2010 and point to Manage Test Controllers.

To change the settings of a test agent

1. If the test agent is registered to a test controller that is registered with a team project, follow these steps:

   1. In Microsoft Test Manager, click the down-arrow on the center group switcher and then click Lab Center.

      To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

   2. Click Controllers.

      The Test Controller Manager is displayed.

2. If the test agent is registered to a test controller that is not registered with a team project, follow these steps:

   1. In Microsoft Visual Studio 2010, click Test and point to Manage Test Controllers.

      The Manage Test Controller dialog box is displayed.

3. Select the name of the test controller whose test agents you want to change in the test controller list. If the test controller does not appear in the list, check that the test controller is registered correctly. For more information, see the following procedure about how to configure a test controller.

4. (Optional) In the Test Agents pane, click the test agent computer for which you want to change the properties.

5. If you are using Microsoft Test Manager, click Configure.

   -or-

   If you are using Microsoft Visual Studio 2010, click Properties.

6. Change the following test agent properties as required:

Test Agent Property	Description
Weighting	Used to distribute load when you use test agents with different performance levels. For example, a test agent with a weighting of 100 receives two times the load as a test agent with a weighting of 50.
IP Switching	Used to configure IP switching. IP switching allows a test agent to send requests to a server by using a range of IP addresses. This simulates calls that come from different client computers. IP Switching is important if your load test is accessing a Web farm. Most load balancers establish affinity between a client and a particular Web server by using the client's IP address. If all requests seem like they are coming from a single client, the load balancer will not balance the load. To obtain good load balance in the Web farm, make sure that requests come from a range of IP addresses. Note You can specify a network adapter, or use (All unassigned) to automatically select one that is currently not being used. To use the IP switching feature, the Visual Studio Test Agent service must be running as a user in the Administrators group for that agent computer. This user is selected during agent setup, but can be changed by modifying the properties of the service and restarting it. To verify that IP switching is working correctly, enable IIS logging on the Web server, use the IIS logging functionality to verify that requests are coming from the IP addresses that you configured.
Attributes	Set of name/value pairs that can be used in test agent selection. For example, a test might require a particular OS. You can add attributes on the Roles tab of the test settings file for Visual Studio. For more information, see Create Test Settings to Run Automated Tests from Visual Studio. Note This setting is only available for agents that are registered with a test controller that is not registered to a team project, because these attributes are only used in test settings for Microsoft Visual Studio 2010.

Test agent weight and test agent attribute changes go into effect immediately, but do not affect tests that are running. The IP Address Range takes effect after the test controller is restarted.

1. (Optional) To change the status of a test agent, select the agent in the list and then select the action from the available choices based on the current status of the agent.

   Note

   If your test agent is running as a process, you manage the status of the test agent from the notification area icon that runs on the computer where your test agent is installed. This shows the status of the test agent. You can start, stop or restart the agent if it is running as a process using this tool. To start the test agent as a process if it is not running, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Visual Studio Test Agent 2010. This adds the notification area icon.

Configuring a Test Controller

To configure a test controller, you must use the Team Test Controller Configuration Tool. When you configure your test controller, you can register your test controller with a different team project collection, or unregister your test controller from a team project collection.

If you want to register your test controller with your Team Foundation Server project collection, the account that you use for the test controller service must be a member of the Project Collection Test Service Accounts group for the Team Project Collection, or the account that you use to run the test controller configuration tool must be a Project Collection Administrator.

To configure a test controller

1. To run the tool to reconfigure your test controller at any time, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Visual Studio Test Controller 2010 Configuration Tool.

   The Configure Test Controller dialog box is displayed.

2. Select the user to use as the logon account for your test controller service.

   Note

   Null passwords are not supported for user accounts.

3. (Optional) If you want to use this test controller with a physical or virtual environment, you must register the controller with a team project collection for the team project that contains your environments. Select Register with Team Project Collection. To find your team project collection, click Browse or you can type the name in Register the test controller with the following Team Project Collection. Make sure that you enter the full path. For example, http://<server name>:8080/tfs/DefaultCollection.

   Note

   For more information about environments, see Setting Up Test Machines to Run Tests or Collect Data.

4. (Optional) If you do not want to use your test controller with a physical or virtual environment, but only to run tests from Microsoft Visual Studio 2010, clear Register with Team Project Collection.

5. (Optional) To configure your test controller for load testing, select Configure for load testing. Then type your SQL Server instance in Create load test results database in the following SQL Server instance.

   For more information about load testing, see Testing Application Performance and Stress and Using Test Controllers and Test Agents with Load Tests.

6. (Optional) To manage the virtual user licenses for load testing, click Manage virtual user licenses.

   The Manage virtual user licenses dialog box is displayed. You can add and remove license keys as required.

7. To apply your changes, click Apply Settings.

   A Configuration summary dialog box is displayed that shows the status of each of the steps to configure your test controller.

8. To close the Configuration summary dialog box, click Close. Then click Close, to close the Team Test Controller Configuration Tool.

Determine the Test Controller Used to Run Your Tests

If there is an issue when you run your tests using a test controller, you might need to determine the test controller that is being used to verify that the test controller service is running, or that the test controller computer is started.

Running Tests with Visual Studio

If you run your tests using Microsoft Visual Studio 2010, the test controller you use to run your tests is displayed in your test settings. Use the procedure in this topic to view your test settings: How to: Edit a Test Settings File from Microsoft Visual Studio.

Running Tests Using Microsoft Test Manager

If you run your tests using Microsoft Test Manager, to use a test controller you must create an environment. When you run your tests, you select one of the following choices:

• Run: This runs your tests using the environment specified in your test plan.

• Run with options: You can use this option to run your tests with a different environment.

To find the environment that is specified in your test plan, use the following topic to view the environments in the Properties view for your test plan: How to: Choose Test Settings and Environments for a Test Plan.

When you know which environment you are using to run your tests, you can use the following procedure to view the details for this environment.

To determine the test controller for your environment

1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

2. Click the down-arrow on the center group switcher and then click Lab Center. Click Lab.

   The Environments view is displayed.

3. Select the environment that you use to run your tests from the list and then click Open.

   The environment is displayed. You can now view which test controller is selected for your environment. The test controller name is the same as the computer name that it runs on.

   Note

   The test controller service is called Visual Studio Test Controller. If you have issues with the test controller, you might want to verify that the computer is started and this service is running on the computer.

Managing Your Agents when You Run Your Tests with a Test Controller

When you add roles for your application to your test settings for Microsoft Visual Studio 2010, you can add agent properties for each of your roles. This determines which test agents are available for this role. When you run your tests using these test settings, the test controller that is selected for the test settings determines the availability of the required agents. These are the following situations that can occur when the agent availability is determined:

• There is no agent available for the role that must run the tests. Your tests cannot be run. You can perform one of the following actions and then rerun your tests:

  • You can wait for an agent to become available for this role to run the tests.

  • If there are any agents that are offline that can be used for this role, you can restart the agent so that it is available.

  • You can add another agent with the correct agent properties for that role to the test controller.

  • You can change the agent properties for this role in the test settings to enable other agents that you want to use.

• There is no agent available for one or more roles that run diagnostic data adapters. Your tests can be run, but the diagnostic data adapter cannot be run. You can run your tests without the diagnostic data adapter, or you can perform one of the following actions and rerun your tests:

  • You can wait for an agent to become available for these roles.

  • If there are any agents that are offline that can be used for this role, you must change the state of the agent to online from Administer Test Controller on the Test menu. In addition, you might have to restart the agent if it has been disconnected from the controller.

  • Verify that any agents that you might need for this test run are not busy running tests. You can check the status of any agents from Administer Test Controller on the Test menu.

  • You can add another agent with the correct agent properties for the role to the test controller.

  • You can change the agent properties for the role in the test settings to enable other agents that you want to use.

Loading Tests from Delay-Signed Assemblies

The test controller and test agents can only load test assemblies that are strongly signed assemblies, or unsigned assemblies. Some test assemblies are delay-signed because they need to have access into production assemblies for the application. However, these assemblies are not strongly signed because they are only test assemblies and are not distributed. These assemblies cannot be loaded because they are delay-signed, so you must disable strong name verification for those assemblies on all machines where the assembly will be loaded including the test controller machine. To disable the delay-signed verification, use sn.exe. The public key token of the delay-signed assembly for which strong name verification is requested to be skipped may also need to be included.

Use the Sn.exe (Strong Name Tool) to disable the delay-signed verification.

This disables strong-name verification, for the specified assembly only, on the computer on which you run the command. You can do this only if you have sufficient permissions.

After the test run has completed, re-enable the delayed-signing verification by using the SN.exe command.

A recommended way to disable and re-enable signing verification is to use the SN.exe commands in scripts. You can disable verification in a setup script and re-enable verification in a cleanup script.

See Also

Concepts

Installing and Configuring Visual Studio Agents and Test and Build Controllers

Setting Up Test Machines to Run Tests or Collect Data

Test Controller and Test Agent Requirements