Installing and Configuring Visual Studio Agents and Test and Build Controllers

To run tests remotely on one or more computers using Microsoft Visual Studio 2010, you have to install a test controller, and test agents. You install a test controller on any suitable computer, and a test agent on each of the computers that are running components of the system under test. The test controller communicates with the test agents to initiate the tests and to collect test result data. For more detailed information about why you might require test controllers and test agents, see Setting Up Test Machines to Run Tests or Collect Data.

Before you install test controller and test agents, you should verify that you have the required hardware and software to meet your specific needs. For detailed information on test controller and test agent hardware and software requirements, see Test Controller and Test Agent Requirements.

There are two principal scenarios in which you install test controllers and agents. The steps summarized here are explained in more detail in the rest of this topic:

• Remote tests. You want to run tests by using Visual Studio 2010, and you want to run the system on one or more computers that are separate from your Visual Studio computer.

  In this case you have to:

  1. Install a test controller on a suitable machine.

  2. Install a test agent on each machine on which the system under test will run.

  3. Register each test agent with the test controller.

• Lab tests. You want to run tests either manually or automatically on a lab environment by using Microsoft Test Manager and Visual Studio Lab Management. In this case you have to:

  1. Install a test controller on a suitable machine.

  2. Register the test controller with a Team Foundation project.

  3. Install a test agent on each computer or virtual machine template on which tests will be run.

     Important

     Do not register the test agents with a test controller. This will be done by lab manager when it sets up the lab environment.

You might also have to install lab agents, build controllers and build agents. The following table shows which ones you need:

You can use test controllers and test agents with any of the following client editions for Microsoft Visual Studio 2010:

• Visual Studio Test Professional 2010

• Visual Studio 2010 Ultimate

• Visual Studio 2010 Premium

You can use build agents with Team Foundation Server, or as part of the workflow capability for Visual Studio Lab Management. Lab agents are only used with Visual Studio Lab Management.

This topic provides the procedures for how to install and configure the components of the Microsoft Visual Studio Agents 2010 software on a physical or a virtual machine. It also contains the steps to install a build controller or build agent to work with Visual Studio Lab Management.

You should have a CD labeled "Microsoft Visual Studio Agents" or a downloaded ISO image to use to install a test controller, a test agent or a lab agent. To access the download page for the Visual Studio Agents ISO use this Microsoft Web site.

If you just want to run automated tests using Microsoft Visual Studio 2010 you can use the following walkthrough to install the test controller and test agents and set up your test settings: Walkthrough: Install Test Controller and Test Agents for Visual Studio Automated Tests.

Use any of the following links for the instructions to install and configure the components that you require:

• Supported operating systems

• Requirements for workgroups and multiple domains

• Install a test controller

• Set up security for test controllers and agents

• Install a lab agent

• Install a test agent

You should use the installation DVD for Team Foundation Server for this installation:

• Install a build controller or a build agent

The following sections help with information about issues that you may encounter for these installations:

• Test Controller and Test Agent Connection Points

• Troubleshooting

For more information about how to manage test controllers and test agents after installation, see Managing Test Controllers and Test Agents.

Prerequisites

The following operating systems are supported for these components:

• Windows Server 2008 original release version or later versions

• Windows Server 2003 SP2 or later versions

• Windows 7 original release version or later versions

• Windows Vista SP2 or later versions

• Windows XP Professional SP3 or later versions

Requirements for Workgroups and Multiple Domains

If you plan to run your test controllers and test agents on machines that are in a work group and not in a domain, or machines that run in different domains that do not have a two-way trust relationship, you must follow these steps as part of your installation process:

1. On each computer where you will install a test controller or test agent, create a local user account that is a member of the Administrators group. Use the same account and password on each machine.

2. Use this user account for your test controllers when you install and configure them.

3. Add this account to the TeamTestAgentService group on the test controller machine.

   Warning

   When you add this user account to the group, do not include the name of the local machine. For example, do not add the user account as <machine name>\<username>. Just add <username> as the user account for the group.

4. Install and configure your test agents using this same account.

Install a Test Controller

A single test controller manages one or more test agents by sending information to test agents about the task that the agent has to perform. By using a test controller, you can run tests on any machine that has a test agent installed on it. In addition, test controllers enable you to use diagnostic data adapters to collect data or impact the test machine when the tests are run. You can run tests on several machines at the same time. For more information about diagnostic data adapters, see Setting Up Machines and Collecting Diagnostic Information Using Test Settings.

You use test controllers in the following situations:

• To create environments by using Microsoft Test Manager

• To assign roles to run tests remotely and collect diagnostics and data remotely by using Microsoft Visual Studio 2010

For more information about environments and roles, see Setting Up Test Machines to Run Tests or Collect Data.

Each test controller runs as a service that manages a set of machines on which the test agent software is installed. You can create multiple environments that are run by the same test controller based on agent availability. For example, you can install one test controller and then install and register nine test agents on virtual or physical machines with this test controller. You can then create three environments that each have three roles and assign the appropriate machine from these nine test agents to the role.

Required Permissions

To install a test controller, you must be a member of the Administrators security group on the computer.

To install a test controller

1. If you have not already started the installation for Microsoft Visual Studio Agents 2010, load the "Microsoft Visual Studio Agents" CD or a downloaded ISO image and then launch setup.exe, which is located in the root folder.

   The Visual Studio Agents 2010 Setup page is displayed.

2. Click Install Visual Studio Test Controller 2010.

   The Microsoft Visual Studio Test Controller 2010 Installation page appears.

3. Click Next.

   The Start page is displayed.

4. Review the License Terms, select I have read and accept the license terms, if appropriate, and then click Next.

   The Options page is displayed.

5. (Optional) To install in a different location, click Browse to select a different installation folder.

6. Click Install to start the installation.

   The Installing Components page is displayed. When the installation is finished, the Finish Page is displayed.

7. To configure the test controller that you have installed, select Configure test controller now. Then click Configure to start the Test Controller Configuration tool.

   The Configure Test Controller dialog box is displayed.

   Note

   You can run this tool to reconfigure your test controller at any time. Click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Visual Studio Test Controller 2010 Configuration Tool.

8. Select the user to use as the logon account for your test controller service.

   Note

   Null passwords are not supported for user accounts.

9. (Optional) If you want to use this test controller with a physical or a virtual lab environment, you must register the controller with the team project collection for the team project that contains your environments. Select Register with Team Project Collection. To find your team project collection, click Browse, and then click Add to add the server if necessary. Or type the name in Register the test controller with the following Team Project Collection. Make sure that you enter the full path if you type the name. For example, http://<server name>:8080/tfs/DefaultCollection.

   Warning

   Test Controller In Same Domain: If you want to register your test controller with your Team Foundation Server project collection, the account that you use for the test controller service must be a member of the [Project Collection]\Project Collection Test Service Accounts group, or the account that you use to run the test controller configuration tool must be a member of the Project Collection Administrators group.

   Test Controller in Different Domain or Workgroup: If you want to register a test controller that is in a workgroup or a different domain that does not have a two-way trust with the domain for your Team Foundation Server, you must also create a local user account with the same user name and password on the Team Foundation Server machine and the test controller machine. This local user account must also be added as a member of the following Team Foundation Server group: [Project Collection]\Project Collection Test Service Accounts. Then you can register the test controller using this local user account when you run the test controller configuration tool.

   Note

   For more information about environments, see Setting Up Test Machines to Run Tests or Collect Data.

10. (Optional) To configure your test controller for load testing, select Configure for load testing.

    For more information about load testing, see Testing Application Performance and Stress, Distributing Load Tests Across Multiple Test Machines Using Test Controllers and Test Agents and Using Test Controllers and Test Agents with Load Tests.

    1. Type your SQL Server instance in Create load test results database in the following SQL Server instance.

       Note

       If you do not have SQL Server installed, you can use SQL Express. For more information about how to download and install SQL Express, visit the following Microsoft Web site. For more information about sizing considerations, see Using Test Controllers and Test Agents with Load Tests.

    2. To manage the virtual user licenses for load testing with a test controller, click Manage virtual user licenses.

       The Manage virtual user licenses dialog box is displayed. You can add and remove license keys as required.

       Note

       You must purchase these virtual user licenses separately. For more information about how to manage your virtual user licenses, see Managing Your Virtual User Licenses for Load Testing with a Test Controller, Configuring Test Controllers and Test Agents for Load Testing and How to: Add Virtual User License Keys for Load Testing.

       Note

       The Visual Studio 2010 Load Test Feature Pack provides active Microsoft Visual Studio Ultimate 2010 with MSDN subscribers a unique Visual Studio Load Test Virtual User Pack 2010 license key to generate unlimited virtual users.

       To download the feature pack, which is available for MSDN Subscribers only, visit MSDN Subscribers Downloads. For more information about the feature pack, see Microsoft Visual Studio 2010 Load Test Feature Pack.

11. To apply your changes, click Apply Settings.

    A Configuration summary dialog box is displayed. It shows the status of each step that is required to configure your test controller.

12. To close the Configuration summary dialog box, click Close. Then click Close again to close the Test Controller Configuration Tool.

    The Visual Studio Agents 2010 Setup dialog box is displayed. From here, you can install any other components that you require on this computer.

    The test controller runs as a Windows Service on the computers on which it is installed. To view the properties of the service use Control Panel. In Control Panel, click Administrative Tools and then click Services. The name of the test controller service is Visual Studio Test Controller. The controller services can be started and stopped like other services in Windows.

    Note

    If a test controller is registered with a team project, then you can configure and monitor it, and any registered test agents using the Test Controller Manager in the Lab Center for Microsoft Test Manager. Otherwise, to configure and monitor the test controller and any registered agents, click Test in Microsoft Visual Studio 2010 and point to Manage Test Controllers. You must have the correct permissions. For more information, see the section about security in this topic.

If you install the test controller on a computer that runs an operating system with user account control (UAC) the registry key HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\system\ LocalAccountTokenFilterPolicy is set to a value of 1. This disables remote restrictions of UAC to allow remote authentication if a test agent is configured to use a local account. For more information, go to the following Microsoft Web site.

Set Up Security for Test Controllers and Agents

The setup for the test controller creates user groups and services that together help provide security for your environment. The following groups are created during setup of the test controller and are used to authenticate users:

User Groups	Created When You Install	Permissions
TeamTestControllerUsers	Test Controller	Users in this group can: Start test runs View results
TeamTestControllerAdmins	Test Controller	Users in this group can: Perform the same tasks as TeamTestControllerUsers Delete, pause, or postpone any test run Add, remove and change the status of test agents Create environments.
TeamTestAgentService	Test Controller	Users in this group can: Connect to a test controller Note The test agent user must be added to this group on the computer that has the test controller installed.

When a test agent connects to a test controller, the test controller checks that the user for the test agent service is a member of one of these groups. If the user does not belong to any of these groups, the connection is rejected.

Administer a Test Controller Registered with Team Foundation Server

If your test controller is registered with a team project collection, you do not have to add the user account for the test controller to these user groups to administer the test controller. If you are a member of the Project Collection Test Service Accounts or a member of the Project Collection Administrators group for your Team Foundation Server then you can administer the test controller.

Administer a Test Controller Not Registered with Team Foundation Server

After the test controller is installed, access to the controller is limited to members of the TeamTestControllerUsers and TeamTestControllerAdmins groups that were created during setup, and to the Administrators group. Add appropriate users, groups, or both to these groups to enable them to access the test controller.

Install a Lab Agent

Team Foundation Server uses lab agents to manage virtual machines that have been configured for Visual Studio Lab Management and can be accessed through Microsoft Test Manager.

You can only install a lab agent on a virtual machine.

Required Permissions

To install a lab agent, you must be a member of the Administrators security group on the computer.

To install a lab agent

1. If you have not already started the installation for Microsoft Visual Studio Agents 2010, load the "Microsoft Visual Studio Agents" CD or a downloaded ISO image and then launch setup.exe, which is located in the root folder.

   The Visual Studio Agents 2010 page appears.

2. Click Install Visual Studio Lab Agent 2010.

   The Microsoft Visual Studio Lab Agent 2010 Installation page appears.

3. Click Next.

   The Start page is displayed.

4. Review the License Terms, select I have read and accept the license terms, if appropriate, and then click Next.

   The Options page is displayed.

5. (Optional) Click Browse to select a different installation folder.

6. Click Install to start the installation.

   The Installing Components page is displayed. When the installation is finished, the Finish page is displayed.

7. Click Finish to complete the installation.

   This virtual machine can now be used as part of a virtual environment.

   The Visual Studio Agents 2010 Setup dialog box is displayed. From here, you can install any other components that you require on this virtual machine.

Install a Test Agent

A test agent can be installed in the following ways:

1. As a service

2. As an interactive process

If you want to run tests that interact with your desktop, such as coded UI tests, you must install the test agent that runs these tests as an interactive process. When you configure the test agent as part of the installation process, you can select how you want to run the agent. For more information about how to run a test agent as a process, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop.

Required Permissions

To install a test agent, you must be a member of the Administrators security group on the computer.

When you configure a test agent to register with a test controller, you must also be a member of the Administrators security group on the test controller machine.

To install a test agent

1. If you have not already started the installation for Microsoft Visual Studio Agents 2010, load the "Microsoft Visual Studio Agents" CD or a downloaded ISO image and then launch setup.exe, which is located in the root folder.

   The Visual Studio Agents 2010 page appears.

2. Click Install Visual Studio Test Agent 2010.

   The Microsoft Visual Studio Test Agent 2010 Installation page appears.

3. Click Next.

   The Start page is displayed.

4. Review the License Terms, select I have read and accept the license terms, if appropriate, and then click Next.

   The Options page is displayed.

5. (Optional) Click Browse to select a different installation folder.

6. Click Install to start the installation.

   The Installing Components page is displayed. The test agent install requires performance tools to be installed to collect performance data when tests are run. When the installation is finished, the Finish Page is displayed.

7. To configure the test agent you have installed, select Configure test agent now. Then click Configure to start the Test Agent Configuration tool.

   The Configure Test Agent dialog box is displayed.

   Note

   You can run this tool to reconfigure your test agent at any time. Click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Visual Studio Test Agent 2010 Configuration Tool.

8. You are presented with two options for how the test agent will be run:

   Service   If you do not have to run automated tests that interact with the desktop, such as coded UI tests or creating a video recording when your test runs, under Run the test agent as, select Service. The test agent will be started as a service. Click Next.

   You can now enter the details about the user when the test agent starts as a service.

   1. Type the name in User name.

   2. Type the password in Password.

      Important user account information
      Null passwords are not supported for user accounts.
      If you want to use the IntelliTrace collector or the network emulation, the user account must be a member of the Administrators group.
      If the agent user name is not a member of the TeamTestAgentService group it will be added to this group, which requires permissions on the test controller machine.
      The user for the interactive process must be a member of the TeamTestAgentService security group or the process will not be able to run the tests assigned by the controller.

   Interactive Process   If you want to run automated tests that must interact with the desktop, such as coded UI tests or creating a video recording when your test runs, select Interactive Process. The test agent will be started as an interactive process instead of a service. Click Next.

   You can now enter the details about the user when the test agent starts as a process, and other options.

   1. Type the name in User name.

   2. Type the password in Password.

      Note

      If you configure the test agent to run as an interactive process with a different user who is not the currently active user, you must restart the computer and log on as this different user to be able to start the agent.

      Important user account information
      Null passwords are not supported for user accounts.
      If you want to use the IntelliTrace or the network emulation data and diagnostic adapter, the user account must be a member of the Administrators group. If the machine that is running the test agent is using Windows Vista or later versions, or any OS that has Least-Privileged User Account, you have to run it as an administrator also (elevated).
      If the agent user name is not a member of the TeamTestAgentService group it will be added to this group, which requires permissions on the test controller machine.
      The user for the test agent service must be a member of the TeamTestAgentService security group or the service will not be able to run the tests assigned by the controller.

   3. To make sure that a computer that has a test agent can run tests after it reboots, you can set up the computer to log on automatically as the test agent. Select Log on automatically. This will store the user name and password in an encrypted form in the registry.

   4. To make sure that the screen saver is disabled because this might interfere with any automated tests that must interact with the desktop, select Ensure screen saver is disabled.

      Warning

      There are security risks if you log on automatically or disable the screen saver. By enabling automatic log on, you enable other users to start this computer and to be able to use the account that automatically logs on. If you disable the screen saver, the computer might not prompt for a user to log on to unlock the computer. This lets anyone access the computer if they have physical access to it. If you enable these features on a computer, you should make sure that these computers are physically secure. For example, these computers are located in a physically secure lab. (If you clear Ensure screen saver is disabled, this does not enable your screen saver.)

9. (Optional) If you are installing the agent so that you can run remote tests by using Visual Studio, you should register this agent with a test controller. If you are setting up a machine or a virtual machine template to use in a lab environment, do not register the agent with a controller at this stage. It will be registered by Lab Manager when an environment is created.

   To register this agent with a test controller, select Register with test controller. Type the name of your test controller followed by a colon (:) and the port number that you are using in Register the test agent with the following test controller. For example, type agent1:6901.

   Note

   The default port number is 6901.

   Note

   If you are setting up this agent in a virtual machine that will be used with Lab, do not register the agent with a test controller. This will be done automatically by Lab Management.

10. To apply your changes, click Apply Settings.

    A Configuration summary dialog box is displayed. It shows the status of each step that is required to configure your test agent.

    Note

    The last step in the installation process resets the network adapter. This might cause a network outage for brief period of time. This is expected behavior.

11. To close the Configuration summary dialog box, click Close. Then click Close, to close the Test Agent Configuration Tool.

    Note

    There is a notification area icon that runs on the computer for a test agent that is running as a process. It shows the status of the test agent. You can start, stop or restart the agent if it is running as a process using this tool. To start the test agent as a process if it is not running, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Visual Studio Test Agent 2010.

    The Visual Studio Agents 2010 Setup dialog box is displayed. From here, you can install any other components that you require on this computer.

Install a Build Controller or a Build Agent

Visual Studio Lab Management uses build controllers and build agents to deploy applications to virtual environments that can be accessed through Microsoft Test Manager. This enables the workflow capability for Lab Management. To install a build controller or build agent for this purpose, you install the build service from the installation DVD for Team Foundation Server. The build service includes both the build controller and the build agent. For more information about Visual Studio Lab Management, see Using a Virtual Lab for Your Application Lifecycle.

The following steps can be used to install a build controller for use with Team Foundation Server or to install a build agent on a virtual machine that is part of a virtual environment.

The build agent is required to enable the workflow capability for Visual Studio Lab Management. The build agent is used by the build definition to deploy the application to the virtual machine. This build agent should not be used to run build definitions that just compile source code, because this build agent will not have permissions to access source code by default.

Required Permissions

To install a build controller or a build agent, you must be a member of the Administrators security group on the computer.

To install a build controller or a build agent

1. To install a build controller or a build agent, you must use the installation DVD for Team Foundation Server. This installation has a selection to install the build service, which will install both a build controller and a build agent. You cannot install them separately.

2. From the installation DVD for Visual Studio Team Foundation Server, start either the 32-bit or the 64-bit setup.exe to match the operating system that you are running.

   The Team Foundation installation wizard appears.

3. On the Welcome page, click Next.

4. On the License Terms page, read the terms of the license. Click I have read and accept the license terms, and then click Next.

5. Click Build Service, and then click Install.

6. If the installation wizard prompts you to restart your server, click Restart Now. After the server is restarted, installation resumes.

7. Select Launch Team Foundation Server Configuration Tool, and then Click Configure.

   Note

   You might need to install .NET Framework version 3.5 to launch the configuration tool for the build service. If the virtual machine has Windows Server 2008 installed as the operating system, then you must use Server Manager to install .NET Framework version 3.5.

8. The build service configuration wizard is displayed. Click Start Wizard.

1. If you are installing a build controller, you need to register the build controller with the team project collection that you are using for Lab Management. To do this, follow these steps:

   1. Click Next.

   2. To select the team project collection that you want to use, click the Project Collection page, and then click Browse.

   3. To find your team project collection, click Browse and then click Servers to add your server if necessary. Or type the name in Register the test controller with the following Team Project Collection. Make sure that you enter the full path if you type the name. For example, http://<server name>:8080/tfs/DefaultCollection.

   4. Click Next. Then select Configure later or use with Team Foundation Lab Management and click Next.

   5. To select an account for the build controller service, you can select Use a system account and then select the recommended default of the Local System account (NT AUTHORITY\SYSTEM) which has administrative privileges. Or you can select Use a user account and enter the Account Name and Password.

      Note

      The account that you use for this service must have administrative privileges on the machine that it runs on.

   6. To select the port to use for connections with Team Foundation Server, select Default port 9191. If you cannot use this default number, select Port number to use a different port.

   7. Click Next and then click Verify.

2. If you are installing a build agent, you do not have to enter any data. You can accept the default settings as you follow the pages for the wizard.

   Note

   You do not have to add the name of the team project collection. This will be done automatically when the virtual machine is deployed by Lab Management.

3. To complete the configuration, click Configure. After it has configured the build service correctly, click Next, click Close, and then click Close.

   The Team Foundation Server Administration Console is displayed. You can close this console.

   Note

   For the latest information about the installation, see the scenario for installing Team Foundation Build Service in Installation Guide for Team Foundation. For information about how to install and configure a build controller, see Configure a Build Machine.

   Now you can install any other components that you require.

Test Controller and Test Agent Connection Points

The following illustration shows the connection points between test controller, test agent, and the client. It outlines which ports are used for incoming and outgoing connections as well as security restrictions used on these ports.

For more information, see Configuring a Test Controller and Test Agent Across a Firewall.

Troubleshooting

If a test controller is registered with a team project collection, you can configure and monitor it, and any registered test agents using the Test Controller Manager in the Lab Center for Microsoft Test Manager. All test controllers that are registered to the team project collection are displayed.

Otherwise, to configure and monitor the test controller and any registered agents, click Test in Microsoft Visual Studio 2010 and point to Manage Test Controllers.

Agents should be in the ready state. If a test agent is not in the list of test agents, or the test agent is in the disconnected state, you can run the Test Agent Configuration Tool on the computer where the agent is installed and the Test Controller Configuration Tool on the computer where the controller is installed.

Verify the following criteria for the test controller:

• The user for the test controller is a member of the TeamTestControllerAdmins group or the user for the test controller is a member of the Administrators group.

• If you are accessing the controller from Microsoft Test Manager, make sure that the test controller is configured for the correct team project collection

• If you are accessing the controller from Microsoft Visual Studio 2010, make sure that it is not configured for a team project collection.

  Note

  You might want to run with elevated permissions, or restart your computer, if the test controller is running on the same machine as Microsoft Visual Studio 2010 and you have added the user to the TeamTestControllerAdmins group when this user is the active user.

If you want to register your test controller with your Team Foundation Server project collection, the account that you use for the test controller service must be a member of the Project Collection Test Service Accounts group for the Team Project Collection, or the account that you use to run the test controller configuration tool must be a member of the Project Collection Administrator group.

Verify the following settings for the test agent:

• The user for the test agent is a member of the TeamTestAgentService group on the computer where the test agent in installed and on the computer where the test controller is installed.

  Note

  If this user is the current user, when you add this user to the test controller machine, you must log off or restart this computer.

• The correct test controller name and port have been entered.

You can also examine the %LOCALAPPDATA%\Temp\TestControllerConfigUI.log and TestAgentConfigUI.log files for errors and more details. You can open these files from the link on the Configuration Summary page when you use the Test Agent or Test Controller configuration tools.

If you are running Windows Firewall, the test controller setup adds the following exceptions to your firewall. Make sure that this exception has been added.

See Also

Tasks

How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop

Walkthrough: Install Test Controller and Test Agents for Visual Studio Automated Tests

How to: Specify the Network Adapter to Use on a Test Controller or Test Agent with Multiple Network Adapters

How to: Specify Timeout Periods for Test Controllers and Test Agents

Concepts

Setting Up Test Machines to Run Tests or Collect Data

Configuring a Test Controller and Test Agent Across a Firewall

Test Controller and Test Agent Requirements