Test Agents support for Visual Studio 2015

  • Article

With the recent release of Visual Studio 2015 CTP6 and TFS 2015 CTP1, we have not included Test Agents for Visual Studio 2015 package as we are still working on updating the agents for enabling more scenarios in the Build and Release workflow. This blog will give you details on how you can get Test Agents working for Visual Studio 2015 CTP6 and TFS 2015 CTP1 releases.

If you are trying out the test execution scenarios on VS 2015 CTP6 releases with TFS 2015 CTP1 and need Test Agents, we recommend you installing the Agents for VS2013. Test Agents and Test Controller for Visual Studio 2013 are compatible with the TFS 2015 CTP1 and VS 2015 CTP 6 releases, all the supported test scenarios will continue to work in this combination. However, we have a known issue with Coded UI tests compiled using VS 2015 CTP tools, we will have this issue fixed for the next CTP release, below are the details to get around this issue in the interim.

Impacted Experience

Newly created Coded UI tests in VS 2015 CTPs/Preview or Coded UI Tests (CUIT) upgraded to Visual Studio 2015 CTPs cannot be executed using the Agents for Visual Studio 2013. When you try to execute Coded UI tests you will see one of the following error messages:

For CUIT Store tests:
              
                 System.IO.FileNotFoundException: Could not load file or assembly ‘Microsoft.VisualStudio.TestTools.WindowsStore.UITesting’

For CUIT non-Store tests:
            
                 System.IO.FileNotFoundException: Could not load file or assembly ‘Microsoft.VisualStudio.TestTools.UITesting’

 

Detailed Error will look like this:

 
 

Work Around

For each Coded UI test project, please add the following assemblybindging redirection tags to the app.config file. If the app.config file does not exist create a new app.config file with the below contents:

 <?xml version="1.0" encoding="utf-8" ?>
 <configuration>
 <runtime>
 <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.QualityTools.WindowsStore.CodedUITestFramework" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.WindowsStore.UITest.Common" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.WindowsStore.UITest.Extension" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.UITest.ExtensionUtilities" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.WindowsStore.UITesting" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.1.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.WindowsStore.UITest.Playback" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.UITest.WindowsStoreUtility" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.WindowsStore.UITest.Framework" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="UiaComWrapper.WindowsStore" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.QualityTools.CodedUITestFramework" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.UITest.Common" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.UITest.Extension" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="Microsoft.VisualStudio.TestTools.UITesting" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 
 <dependentAssembly>
 <assemblyIdentity name="UiaComWrapper" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
 <bindingRedirect oldVersion="14.0.0.0" newVersion="12.0.0.0"/>
 </dependentAssembly>
 </assemblyBinding>
 </runtime>
 </configuration>

 

 

Please continue to provide your valuable feedback.

Comments

  • Anonymous
    February 27, 2015
    Do we have to run 2013 Test Agent and 2013 Test Controller with Update 4 applied?

  • Anonymous
    March 01, 2015
    Any actually improvements to CodedUI planned for managing UIMaps and Editing action recordings?

  • Anonymous
    March 01, 2015
    @RCook Yes, you need to run 2013 Test Agent and 2013 Test Controller - Update4.

  • Anonymous
    March 01, 2015
    Our team continues to use a mix of Visual Studio editions (2010, 2012, 2013) and later this year I assume some of our developers will move to 2015. Will we have to have all developers upgrade to 2015 if a single developer writes a CodedUI test in 2015 and the other developers want to edit/update it, or by the time you release will codedUI tests be able to be created in 2015, edited in 2013, and then re-opened in 2015, modified in 2010, and then executed on 2015 test agent without any compat issue?

  • Anonymous
    March 02, 2015
    @Tim L No improvements to Coded UI for managing UIMaps and Editing Action Recordings are in the pipeline for VS 2015 RTM

  • Anonymous
    March 04, 2015
    @nathell on your scenario projects are definitely compatible to open in previous VS version and you can always do edits. One best practice is to standardize on the toolset/VS version, if a project is edited in VS 2015 and uses any latest references/framework features then it may not work well on the older versions - so you need to be careful in recommending the tool set/feature set that developers in the team are using.

  • Anonymous
    March 24, 2015
    The comment has been removed

  • Anonymous
    August 12, 2015
    The comment has been removed

  • Anonymous
    August 27, 2015
    Have the CodedUI issues been resolved in the RTM of TFS2015?  We are still unable to get it working.  Is there documentation/tutorials to assist? Robb @chief7

  • Anonymous
    November 30, 2015
    any update on this issue, is work around described here is still the best solution?

  • Anonymous
    December 28, 2015
    Welp, this is still broken. Any update on when VS 2015 will support the GUI testing framework?

  • Anonymous
    February 21, 2016
    When I add the configuration above to my App.config for each project in my solution I get the warning " No test found. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again." Without it I get the warning "Unit Test Adapter threw exception: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified." I tried to change urn:schemas-microsoft-com:asm.v1 to urn:schemas-microsoft-com:asm.v2 as it is recommended here: social.msdn.microsoft.com/.../running-coded-ui-tests-from-mtm-tfs-2015 Any ideas why this happens?

  • Anonymous
    February 21, 2016
    Sorry - my last comment was not 100% clear. I get the same result when I use urn:schemas-microsoft-com:asm.v1 and urn:schemas-microsoft-com:asm.v2. It is always the "No test found...." warning.