Update TestSettings utility

  • Article

In dev10/dev11, test settings in MTM does not expose all the configuration options which are present in Visual Studio. These configuration options include bucket size, deployment timeout among few others.

I have created a small utility named UpdateTestSettings which you can use to update these parameters. To start with, it provides options to update/view the bucket size, update/view the deployment timeout and going forward I will add more options to this list based on your requirements. In case you want to extend it your selves, you can use the 2010 source code from here or 2012 source code from here or 2013 source code from here.

Here are the steps which you need to do to use this utility: -

- Download/extract this utility from here for 2010 version, from here for 2012 version and from here for 2013 version. Ensure that the machine has appropriate version of Visual Studio or Microsoft Test Manager installed.

- Open a command prompt, change the directory to the one where you have extracted this utility and run it using the following syntax.

UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>]

Examples:

UpdateTestSettings /collection:https://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000

UpdateTestSettings /collection:https://abc:8080/tfs/DefaultCollection /teamProjec:myProject /settingsname:My2_0_App

Comments

  • Anonymous
    September 11, 2011
    Hi I need to know about MTM , How it works for performance testing. i have installed it..noe i need to use it for Load and Performance testing need help urgently...!!Digvijay Singh

  • Anonymous
    September 11, 2011
    Digvijay,MTM is not meant for Load / performance testing. You need VS Ultimate for it. You get can VS Ultimate trial version from here(add link) in case you don’t have it.Please refer to below walkthroughs for details on how to do Load / Performance Testing using VS Ultimate:Web test Walkthrough:msdn.microsoft.com/.../aa337591(v=VS.100).aspxLoad Test Walkthrough:msdn.microsoft.com/.../ms182594(v=VS.100).aspxAnd you can post your questions around Load / Performance testing on this forum.social.msdn.microsoft.com/.../threadsRegardsAseem Bansal (on behalf of Rubel)

  • Anonymous
    December 18, 2011
    After the settings, do we need to restart the Microsoft test manager?

  • Anonymous
    December 19, 2011
    Nikhil, The settings page in MTM should be refreshed to get the updated settings from the server. I think that should be enough and restart of MTM should not be required.RegardsAseem Bansal

  • Anonymous
    December 19, 2011
    The comment has been removed

  • Anonymous
    December 19, 2011
    To update the script timeout, you should modify the code of this utility as discussed in the below thread.social.msdn.microsoft.com/.../5a6f3af4-9e28-42d9-8830-f800969d8b28RegardsAseem Bansal

  • Anonymous
    December 20, 2011
    Thanks a lot Aseem. It is working now.

  • Anonymous
    July 16, 2012
    Thanks

  • Anonymous
    December 19, 2013
    Can you please tell me what is deployment time out? what is the use of it?

  • Anonymous
    December 19, 2013
    The testcontroller deploys the test binaries and their dependencies from the build drop location to the test agent machines so that test agent can run them. The timeout for that activity is called deployment timeout.

  • Anonymous
    January 24, 2014
    The comment has been removed

  • Anonymous
    March 31, 2014
    Aseem:  Since your app works on versions < 2013, I need to try to create an updated version.  Can you point me to anything that describes where the Bucket Size property is exposed in the TFS api?  I appreciate your help.

  • Anonymous
    May 03, 2014
    Question:  #1. I know in TFS Build on VS, we can updated the parallelTestCount attribute which allows to run multiple tests, but I do NOT find this attribute exposed in the TestRunConfiguration class. How can we configure so that multiple tests run on a given agent.#2.In MTM test environment with multiple test agents, can the tests run in Pararallel on a test agent.Eg: Test Env1: 5 test agents, 1500 test cases. I know that each agent will be running 1 test at a time. So at any given time, we have 5 tests running.Instead can I run 10 tests on test agent, so a total of 50 will be running at any given time??I know in TFS Build on VS, we can updated the parallelTestCount attribute which allows to run multiple tests, but I do NOT find this attribute exposed in the TestRunConfiguration class.

  • Anonymous
    June 09, 2014
    The comment has been removed

  • Anonymous
    June 09, 2014
    Hi Aseem,The right side text in your article is being cut. Could you please mention again the command you are using for the utility. And what is bucket size?Thanks,Geetika

  • Anonymous
    June 09, 2014
    and I have VS 2013. Will it work with this one? Or should I install 2012?

  • Anonymous
    June 09, 2014
    Here you go: -UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>]Examples:UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000Secondly it should work with VS 2013 as well but you might have to recompile the sources with 2013 which should be simple.RegardsAseem Bansal

  • Anonymous
    June 12, 2015
    Hi Asim, This tool doesnt seem work with TFS2013, we just upgraded to TFS2013 and i am trying to set the bucket size to 1, we have 100 clients lab environment but test controller distributes tests only to last 7 clients instead of all 100. ( it was working well with 2010). lets say we have 500 test cases, it distributes one test to each of the client. once client finishes test it takes another test from test controller. can you help to find out whats going on? Thanks Srini

  • Anonymous
    June 13, 2015
    Srini there are no changes in the distribution logic. If you can share your testsettings file it would help. Beyond that you can check the status of your agents prior to the run and see if that shows any hints of any issue. Another thing to check is if theses last 7 clients have something in common - in comparison to the others. That should also help us correlate back to what we know. if none of this is providing you with any help. I would recommend enabling the test controller traces and then starting your run. The traces should have data if any agent is being skipped.

  • Anonymous
    June 20, 2015
    Hi Allen, Now its working after i put the app key settings MinimumTestPerAgent as 1 in test controller. Now we are having different issue, after setting up the environment when i connect to machines thru lab viewer after certain time the machines gets locked, we have to enter password to open it again. we are using windows 2012R2 as hyper-v servers and clients machines are virtual machines with win7 on it. anyone experienced this issue? Thanks Srini

  • Anonymous
    July 08, 2015
    Is there overhead or poor performance impact associated with setting the Bucket Size to a very small number such as 1 or 2?

  • Anonymous
    January 20, 2016
    I have added ScriptTimeout to the tool and made it available here. you can use as follows UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>] [/scriptTimeout:<value>] Examples: UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000 UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000 /scriptTimeout:900000 UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProjec:myProject /settingsname:My2_0_App /scriptTimeout:900000

  • Anonymous
    January 20, 2016
    link for downloading it here.. gallery.technet.microsoft.com/Update-Script-Timeout-in-c40af3ee UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>] [/scriptTimeout:<value>]

  • Anonymous
    January 27, 2016
    gallery.technet.microsoft.com/Update-Script-Timeout-in-c40af3ee doesn't go anywhere for me

  • Anonymous
    January 27, 2016
    How do I delete comments? that link is actually fine.

  • Anonymous
    January 27, 2016
    Is there a way to do this without TFS? My test rig is made up of VS2013.5 test controller and agents, no TFS, but I'm still getting script timeouts because my application takes exactly 7 minutes to start. How do I increase the script timeouts in my .testsettings files?