Publish test runs against lab default templates

  • Article

Last week one of the internal customer mentioned that he has customized the lab default template in such a way that instead of running the test cases on the lab machines using our built-in flow of test controller/agent, it was launching mstest.exe directly on the lab machines from the build controller/agent. The tests were executing fine and results were visible in the build logs but the summary section did not have the test results and the customers main question was how to bring the test results also in the summary section.

The customer had tried out using the publish command available in mstest.exe and had published the results against the right platform/flavor of the lab build but still it was not showing the summary section. This was because the lab summary page does not look at the unit tests results while preparing the summary.

The lab summary page can only show the results from the test runs that are published against a test plan. So we asked the customer to create a place-holder test plan and publish the test results against that plan. Once the customer started doing that, the results were visible in the summary section. 

Here are the exact steps that customer did here: -

  • Created a place-holder test plan, test suite and test configuration. This was a one time activity and was done manually.
  • Started refreshing the test cases in the test suite with every build so that any test methods added/removed can be updated in the test suite. Since this was a recurring activity, customer modified the customized workflow and added tcm import command.
  • Started publishing the trx against the test plan. Since this again a recurring activity, tcm publish command was added to the workflow.
  • Added the details of the published test run in the step above, into the build so that build summary can show it. Along with the above published command, following script was added.

# Load Client Assembly
[Reflection.Assembly]::Load(“Microsoft.TeamFoundation.Client, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”);
[Reflection.Assembly]::Load(“Microsoft.TeamFoundation.Build.Client, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”);
[Reflection.Assembly]::Load(“Microsoft.TeamFoundation.Build.Common, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”);

# Define parameters
$tfsCollectionUrl = ' https://myserver:8080/tfs/DefaultCollection';

$projectName = ‘myproject’;
$buildUri = New-Object System.URI('vstfs:///Build/Build/187');
$testRunId = '157' $message = 'Foo suite'

# Define the summary node constants
$deploymentType = [Microsoft.TeamFoundation.Build.Common.InformationTypes]::DeploymentInformation;
$deploymenFieldType = [Microsoft.TeamFoundation.Build.Common.InformationFields]::DeploymentInformationType;
$testType = [Microsoft.TeamFoundation.Build.Common.DeploymentInformationTypes]::Test;
$urlFieldKey = [Microsoft.TeamFoundation.Build.Common.InformationFields]::Url;
$messageFieldKey = [Microsoft.TeamFoundation.Build.Common.InformationFields]::Message;

# Find the build
$tfsCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($tfsCollectionUrl);
$buildService = $tfsCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer]);
$build = $buildService.GetBuild($buildUri) ;

$parentNodes = $build.Information.GetNodesByType($deploymentType, $true);

# Create the test run summary node
$parent = $parentNodes[0]
$newNode = $parent.Children.CreateNode();
$newNode.Type = $deploymentType;
$newNode.Fields.Add($deploymenFieldType, $testType);
$newNode.Fields.Add($urlFieldKey, $testRunId);
$newNode.Fields.Add($messageFieldKey, $message);

# Add the test run summary node to build
$parent.Save();

Also note that customer had extracted the test run Id from the output of TCM.exe using something like this.

$testRunLine = TCM.exe .... | Select-String "Test run created:"

if ($testRunLine)
{
$testRunLine = $testRunLine -Replace("Test run created: ","")
$testRunLine = $testRunLine -Replace(" ","")
$testRunId = $testRunLine

  Write-Host "Id:" $testRunId
}

Enjoy !!