How To: Do image comparison in Coded UI Test

This is guest blog by my colleague Rajeev Kumar.

The Coded UI Test does not provide the API to compare images. There are cases when user wants to verify the changes in the UI by comparing the images of two controls. This can be easily achieved using Test API with Coded UI Test. The overview of Test API is described here and Test API visual verification is described here.

Getting the Test API

• Download the Test API from here.
• Add reference of TestAPICore.dll in the Coded UI Test project.
• Add this using statement in your code-  using Microsoft.Test.VisualVerification;

General Test API terms

• Snapshot: A pixel buffer used for representing and evaluating screen image data.
• Verifier: An object which determines whether a snapshot passes against specified inputs.
• Actual: The snapshot being evaluated.
• Master: The reference data (image) which is used to evaluate the actual snapshot.
• Tolerance: The accepted bounding range based on which the actual snapshot will be accepted as valid.

The general Visual Verification workflow

• Search the UI Test Control.
• Capture the image of UI Test Control.
• Generate an expected snapshot (e.g. load a master image from disk, etc.)
• Compare the actual snapshot to the expected snapshot and generate the difference (diff) snapshot.
• Verify the diff using a verifier.
• Report test result.

Sample code to use Test API.

• From the Test API copy these files to Coded UI Test:
  • Samples\MSTest\Tests\bin\Debug\Master.png
  • Samples\MSTest\Tests\bin\Debug\ToleranceMap.png
  • Samples\MSTest\Tests\bin\Debug\SampleApp.exe
• Add these files in the Coded UI Test project and make “Copy to Output Directory = Copy Always” in the property window for each files.
• Now add this test method the Coded UI Test.

 [TestMethod]
[DeploymentItem("Master.png")]
[DeploymentItem("ToleranceMap.png")]
[DeploymentItem("SampleApp.exe")]
public void CompareImagesTest()
{
    // Launch the application.
    using (ApplicationUnderTest app = ApplicationUnderTest.Launch(
                Path.Combine(Directory.GetCurrentDirectory(),
                "SampleApp.exe")))
    {
        // Select the check box, so that the background will change.
        WpfCheckBox checkBox = new WpfCheckBox(app);
        checkBox.Checked = true;

        // Get the actual, master and toleracne snapshots.
        Snapshot toleranceMap =
                Snapshot.FromFile("ToleranceMap.png");
        Snapshot master = Snapshot.FromFile("Master.png");
        Snapshot actual = Snapshot.FromWindow(app.WindowHandle,
                WindowSnapshotMode.ExcludeWindowBorder);

        // Get the difference of two images.
        Snapshot difference = actual.CompareTo(master);

        // Save the file to for manual viewing.
        master.ToFile(@"Master-expected.png", ImageFormat.Png);
        actual.ToFile(@"Master-actual.png", ImageFormat.Png);
        difference.ToFile(@"Master-difference.png",
                            ImageFormat.Png);

        // Use SnapshotToleranceMapVerifier to verify the comparison.
        SnapshotVerifier verifier =
                new SnapshotToleranceMapVerifier(toleranceMap);
        Assert.AreEqual(VerificationResult.Pass,
                verifier.Verify(difference));
    }
}

Description of the code

• Launch the sample application.
• Check the check box on the sample allocation so that the app changes its background color.
• Create the Snapshot object for master, actual and tolerance map.
• Compares the actual image with the stored copy of master image.
• Saves images files for manual diagnosis (in the test fails, then these images are useful)
• Then verifies the differences of two images are within tolerance as given by tolerance map image.

This was guest blog by my colleague Rajeev Kumar.

Comments

• Anonymous
  April 08, 2010
  Wow seems remarkably like my Mix demo...

• Anonymous
  April 08, 2010
  Yes, that was an inspiration for this. We just built integrated scenario with CUIT on this.

• Anonymous
  June 09, 2010
  Thanks for the post! One question, what is the process to generate a tolerance map?  

• Anonymous
  July 12, 2010
  Sorry for late reply.  For some reason, my blog settings got messed up and I had trouble login in.  In general, the Coded UI Test forum (social.msdn.microsoft.com/.../threads) is better place to post your questions as that is monitored by many folks & you will get faster response. Today you have to generate the tolerance map by hand.  Please refer the TestApi documentation for this. Thanks, Gautam

• Anonymous
  July 27, 2012
  Gautam I really appreciate your effort,help,dedication,understanding,time management.................I really really thankful to you and your understanding!

• Anonymous
  August 20, 2012
  Hi Gautam, Can you please help in assisting how the ToleranceMap.Png gets created by hand. Thanks, Suman

• Anonymous
  August 22, 2012
  Check the Test API documentation - http://testapi.codeplex.com/