Use Data-Driven Test for Parameterized Tests

  • [アーティクル]

In daily development, we usually have a requirement to write paramterized test cases. I used to use the following two simple methods Method 1 and Method 2 to accomplish such requirments, but their short comings are obvious. The first method works for few variations, but it is ineffient if there are too many parameters. The second method need additional logics for error handlings, otherwise the former failed case would break the entire case. But it is still hard to trouble shoot such cases as all messages are huddled into the same case.

This article aims to introduce the third way, which takes advantage of Data-Driven Test for such requirement.

The following Method 3 is a sample to use Data-Driven Test. It would logically split a test case into several sub test cases. To some extend, such method doesn't have the short comings in the former two methods.

But I think there still exists some open issues for Method 3. Would further investigate whether there is a better solution.

  • The first issue is the test case display name. With method 3, the sub test cases' names would be like "TestTransfer (Data Row 0)" and "TestTransfer (Data Row 1)", which are not informative.
  • The second issue is that the xml file may be also duplicate, if I have several kinds of parameters and want to test all combinations of them.
    • E.g. if there is another parameter "DataType", the xml file would be like "1024, txt", "1048576, txt", "1024, jpg", "1048576, jpg".

 

Method 1: Create new test methods for each variation. E.g.

 

        [TestMethod()]
        public void TestTransfer_1KB()
        {
            TestTransfer(1 * 1024);
        }

        [TestMethod()]
        public void TestTransfer_1MB()
        {
            TestTransfer(1 * 1024 * 1024);
        }

        private void TestTransfer(long dataSize)
        {
            // ...
        }

 

Method 2: Put all variations into a single method. E.g.

 

        [TestMethod()]
        public void TestTransfer()
        {
            var allDataSize = new long[] { 1 * 1024, 1 * 1024 * 1024 };
            foreach (var dataSize in allDataSize)
            {
                TestTransfer(dataSize);
            }
        }

 

Method 3: Data-Driven Test. E.g.

        public TestContext TestContext { get; set; }

        [TestMethod()]
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "TransferTest.xml", "Transfer", DataAccessMethod.Sequential)]
        public void TestTransfer()
        {
            TestTransfer(long.Parse(TestContext.DataRow["DataSize"].ToString()));
        }

 

TransferTest.xml:

 <Settings>
  <Transfer>
    <DataSize>1024</DataSize>
  </Transfer>
  <Transfer>
    <DataSize>1048576</DataSize>
  </Transfer>
</Settings>