Putting code examples where my mouth is
I back up the high level preaching with code.
I have had requests to back up the “high level concepts” I am talking about with concrete examples. This is an example of sticking with the simplest possible automation versus using a more “perfect” solution.
Test automation should be optimized for a simple set of goals.
In a nutshell I advocate making tradeoffs that make your test automation.
• Simple
• Fast
• Deterministic
• Diagnostic
• Maintainable
There are many benefits
You can deliver more tests earlier. Your test automation is easier to understand and hand off. Most of all, this pattern has worked for me in the past when other patterns failed.
On to the Examples
I prefer to create 10 or 100 tests that very narrow and tied to the code versus spending the same amount of time to create one test that’s very general and handles all the possible errors the code could generate.
a very simple test that is narrow and in this case makes assumptions about the install path:
Cost to dev and test 10 minutes
public void MyFeatureIsInstalled()
{
string defaultFile = @"C:\Inetpub\wwwroot\default.aspx";
bool isDefaultInstalled = System.IO.File.Exists(defaultFile);
string message = string.Format("{0} exists = {1}", defaultFile, isDefaultInstalled);
Assert.IsTrue(isDefaultInstalled, message);
}
Example of the same test that’s more flexible, but may not be worth the time to create. This is based on real tests that I have seen in groups around Microsoft. Methods highlighted in bold are further code that would need to be written and tested. The complexity this type of “framework” adds almost always ends up making SDETS less productive rather than more productive in practice. There are a lot of hidden taxes.
Cost to dev and test 60 minutes + extra time for the framework.
public void MyFeatureIsInstalled()
{
string defaultFileLocation = getInstallLocationFromRegistry();
string defaultFilename = TestHelperMethods.GetNamedParameter("defaultFileName");
if (String.IsNullOrEmpty(defaultFileLocation))
{
defaultFileLocation = TestHelperMethods.GetNamedParameter("defaultFileLocation");
}
if (String.IsNullOrEmpty(defaultFilename))
{
Log.Fail("Unable to get default file name from test parameter system.")
}
string defaultFile = string.Format(@"{0}\{1}", defaultFileLocation + defaultFilename);
bool isDefaultInstalled = System.IO.File.Exists(defaultFile);
string message = string.Format("{0} exists = {1}", defaultFile, isDefaultInstalled);
Assert.IsTrue(isDefaultInstalled, message);
}
Know when to use the various tools you have
I am not saying you never invest in the second type of automation. What I am saying is that defaulting to the perfection style will slow you down unnecessarily. Test code has a different mission than product code.
Deliver some simple, fast to write tests to give your team some automation cover before you invest in flexible parameter driven methods.