Motley says: "Just use a batch file to run unit tests"

  • Article

Summary

 

Motley:  Just use a batch file to execute unit tests - it's easy and quick.

 

Maven: Use a unit test framework to write and execute unit tests. A framework makes tests easy to write, executes tests quickly and consistently, and provides features that make your life easier, such as logging.

______________________________

 

[Context: Motley was talking to a co-worker about unit testing]

 

Motley: I was talking with Marty the other day about unit testing. I was telling him that I was using the built-in Visual Studio unit test framework for my testing. He indicated I was nuts. After I popped him in the nose for calling me names, I let him finish.

 

Maven: Why would he think you're nuts, besides the obvious reasons?

 

Motley: <pow>

 

Maven: Ouch, dude! What was that for?

 

Motley: Calling me names! Anyway, he was telling me about his method of compiling tests as executables  adding tests to a batch file, and executing the batch file. He says it's much faster and a big time saver.

 

Maven: I beg to differ! Using a test framework like Visual Studio has all sorts of advantages, including quick, integrated test execution.

 

Motley: Are you using the word "quick" and phrase "Visual Studio unit test execution" in the same sentence? You need to kick your drug habit.

 

Maven: Granted, Visual Studio 2005 was not super efficient in terms of test execution, but boy, you should see Visual Studio 2008! Big improvement. In any event, it's still quicker overall than using a batch file.

 

Motley: The fact that test execution is integrated into the IDE is definitely a bonus, and tests run reasonably quick after the framework startup has cached its state. However, batch files can still be quicker don't you think? And what if I don't want to use the VS framework?

 

Maven: I doubt it that batch files are quicker. Think about the context switch in moving from an integrated environment to a command-line environment. It's not very efficient over the long haul. Plus, how do you know the tests were successful? You either have to rely on a kludgy exit code or watch the results. We need something more robust than that.

 

Motley: Fine. I see a few benefits to using a test framework. What else?

 

Maven: Writing the tests. A good test framework like Visual Studio or NUnit makes it real easy to not only execute tests but write them. For C# there is a well-defined set of attributes that you can use to tell the test framework how to execute tests. Then the framework takes over, reflects on the compiled assembly, runs the tests you tell it to run, and reports results. It becomes obvious whether your tests pass or fail after execution, and most frameworks will give you a nice log of your results.

 

Motley: But if I just create executable code it's pretty easy to write the tests.

 

Maven: Yes, but not as robust. A test framework works consistently across all tests and reports results uniformly. You don't have to remember to set return codes and check them in a batch file, for example. All the common infrastructure is encapsulated behind the framework. This allows you to concentrate on writing your tests, which is where your attention should be. Plus, you can write tests using a common pattern.

 

Motley: What pattern would that be?

 

Maven: Most test frameworks support the following pattern for writing tests:

  1. Setup a suite of tests: Write some code that sets up the execution of a set of tests. Tests are contained within a class, and an attribute like [ClassInitialize] is used to run initialization code that is common across all tests.
  2. Setup each individual test: An attribute like [TestInitialize] is used to act as a sort of constructor for each test. Perhaps each test needs some test class member variables to be initialized prior to test execution. Here is a good place to put that code.
  3. Write the test: A test method is typically a method with no parameters that returns void. An attribute like [TestMethod] tells the framework that the method is a test. You use a built-in test framework object like Assert to determine whether a condition passes or fails, and this tells the test framework what the result is.
  4. Clean up each individual test: An attribute like [TestCleanup] is used to act as sort of a destructor for each test. Use it to clean up anything that you set up in the [TestInitialize] method.
  5. Clean up the suite of tests: The [ClassCleanup] attribute is called after the execution of all tests to clean up whatever you set up in the [ClassInitialize] method.

 

Motley: But you could do that yourself with an executable and a batch file.

 

Maven: Yeah, you could. But why do a bunch of extra work, have a less robust solution forcing more test debugging, and avoid using a well-implemented framework that is regularly enhanced and maintained and does some work for you?

 

Motley: Okay, okay. I'm heading over to Marty's desk right now...

______________________________

 

Maven's Pointer:  It's not important what framework you choose - even if its homegrown - but it is important to use a well-written framework. Visual Studio and NUnit are great choices for the .NET platform. If neither of those meet your needs, feed requirements back to the developers. If you decide to implement your own, sticking to the pattern mentioned above will ease the learning curve for test writers.

 

Maven's Resources: 

Comments

  • Anonymous
    September 24, 2007
    The comment has been removed