Testing Against The Passage of Time

  • Article

This is the fourth in a small series of posts about testing against non-determinism. In this installation, I'm going to cover the passage of time.

In my former post, I demonstrated how you can use the Provider Injection pattern to decouple your test target from a direct dependency of the current system time. In unit tests, the DateTime provider will serve up whatever value is defined by the test developer, while in production, the provider will serve up the real system time.

You can use this ability to control the apparent system time to greatly speed up time-dependent code. Although I must admit that it's not everyday that I write code that must execute for a particular length of time, I have run into this scenario from time to time, and each time I've used this principle to accelerate time during testing.

Consider this simple example:

 public void DoSomeWaiting()

 {

     TimeSpan waitTime = TimeSpan.FromSeconds(30);

     DateTime initialTime = DateTime.Now;

  

     while (DateTime.Now < initialTime.Add(waitTime))

     {

         Thread.Sleep(TimeSpan.FromMilliseconds(10));

     }

 }

In a real-life situation, you will probably have a reason for doing something like this, but the main point is that this method simply blocks for 30 seconds before it returns. Since every invocation of DoSomeWaiting takes 30 seconds, this method is a pain to unit test - particularly if you are using test-driven development (TDD). Each unit test will take at least 30 seconds, and if you have a suite of tests, you can multiply this period with the number of tests involving this method. Such a suite may easily take several minutes to execute.

For an automated build verification test, this may not be a big deal, but in TDD this is disastrous, since it totally destroys the fast-paced code-compile-test development cycle.

To amend this problem, you can alter the implementation of the method. If I add the method to the TimeConsumer class from my former post, I can implement it in this way:

 public void DoSomeWaiting()

 {

     TimeSpan waitTime = TimeSpan.FromSeconds(30);

  

     DateTime initialTime = this.dateTimeProvider_.Create("Now");

     while (this.dateTimeProvider_.Create("Now") < initialTime.Add(waitTime))

     {

         Thread.Sleep(TimeSpan.FromMilliseconds(10));

     }

 }

Instead of using DateTime.Now, I use an externally supplied ServiceProvider<DateTime> to get the 'current time' (as usual, I should point out that although I use Service Locator 2, you can also apply the principle in other ways, for example with IServiceProvider implementations).

With this implementation, I can now write a unit test that will execute orders of magnitude faster:

 [TestMethod]

 public void PerformAcceleratedWait()

 {

     ServiceProvider<DateTime> dateTimeProvider =

         new ServiceProvider<DateTime>();

     dateTimeProvider.Preset(new DateTime(2007, 5, 7), "Now");

  

     TimeConsumer tc = new TimeConsumer(dateTimeProvider);

  

     Stopwatch watch = new Stopwatch();

     watch.Start();

  

     ThreadPool.QueueUserWorkItem(delegate(object state)

     {

         Thread.Sleep(TimeSpan.FromMilliseconds(10));

         dateTimeProvider.Preset(new DateTime(2007, 5, 7, 0, 0, 31), "Now");

     });

  

     tc.DoSomeWaiting();

  

     watch.Stop();

  

     Assert.IsTrue(watch.Elapsed < TimeSpan.FromSeconds(30));

 }

Instead of having to wait for 30 seconds, this unit test executes much faster: On my system, watch.Elapsed tends to be slightly higher than 10 ms!

Being able to run a suite of tests where you can compress arbitrary periods of time down to almost no time is a great addition to your tool belt. If you are curious about this technique, I used it quite extensively for the StarShip quickstart for Service Locator 2. In those tests, I typically compress hours into a few milliseconds of test time, so it sure saved me a lot of time.

Comments

  • Anonymous
    May 14, 2007
    Although computers tend to be rather deterministic in nature, you will sometimes have to deal with concepts

  • Anonymous
    May 14, 2007
    Why there is no special "Debug" version of .NET framework that will enable scenarios like this one ? As well - why developers has to solve testing problems each time in their code making it more complicated even in production environment (and more buggy as result) - and not using some "hackery" tools that will enable solution for testing once and forever ? One of example of such a tool will be dynamic injection/rewriting of methods like DateTime.Now. I.e. code can looks like using(Injection inj = Injection.Create<DataTime>()) // No args for static, or with args for specific instance { inj.Property("Now") = delegate() {  return new DateTime(2007,1,2,3,4,5); }; // Call do stuff and other } // at end of injection (Dispose/Stop method) - everything is reverted back to normal It's strange to hear from platform vendor that there is no tools/features for easy testing.

  • Anonymous
    May 14, 2007
    Hello TAG Thank you for your comment. It wouldn't be a very good idea to have a special debug version of the framework that would automatically enable scenarios like this one, since obviously, you don't want to test your debug code; you will want to test your release code. Still, something close to the example you give can be achieved with TypeMock. However, most agile developers (including myself) tend to shy away from TypeMock, since it tends to make software designers lazy. The approach outlined above is not "hackery". In fact, it follows some very well-known and accepted design patterns, that has a lot of benefits. Forcing software designers to model Dependency Injection into their API is very beneficial for a number of reasons. Testability is only one of those benefits, but another very important benefit is proper isolation between different parts of an application, enabling multiple teams to work independently of each other. Extensibility is a third benefit that usually follows. TypeMock, as well as the code you propose, don't enforce good design practices. In fact, it enables you to write some pretty sloppy code and still get away with unit testing, but isolation, extensibility, etc. goes right out the window.

  • Anonymous
    May 16, 2007
    Well. You has thrown so many buzzwords on me assuming that I know all of them and will be able to understand how they apply in this situation. Your approach for testing is possible only in case if you control entire application code. But just imagine some third-party API that already was supplied to you and not using  Provider Injection patten - but use plain DateTime.Now. This can be anything like Infragistics or Microsoft own libraries. It's good for Microsoft developer to talk about   isolation and multiple teams then you control entire framework. But for rest of world we have something that Microsoft give us and we have to spend half a hour (or more) and add failure points in our production code for such a simple scenario like DateTime.Now. I understand you objections on creating reusable library or custom framework build - Microsoft developers does not matter that to code either it or custom workaround. But for rest of world - once you will create generic library - it will make big difference.

  • Anonymous
    May 21, 2007
    took me a second glimpse to realize what you were doing. for TAG, you could use the same concept applied in this code, but changing the actual time (if .Now was used in the actual code being tested) all you would have to do would be to save the original time before starting the test, then restore it at the end (using a try..finally), I have used that technique before, I guess I should blog about it

  • Anonymous
    May 21, 2007
    The comment has been removed

  • Anonymous
    May 21, 2007
    Hi Eber Thank you for your comment. As far as I understand, you are talking about changing the actual system time? While this is certainly possible, I wouldn't recommend it, since it may cause funny things to happen on your system in general. Imagine a build server that executes builds and build verification tests (BVTs) at scheduled times. Changing the system time in a BVT may cause a new build to be initiated, while the first is still running. All sorts of other weird things are also likely to happen. I definitely wouldn't want any test suite to do such a thing on my workstation. In addition, you need to be an Administrator to change the system time, so this approach will also require your test suite to run with Administrator privileges, which means that you will not be able to test your code with reduced privileges. Finally, I think this approach will be more resource intensive. In my example, I'm able to compress 30 seconds into 10 ms, but I doubt that you can change the system time twice within 10 ms; but I may be wrong on that point.