Typemock

Roy recently posted his thoughts on Typemock - whether it's too powerful and will kill TDD. His conclusion seems to be a confident no, and he argues well, but I have to disagree with some of his points.

However, first of all, I have to state that I find Typemock an amazing product that definitely has a raison d'être. If I ever find myself in the situation where I need to deal with an (otherwise) untestable legacy application, I would definitely use Typemock to develop a suite of unit tests that could work as a safety net while I refactor the application towards testability. In that scenario, Typemock can help, and as far as I'm aware, it's the only product on the market that can do what it does.

Incidentally, that covers Roy's argument number 3, and since I agree, I'm not going to spend much more energy on that. His other arguments, hower, I think warrant some scrutiny.

"You can still design for testability and use Typemock"

Yes, I agree, but I can also still design for testability and use Rhino Mocks, so why should I use Typemock? TDD and how to achieve proper isolation between dependencies are techniques deeply ingrained into the way I write code, but still, why would I want to waste my brain power constraining myself instead of having a tool do that?

My family and I recently spend an entire month on vacation in Florida, and since it was warm and you can buy excellent ice cream, I came back larger and heavier than when I left. I want to get rid of that extra weight again, and to do so, I try to eat more healthy, and particularly no ice cream, chocolate, etc. Now, it would be more 'powerful' for me to keep those items in the house, but that would mean that I would have to exert much more self-discipline to not eat it when the need arises. Instead, I voluntarely choose to constrain my situation by making sure such items are not to be found in the house.

This situation is entirely analogous to the choice between Typemock and other, more traditional, mocking frameworks. I don't want to pick a tool that's too powerful for the job, and then spend energy disciplining myself, when I can pick another tool that does this automatically for me.

"If you were doing bad design before, the tool won't help you, just the technique."

Okay, but if that's true, Typemock will not help me, either.

In any case, I don't believe that this statement is true. The very benificial constraint introduced and enforced by a classic mocking framework can force you to think about things you wouldn't otherwise have considered.

Back when I started with TDD I was a fairly experienced (5 years) developer, but I still had a lot to learn (I still do). I'd heard about unit testing, the test-first principle and something called NMock, and that was about how much I knew about TDD and designing for testability. During development of a fairly complex piece of software with a lot of weird business logic, I repeatedly had a very enlightening experience: I designed a bit of the API to address a business problem only to discover that I couldn't test it. That invariably caused me stop and rethink my design, and I always came up with an alternative that allowed me to test my code. As I've later learned, I simply discovered a lot of the patterns in xUnit Test Patterns by myself.

The point is that the tool helped me become a better software designer. This may be the case for some developers, while others will probably just give up on the way - I'm not saying that it's a one-size-fits-all solution, but that there are situations where the tool actually helps you become a better developer (FxCop/Code Analysis is another good example). If you approach the tool with an open mind, it can help you!

"Sometimes you just can't refactor your legacy code into testability."

As I wrote above, I totally agree, and Typemock is an excellent choice in that case, but that doesn't mean that Typemock is the best choice in other scenarios. Dealing with legacy code is a situation distinctly different from greefield development, and I don't believe that the choices of tool, strategy, technique etc. logically carries over from one category to the other.

"Pair Programming, Test reviews and Design reviews trump any tool"

No argument there, but again, this just put Typemock on par with all the other mocking frameworks out there.

"If you're not sure how you'd design something and test it, you can always ask!"

Yes, and you are also welcome to ask me about design for testability.

However, to pose a question at all requires that you realize that you have a problem in the first place. Since Typemock can mock anything, you may be happily chugging along without realizing that there's something fundamentally flawed about your design. If that's the case, why would you ask a question about testability in a forum?

What I like about traditional TDD is that it stops you if you are trying to do something wrong, and that should prompt you to ask!

In short, I'd currently pick Rhino Mocks for all greenfield development. For dealing with legacy code, I'd pick Typemock.

Comments

• Anonymous
  March 04, 2008
  PingBack from http://msdnrss.thecoderblogs.com/2008/03/04/typemock/

• Anonymous
  March 04, 2008
  Thanks for the thoughtful arguments. Here's an interesting idea: what if Typemock had a "mode" (like mocks have 'strict' and 'non strict' modes) where you could only do "classist" mocking - no statics etc.. if you were using anything not "classist" you'd get an exception that this is not allowed which would force you t stop and think. then you'd have a "free" mode where you could do anything you want (when working on legacy code for example) would that help? and, Typemock is not "on par" with rhino mocks, since it has other features such as IDE and debugger integration and such. Roy

• Anonymous
  March 04, 2008
  Hi Roy That would certainly help, although I'm not sure such a feature would turn me into a Typemock convert :) It elegantly solves all the issues I mentioned in my post, but I purposedly didn't touch on a couple of other obstacles to adoption that I find is more political or mercantile than strictly technical:

• Typemock costs money, although I know there's a feature-limited free edition available as well.
• Typemock needs to be installed, so you can't just throw the binary into your source control system and automagically make it work for all people who can figure out how to do a 'get latest'. On the plus side, however, is that if an organization already uses (and pays for) Typemock for legacy development, they can reuse it for greenfield development with the benefit of being comfortable with the tool and its syntax, as opposed to having to learn a separate API, while still harvesting all the benefits of traditional mocking. It is beginning to occur to me that my general attitude in this whole debate is probably flavoured by the fact that I've been doing greefield development all my career :)

• Anonymous
  March 05, 2008
  Brilliant idea Roy. I think that's exactly what TypeMock needs.

• Anonymous
  March 06, 2008
  And now I am a classic, too?

• Anonymous
  March 06, 2008
  I think that it's perfectly valid to use TypeMock in a non-legacy codebase. I'm all for decoupling but I got sick to the back teeth of extracting interfaces from every class that I wanted to mock (and I'll ignore the virtual method approach). I was producing interface-implementation pairs that certainly didn't clarify my design, weren't useful in production and were not useful extensibility points. These days I'm far more in favour of informed decisions on decoupling.

• Anonymous
  March 06, 2008
  Hi Colin Thank you for your comment. Apart from the many other things that bother me about that approach is that you make your unit tests dependent on a particular tool (Typemock) even when you don't have to (greenfield development). An overuse of mocks can lead to overspecified tests, so in fact I tend to use mocks sparingly. They are a tool in my toolbelt, but I use stubs and fakes at least as much as I use mocks, and that wouldn't be possible if I didn't have the proper design in place.

• Anonymous
  March 07, 2008
  Mark, I would say that , at the moment, each mocking framework has such a different syntax, that once you start using one (rhinomocks, nmock or typemock) you're pretty much committed to it because changing all the mocking code would be too hard, so I don't agree with that sentense. I agree that overspecified tests are bad, but that has nothing to do with what framework you use. you could do overspecified tests with manual mocks, fakes and stubs just as easily. It sounds like your objections to typemock are based mostly on feeling, not on fact (assuming my suggestion from the comment before were used, you'd still not use it because it uses 'arcane' clr apis not meant for this - how is that a valid argument?)

• Anonymous
  March 07, 2008
  Hi Roy Now you are putting words in my mouth. AFAIR, I never used the word 'arcane' in this discussion. Regarding the interchangability that proper design for testability provides, I'm not talking about changing one mocking framework for another. What I'm talking about is that for each test that you write, you have to write it correctly. For some tests, a mock object is the correct choice, but for other tests, a fake may be a better choice, if you want to avoid different kinds of unit testing anti-patterns. You can only use a fake (or a stub) if your API is truly testable, because it has nothing to do with how a particular mocking framework works. That's why I disagree with Colin. What I've been saying all along is that Typemock makes sense if you have legacy code. If you already made the investment (both in time and money) in Typemock for that reason, and given that your suggestion above is implemented, I have no objection to use Typemock for greenfield development as well. On the other hand, for people who only do greenfield development, I don't see what Typemock offers over, say, Rhino Mocks. Given that Rhino Mocks is free and much easier to install and move around, I'd choose that instead. That is, in my book, a pretty rational argument.

• Anonymous
  March 07, 2008
  First: to quote your comment on Ayende's blog: "as opposed to Typemock, which relies on an arcane API of the .NET framework which was originally created for an entire different purpose." Second: Talking pure framework, Typemock offers several things over the other frameworks, regardless of legacy code: Debugger evaluation and highlighting of mocked methods, mocking chained calls (a.b().c()) in one swoop, having he ability to swap arguments at runtime and more. These are pure framework features, which work just as well on greenfield code. In that regard, forgetting the fact that you can also test any "non testable" class in your legacy code, those are some of the reasons Typemock costs money. Also, don't forget support around the clock, guerneteed, which is something which you do get with the other frameworks but is purely based on oren's time, for example. I'm surprised you made such claims without checking and actually comparing the framework features first. This is hardly like you. Roy

• Anonymous
  March 07, 2008
  Regarding the use of 'arcane', in retrospect I admit that it sounds dismissive, but I only used it in the dictionary sense of the word: "Known or understood by only a few", which I think describes the Profiling API pretty well :) In any case, I didn't mean it to carry a particularly negative connotation: I am very impressed just by the idea of using the Profiling API for mocking purposes and full of awe that it has been possible to pull it off. Regarding your second point, I don't think any of us ever make decisions with perfect knowledge of the entire subject - at least, I don't, but I normally try very hard to keep an open mind and listen attentively to rational arguments. I must admit that some of the features of Typemock that you mention has escaped my attention, but if that is the case, I'm probably not the only one, so they sound like good candidates for future blog posts (hint, hint). Working for Microsoft I obviously don't have a problem with companies trying to make money on software :) The reason I'm bringing up cost is only because the benefits from using a piece of software have to be greater than the cost. Whith legacy code, this is clearly the case with Typemock. With greenfield code, this evaluation is more difficult, since some of the competition is free. Up until now, Rhino Mocks has appeared to me to be the most attractive combination of benefit and cost, but since someone, somewhere is always pushing the envelope, this may not be the case tomorrow - indeed, Typemock may end up offering the most valuable package.

• Anonymous
  March 10, 2008
  > You can only use a fake (or a stub) if your > API is truly testable, because it has > nothing to do with how a particular mocking > framework works. That's why I disagree with > Colin. Not sure I get this. > For dealing with legacy code, I'd pick > Typemock. So my question is what do you define as legacy? > On the other hand, for people who only do > greenfield development, I don't see what > Typemock offers over, say, Rhino Mocks. Choice. If I don't want to define an interface and inject but I do want to stub/mock out that class in a particular test then I can. If I want to mock out a concrete domain class in a particular test, I can.

• Anonymous
  March 12, 2008
  Hi Colin Thank you for your comment. You are talking about choice, but ironically, so am I: I want to be able to choose how to implement a test double for each test I write. For some tests, I'd like to verify indirect outputs, so I need to use either spies or mocks. For some tests, I may only care about direct outputs, so I just need something to provide canned answers to my SUT, since I don't really care about the specifics of how my SUT communicates with its DOC. In such cases, stubs or fakes are much more appropriate. While you can configure a mock framework to act like a stub, it's a pain, and you can't make it behave like a fake at all. This all boils down to the fact that a proper testable design is mandatory, since relying on, say, Typemock's interception abilities would make it impossible to write a test that uses a fake instead of Typemock. To me, the ability to mock something that only Typemock can mock has nothing to do with choice, but rather Vendor Lock-In. Regarding legacy code, I define it like most other agile developers do: Code that isn't testable (by other methods than Typemock).

• Anonymous
  March 17, 2008
  The comment has been removed

• Anonymous
  March 18, 2008
  Hi Colin Thank you for your comments - let's agree to disagree :) As far as I understand, your reasoning is motivated by the dimension of freedom you find most important, and so is mine. Our goals seem to differ, though, which cause us to reach different conclusions. There's nothing wrong with that. As I remarked to Roy, my thinking is heavily influenced by my professional experience, and I've mostly done greenfield development for my entire career. As Steve Freeman remarked at http://www.mockobjects.com/2008/03/another-round-in-testability-debate.html, he rarely extracts interfaces, but rather implements them. That sounds a lot like the way I work, too.