Contract Checks: Once and Only Once for Unit Testing (Part I)

In her post that started this thread, Stef mentioned Meszaros' book on xUnit.  My reaction when I bought the book was a bit different from hers; it was what I would now describe as a smell.  My instinct was that there’s something wrong if one needs to spend a large fraction of one’s architectural effort on code which is not part of the shipped product.

Note that I am not saying that effort spent on testing and test automation is valueless if the tests aren’t part of the shipped product.  Spatial has been running hundreds of thousands of (scripted) ACIS tests every night since before I joined the company oh so many years ago; our test suite is one of our most valuable pieces of IP.  Instead, my instinct was that that typical usage of what Meszaros calls the "Four-Phase Test" pattern probably violates some core OO design principle, which in turn requires expending a lot of (architectural) cleverness to manage the resulting complexity. 

For reference, let me explicitly write down my (simplified) version of the four phases:

1.      Set up the input data (objects) and owning object(s) for the method(s) to be tested.

2.      Execute the method(s).

3.      Perform validation tests (usually a set of assertions) on the outputs (including the owning object(s)) to check the result.

4.      Return the xUnit application to its (hopefully) original “clean” state.

My original thought was that construction of data objects during setup (step 1) violates data hiding, which leads to tight coupling between the code and the tests.  Over the years, however, I’ve realized that there’s a much deeper issue with the steps above - you can probably guess what it is from the title of this post. J  Here’s my opinion:

The  xUnit tools often foster a testing methodology which violates a fundamental tenet of XP/Agile programming - Once and Only Once. 

This is because xUnit encourages developers to put validation code in the test (step 3), i.e. outside the method being tested.  This is turn results in either duplication of the validation code in multiple tests, or architectural complexity in the test component to ensure that the same validation code is called from multiple tests.  Obviously, I think the solution to this dilemma lies in the introduction of contract checks, but that will have to wait until Part II....