When I read Stef’s latest blog entry the Sunday morning after she posted, I had three thoughts:
- COOL!! Thank you Stef for a great segue opportunity!
- Drat. Now I’m going to have to change the order of the next couple of posts I was planning.
- Double-drat. I’m so psyched about this that I need to write it up right away.
So there I was, Sunday morning, typing away…. :)
What I was originally planning to talk about in this post was client/server programming and Robert Martin’s books on Object Oriented Design. Instead, I’m going to follow through on the teaser from my last entry: the interplay between unit testing and contract checking. There’s too much here for one post, so I’m going to break it up into two. This post will focus on a description of contract checking and how we practice it here at Spatial.
First, what do I mean by "contract checking"? I mean a weaker form of the ideas which were codified by Bertrand Meyer in his Eiffel programming language; there’s a good explanation here. To summarize the parts that we’ve used at Spatial the most:
- Methods and functions have preconditions which should always be true upon entry.
- Methods and functions have post-conditions which should always be true upon exit.
- Objects have class invariants which define their valid states. A public or protected method of a class should not break the class’ invariants. When we use invariants, we insert them into pre- and/or post-conditions.
- Preconditions, post-conditions, and invariant violation are all tested within blocks of code which are turned off in release runs. The reasons for not testing release runs are performance and (more importantly) to protect customers from having an otherwise harmless contract failure cause their applications to crash. We’ve found that contract failures should abort when being run in a batch testing environment (so that they can’t be hidden) but throw when in a debug environment (for developer convenience).
What we’ve done at Spatial to retrofit support for contract checking in our C++ code is to define ContractBegin and ContractEnd macros which define scopes which can be turned off in release runs, and ContractAssert and ContractFail macros which signal failure in non-release runs. Typically, the contract checks go into pre- and post-condition blocks at the beginning and end of the method, but a contract check can also be inserted in the middle of the routine in the same way one might add an assertion. Developers add these checks at their will in code that they're working on; this allows incremental instrumentation of our code.
So contract checking here at Spatial is not the full-blown formalism of Design by Contract. On the other hand, it’s much more than a fancy name for assertions. By formulating the assertions in terms of pre- and post-conditions on a method, contract checking shifts the focus of the methodology into an exact specification of the responsibilities and behavior (i.e. contract) of the methods which make up the interface of a software development kit, which I think is the essence of Design by Contract. Now that I think about it, this is one of those pragmatic trade-offs that Kevin was talking about in his recent post.
So what does this have to do with unit testing? I would say that it’s a complementary methodology which should be used to relieve unit testing of the burden of doing things that it’s not good at. And as I type this, I realize that I should point out that I’m using "unit test" to describe a much wider range of testing strategies than strict unit testing – I think a better description might be "external exhaustive testing" - which are used to ensure program correctness and include system testing, test driven development, test plan specification, and so on. All of these methodologies work well on simple systems, but have trouble managing the complexity of large software (eco-) systems. What contract checking does is to move a large part of the complexity of the testing out of the test code and into the application code, which allows these methodologies to more easily scale to large, complex systems. The test code then becomes a vehicle for:
- Making sure that the application actually does what it’s supposed to, for example that a Boolean operation between two solid models actually results in the correct answer. (This is equivalent to Stef’s category to "validate that the code does what they expect".)
- Act as a driver to ensure that (almost) all code branches and the associated contract checks are passed through at least once. This is similar to Stef’s second category, with the benefit that failed contract checks give very good localization information as to where things went wrong.
Well, that’s enough for this post. Next time I’ll talk about the software engineering principles which external testing methodologies run afoul of (and which are well handled by contract checking), how these can be understood in terms of a software ecosystem, and how this explains Stef’s experience with our 3D InterOp test suite.
