Permissions Tag this page Edit this page Login

Your Tests' Worth

What I don't like about the TDD(Test-Driven Development) movement is how fervently religious its practitioners can sometimes be. I like TDD just fine, but like anything else - it's just a tool. I write tests for some situations, for others, I don't - and I've spoken to many who are like me in this respect. I've also seen TDD done badly - which if persisted - is counter-productive. What I'd like, is to have some way to evaluate how well or poorly you are doing TDD and whether it makes sense at all for the given situation. Put succinctly: when is TDD worth it? If you are already writing tests, this is a small set of criteria that keeps you focused on what should be the desired outcome. If you are thinking about writing tests, these should help you decide whether you want to start getting serious.

The criteria I've come up with are cost, test-time, and regression-fail-rate.

Cost

Obviously, the cost of writing the tests is a big factor and must be minimised. How much are you investing into writing and maintaining the tests? How easy is it to add a new test? How often do existing tests have to be modified? To minimise these costs, you often make use of or build frameworks and tools to make testing easier, but there is a trade-off: how much are you investing into building or integrating testing frameworks and tools?

Test-Time

Test-time is the amount of time it takes for your test suite to run. This is a number you want to minimise because a large test-time slows down a developer's development cycle and will be a large on-going tax to the cost of writing/maintaining tests. This is the reason seasoned TDD'ers will prefer unit tests to functional tests, and also the reason they like to use mock frameworks - these are techniques that can reduce test-time.

Regression-Fail-Rate

The core benefit of TDD is that it gives you the safety to make big changes in your software little time, so how well are you making use of it? Regression-fail-rate is the rate at which your previous testcases fail. Previous meaning all but the testcase you just added and are working on passing. A high number means you are making large changes in your code consistently. If your regression-fail-rate is very low, it means you may not be using the full benefit of TDD. 

Conclusion

To get the most out of TDD, you need to keep its maintenance cost to a minimum while changing your software at a rapid rate.

Leave Comment

optional