Two weeks ago at Agile 2009 Jim Shore and Arlo Belshee led an open-ended discussion session on end-to-end tests and how to get rid of them. Part of this session was lightning talks based on small-group discussion. This is a refinement of the lightning talk I gave.
A lot of teams have a legacy test codebase to match their legacy code codebase. Part of the solution to end-to-end testing for them is to find ways to whittle down their existing end-to-end tests. Wholesale deletion is possible, but emotionally/politically difficult. I propose piecewise deletion, starting with the most important areas.
Whenever an end-to-end test bites you — there’s a false positive, six hundred tests fail because you change one method, or you can’t change an assumption because too many tests call it — kill something that’s part of the problem. Sometimes it’ll be obvious what to change: if this end-to-end test is failing for no good reason, remove it. Sometimes it’ll be harder. Instead of determining the exact right thing to remove, remove something and keep going. Remove a helper method and the tests that use it. Remove a test hook from the production code and the tests that use it.
If you keep repeating this action, you’ll slash away at the test code that causes the most problems. This may mean that you have no tests left for some areas of the code. That’s a valuable exposure of information: this code is so problematic that worthwhile tests cannot be written for it. Now you know where to concentrate your refactoring and rewriting efforts.
The original talk was focused on end-to-end tests, but I think this is a broader tool. Whenever you have code that is a never-ending source of trouble and pain, start treating it more roughly. Eradicate some of the mess each time you walk through the code, even if you don’t know where the absolute worst spots are. There’s no sense in using laproscopic surgery on a leg that’s dead to the knee.