Always test.
That is the reflex TDD leaves you with, and I understand why it exists. Testing is an awesome tool, and is so widely applicable that it has gained a lot of traction with devs. Nowadays, a lot of devs would agree with the statement that testing is good, but that’s like saying a hammer is good. It’s a tool in your toolbox, it can be widely applicable, but it can be both good or bad, depending on the context.
Tests are good?
Tests can be good, and they often are not just good, but awesome. There is no doubt in my mind about that.
What is interesting though is what makes it good. There are many well documented benefits of testing… but what I especially like personally, is that it reframes the problem. Instead of asking for the solution to a problem, the test asks what does the solution look like? Some solutions are so hard to describe that to describe the solution, would take almost the same amount of information to provide the actual solution. But more interestingly, some solutions have some properties that are so compact and easy to describe, without giving the solution. So in a way, it feels like writing the test, is just writing the program in a different way.. and if they don’t agree, the test fails. Tests like this feel the most satisfying to write.
Another thing I like about testing is that it is like a visualisation exercise before doing the actual thing. It primes your mind, forces you to think about the whole problem space and guides your execution, so you’re not all wobbly and shaking. Its an exploration of the problem space before you even do anything. Its usually the first thing I reach out for when trying to validate my understanding of a bug.
So yeah, tests can be good, but it still has a trade-off because testing is not free.
Tests are bad?
This might anger some folks, but writing tests can be a bad trade-off. Writing tests will always have a cost, and some tests will never see any benefit. I think a mental shortcut, to always test is like buying all the insurance you can get.. it costs money, but some of that insurance might not be doing anything for you (even your peace of mind).
So when is the trade-off bad?
First is when there is little value.. Some code (especially in modern languages) are already self documenting and look declarative enough that writing a test is just rewriting the same code but in a less readable way. A good example is a function that does a pattern match - it self documents, its readable and a test, doesn’t add anything to it. It would be like testing arithmetic, by doing the same arithmetic.
Second is when the cost is high - my anecdotal observation over the years is that, when something is hard to test, the test has usually low value. When the test is hard to write, it usually follows that it is hard to read, making it less useful for documentation, make it hard to update, and might have some holes in it because it is hard to detect. New joiners, will not only skip it and read the code directly to understand it, they will also probably struggle to update it. There are still cases when you might want to do it because the benefits or the stakes are just that high.. but you also have other avenues like refactoring the code to make it easier, then the value of the tests just go up (lower denominator).
The two points above talk about one side of the trade-off, one is when the benefit is low, the other is when the cost is high. And that’s generally when I would say its bad.
So what am I really saying?
I’m saying that the decision whether to test something or not is an interesting and valuable decision, and I don’t think you should skip it and default to “always test”. I think a heuristic makes little sense, since thinking through it only takes a few minutes, and the effect would last for as long as you maintain the code. I think it is hard to make a good “rule” of thumb on when to test, because the circumstances and constraints are always different, not to mention the code is always unique. So I think it is always worth to put thought into whether to test or not, instead of being predisposed to always test, or blindly having a target test% coverage.
So
None of this is a case against tests. I think TDD is a good instinct but a little too confident. Testing is usually good. Writing the test first is often better. But that also doesn’t mean that not writing tests can also be good.
So - Always think about it.