One of the main reasons why I wrote the Unit Testing book (despite there being quite a few unit testing books already) is because the existing books and other resources rarely discuss economic implications of unit testing, whereas in my view, this is one of the most important discussions to have in any area of software development.

In economics, there’s the notion of seen vs unseen, which means that, when implementing a policy, you need to focus on both sides of it:

  • The desirable results, which are the seen part of the equation

  • The unintended consequences, which are the unseen part.

You will always have both, no matter the policy. The decision as to whether to implement the policy boils down to carefully weighting potential results against potential unintended consequences — the seen vs unseen.

The same is true for software development and unit testing in particular. You should take into account both the seen and unseen parts when writing tests.

So, what are these parts when it comes to unit testing?

  • The seen part is the benefits unit tests provide you with.

  • The unseen part is the (often hidden) costs.

Both are critically important. The costs are even more so because we, as humans, tend to give preference to the short-term benefits and disregard the long-term costs.

(In economics, there’s a whole theory called Time-Preference Theory that studies the implications of this tendency to discount the long term in favor of the short term.)

The seen part of unit testing is how good your tests are at protecting you against bugs. The unseen — how much that protection costs you. The cost component consists of the two major subcomponents:

  • Maintainability (how much effort it takes to keep the test operational)

  • Resilience to refactoring (how much noise this test generates)

The biggest impediment to writing good tests is not taking into account the costs of unit testing.

As a result, people tend to produce a lot of low quality tests. Instead of helping the project to grow, such tests drag it down. The reason is that they require a lot of time to maintain and update after refactoring.

There are second-order effects too: you start to lose trust in your tests, which leads to fewer refactorings and, ultimately, code deterioration.

Always pay attention to the costs component. Only allow the most valuable tests in your test suite; disregard the rest.

It’s better to have a small set of highly valuable tests than a large number of mediocre ones.

--Vlad


Enjoy this message? Here are more things you might like:

Consulting and training — I offer consulting and group training for organizations on Unit Testing and Domain-Driven Design. Reply to this email with the word "consult" to discuss.

Unit Testing Principles, Patterns and Practices — A book for people who already have some experience with unit testing and want to bring their skills to the next level.
Learn more »

My Pluralsight courses — The topics include Unit Testing, Domain-Driven Design, and more.
Learn more »