I remember starting with unit testing many years back. One of the most annoying obstacles were class’s private methods.

To achieve good test coverage, I needed to exercise all the edge cases in the business logic under test, but a lot of that logic was hidden behind private methods.

I remember even searching for (and finding!) tools that automatically give your tests access to all private methods. Very convenient, isn’t it?

As you have probably guessed already, it didn’t work out well. This approach did fine for a while, but as the number of such tests grew, it became increasingly harder to refactor the project’s code base.

It’s no wonder — private methods are private for a reason. They are implementation details that you don’t want to ever bind your tests to. Otherwise, your tests will cement those implementation details and make it harder to refactor them.

The cementing effect is, once again, due to contributing to the noise component of the signal-to-noise ratio:

Each refactoring starts to generate lots of false failures. Such failures steer your attention away and discourage you from refactoring altogether.

Nowadays, most programmers focus on the signal part of the test accuracy formula, and completely disregard the noise part.

Don’t get me wrong — it’s important that your tests are capable of finding bugs. But in the long run, reducing the noise is equally important.

Why so much attention to noise, you might ask?

Because the two components influence the project differently over time: while it’s important to have good protection against bugs very soon after the project’s initiation, the need for resistance to refactoring is not immediate.

In the short term, false failures (large noise) are not as bad as false passes (weak signal). In the beginning of a project, receiving a wrong warning is not as bad as not being warned at all and running the risk of a bug slipping into production.

But as the project grows, false failures start to have an increasingly large effect on the test suite:

The reason why is because the importance of refactoring is also not immediate; it increases gradually over time. You don’t need to conduct many code clean-ups in the beginning of the project. Newly written code is often shiny and flawless. It’s also still fresh in your memory, so you can easily refactor it even if tests raise false alarms.

But as time goes on, the code base deteriorates. It becomes increasingly complex and disorganized. Thus you have to start conducting regular refactorings in order to mitigate this tendency. Otherwise, the cost of introducing new features eventually becomes prohibitive.

Far fewer projects get to later stages, though. Developers face the problem of unnoticed bugs more often than false alarms that swarm the project and hinder all refactoring undertakings. And so, people optimize accordingly.

Nevertheless, if you work on a medium to large project, you have to pay equal attention to both false passes (unnoticed bugs) and false failures (false alarms).

Unit testing private methods (along with overuse of mocks that we discussed yesterday) is one of the biggest factors contributing to the number of false failures.

This is the unit testing mistake #4:

"Giving special privileges to tests."

Don’t expose class’s methods or state that you would otherwise keep private for the sole purpose of unit testing. Your tests should interact with the system under test exactly the same way as the production code.

Remember, your tests are just another client of the production code; they shouldn’t have any special privileges.

For information about unit testing anti-patterns, incluing the remaining two — leaking domain knowledge to tests and code pollution — see my book:

https://www.amazon.com/gp/product/1617296279

(Use the nwsentr40 code during checkout on the Manning book page to get a 40% discount.)

Tomorrow: You’ll see why targeting a high test coverage number isn’t as good an idea as most people think.

Vladimir Khorikov