Last week, we discussed the difference between managed and unmanaged dependencies:
-
Managed dependencies are out-of-process dependencies that are only accessible through your application; interactions with them aren’t visible to the external world. A typical example is the application database.
-
Unmanaged dependencies are out-of-process dependencies that are observable externally. Examples include an SMTP server and a message bus.
Here’s the image that you can use to visualize the difference between managed and unmanaged dependencies (clickable):
If an out-of-process dependency only connects to your application (like the Cloud database), then only your system can observe its state, which means it’s a managed dependency.
Conversely, if the out-of-process dependency connects to other applications (like the message bus), then those other applications can also observe the dependency’s state (or rather, changes in that state). Hence, the dependency is unmanaged.
1. Unmanaged dependencies and Docker
When it comes to out-of-process dependencies and mocking, the guideline is the following:
Use real instances of managed dependencies; replace unmanaged dependencies with mocks.
In other words, you shouldn’t mock your database, but you should mock the message bus.
Here’s a question I received on this topic:
Should the message bus always be mocked? In the age of Docker, we can run a real message bus inside a container. In this case, we don’t need to mock the message bus, like in the example with the DB.
And if only our application writes to and reads from the message bus, is it still an unmanaged dependency? I totally agree with the DB scenario, but if our application is the only one writing to the message bus, does such categorization make sense?
I’ll tackle the second question first because it’s easier.
Yes, such a message bus is still an unmanaged dependency because in production, it’s still accessible by other applications. We don’t count the dependency’s clients during testing, we count them during the use in production.
It doesn’t matter if you run the out-of-process dependency in a Docker container or not. Containers don’t change the dependency’s categorization.
2. Unmanaged dependencies and mocking
Now, let’s talk about the mocking aspect. Since you can isolate an instance of the message bus for testing, should you really mock that message bus? Wouldn’t it be better to use a real instance of it?
Let’s take apart this question. First of all, it doesn’t really matter whether you use Docker or not. You can have a separate instance of an out-of-process dependency for testing without Docker containers. Granted, it’s usually easier to do so with Docker, but there’s nothing inherently preventing you from running a test message bus elsewhere.
There’s something to be said about using containers for each test run, not just as one shared message bus instance. It might seem compelling because it would allow you to parallelize your tests.
In practice, though, this approach creates too much of an additional maintenance burden. With Docker, you need to:
-
Maintain Docker images
-
Make sure each test gets its own container instance
-
Batch integration tests (because you most likely won’t be able to create all container instances at once)
-
Dispose of used-up containers
If you decide to use a real message bus in tests, it’s more practical to have just one its instance per developer.
Now, what about mocking? Do you always have to mock all unmanaged dependencies? Or can you use their real instances like you do for managed dependencies such as the database?
No, you don’t have to always mock unmanaged dependencies. Let me refine the mocking guideline above:
You should always use real instances of managed dependencies in tests. You can (but don’t have to) replace unmanaged dependencies with mocks.
This guideline is meant to discourage the use of mocks for the database and other managed dependencies, because it leads to brittle tests. However, you don’t have to mock unmanaged dependencies. From a test brittleness perspective, it’s perfectly fine not to mock them if you can.
Alright, so you don’t have to mock the message bus. But should you? Which option is better?
To answer this question, we need to resort to our framework with the 4 components of a good test, which I described in the Unit Testing book. Here are those components:
-
Protection against regressions — How good your tests are at finding bugs
-
Resilience to refactoring — How good your tests are at not raising false alarms
-
Fast feedback — How quick your tests are
-
Maintainability — How easy it is to maintain your tests
Let’s look at our options (mocking and not mocking the message bus) using the above 4 components. We’ll start with the 2nd one.
2.1. Resilience to refactoring
The 2nd component, resilience to refactoring, is the reason for the guideline about not mocking the database and other managed dependencies.
If you mock the database, your tests will raise false alarms each time you modify that database. This, in turn, will make your tests brittle and decrease their value to the point where they do more harm than good.
When it comes to the message bus and other unmanaged dependencies, however, you can safely mock them without damaging the 2nd component, resilience to refactoring. Same goes for not mocking the message bus; this doesn’t make your tests brittle.
If we represent our 2 options graphically, here’s how it’s going to look:
With mocks | Without mocks | |
---|---|---|
Resistance to refactoring |
Good |
Good |
2.2. Protection against regressions
Now, what about the 1st attribute, protection against regressions? Which option better protects you against bugs?
Definitely the option without mocks.
When you test your application against a real message bus, your tests go through not only your application, but the message bus as well, thus increasing the chance of finding bugs in the overall system.
This is how the 2 options would look in terms of the first 2 components:
With mocks | Without mocks | |
---|---|---|
Protection against regressions |
Moderate |
Good |
Resistance to refactoring |
Good |
Good |
2.3. Fast feedback
What about the speed of feedback?
Mocks definitely win here, because you don’t need to communicate with a real application.
I’m not sure how slow a test would become if you run it against a real message bus, though. I would imagine it’d slow down but not that much.
Here’s my estimate for the 3rd component:
With mocks | Without mocks | |
---|---|---|
Protection against regressions |
Moderate |
Good |
Resistance to refactoring |
Good |
Good |
Fast feedback |
Good |
Good/Moderate |
2.4. Maintainability
Finally, we have the maintainability metric. It consists of two subcomponents:
-
How easy it is to understand the test, which is a function of the test size
-
How easy it is to run the test, which is a function of the number of out-of-process dependencies the test works with
It’s definitely harder to maintain a test that works with a real message bus, primarily because you’d have to make sure that message bus remains operational throughout each test run.
I would estimate this last metric as follows:
With mocks | Without mocks | |
---|---|---|
Protection against regressions |
Moderate |
Good |
Resistance to refactoring |
Good |
Good |
Fast feedback |
Good |
Good/Moderate |
Maintainability |
Good |
Bad |
3. Conclusion
So the choice between mocking vs not mocking the message bus boils down to the choice between better protection against bugs vs worse maintainability.
I would say the additional burden of maintaining a test message bus isn’t worth the additional gains in protection against bugs.
You only need to test the end-to-end integration with the message bus once. After that, you only need to check what messages your application generates for that bus.
And you can do that initial end-to-end testing manually. There’s no need to come up with additional test infrastructure and take on additional maintenance burden for a task that can be resolved within a couple hours of manual labor.
Not everything should be automated away. Consider the long-term costs of both approaches.
--Vlad
https://enterprisecraftsmanship.com
Enjoy this message? Here are more things you might like:
Workshops — I offer a 2-day workshop for organizations on Domain-Driven Design and Unit Testing. Reply to this email to discuss.
Unit Testing Principles, Patterns and Practices — A book for people who already have some experience with unit testing and want to bring their skills to the next level.
Learn more »
My Pluralsight courses — The topics include Unit Testing, Domain-Driven Design, and more.
Learn more »