In our last Elixir blog post, “Functional Mocks with Mox in Elixir”, we discussed how testing across module boundaries could be made easier by creating a Behaviour for a collaborating module, then utilizing the wonderful framework Mox to substitute a lightweight mock module in tests.
This approach is well and good when you have very concrete module boundaries that are well-defined and coarse enough to warrant the ceremony of wiring up a behavior for a module boundary. But what if we aren’t necessarily interested in all the work in creating a mock, and we need something simpler and more lightweight?
Join us as we discuss some alternative ways to write lightweight tests across function or module boundaries.
Dependency Injection (or DI) is a software development system capability popularized in the object-oriented world, in which a software system provides a framework for accessing system dependencies. The test framework in these systems oftentime hooks into this dependency injection framework in order to replace live dependencies with test doubles. Practitioners of TDD (like us here at Carbon Five!) use test doubles in our tests to write concise, focused unit tests that verify behaviors with collaborating concepts. Additionally, using test doubles lets us test-drive out certain feature paths without actually having to have first implemented our collaborating dependencies.
In our prior post, we leveraged the use of module attributes to do the “injection” of the module dependency:
module MyApp.MyModule do | |
@my_collaborating_module Application.get_env(:my_app, :my_collaborating_module) | |
def perform() do | |
@my_collaborating_module.do_something() | |
end | |
end |
So in this case, the module attribute lookup in tandem with a system configuration setting is the manner in which we inject the MyCollaboratingModule
dependency. A Behaviour is then used to conform that dependency to the expected API.
However, this approach requires we create a Behaviour, wire up test and production implementations, and load up Mox in all of our tests. That’s a lot of ceremony!
Are there are other more lightweight ways to do dependency injection? Sure there are! Let’s look at a few others:
In Jose Valim’s popular “Mocks and Explicit Contracts” blog post, he discusses a very lightweight opportunity to decouple your dependencies. Say your app references an implicit dependency on MyApp.MyCollaboratingModule
:
module MyApp.MyModule do | |
def perform do | |
MyApp.MyCollaboratingModule.do_something() | |
end | |
end |
This makes your tests difficult to write, because then your tests will run everything under the hood of MyCollaboratingModule
. What if it’s particularly slow, or hairy, complicated, or makes a call to an external service? Let’s refactor a bit.
module MyApp.MyModule do | |
def perform(do_something \\ &MyApp.MyCollaboratingModule.do_something/0) do | |
do_something.() | |
end | |
end |
Now I can write my tests and pass in a stub function (or verifying function):
test "calls MyCollaboratingModule.do_something()" do | |
fake_do_something = fn -> send self(), :do_something_called end | |
MyApp.MyModule.perform(fake_do_something) | |
assert_received :do_something_called | |
end |
There is a great simplicity in this approach!
You can also do the same thing, but instead of passing the function, you pass in the entire module:
module MyApp.MyModule do | |
def perform(collaborating_module \\ MyApp.MyCollaboratingModule) do | |
collaborating_module.do_something() | |
end | |
end | |
# And in the tests: | |
test "calls MyCollaboratingModule.do_something()" do | |
defmodule FakeCollaboratingModule do | |
def do_something do | |
send(self(), :do_something_called) | |
end | |
end | |
MyApp.MyModule.perform(FakeCollaboratingModule) | |
assert_received :do_something_called | |
end |
Similarly, there is a nice simplicity to this approach
Let’s imagine we begin with this software system – an HTTP endpoint that receives a HTTP request and builds a string to be read back in a text-to-voice system.
defmodule ResponseHandler do | |
def handle_response( | |
http_response, | |
parse_http_response \\ &HttpResponseParser.parse/1, | |
generate_speech \\ &SpeechGenerator.generate/1 | |
) do | |
parse_http_response.(http_response) | |
|> do_more_work() | |
|> generate_speech.() | |
end | |
end | |
defmodule HttpResponseParser do | |
def parse(http_response, get_timezone_from \\ &TimezoneHelper.timezone_from/1) do | |
timezone = get_timezone_from(http_response.headers["DATE"]) | |
more_things = do_more_parsing() | |
%ApiResponse{timezone: timezone, status: http_response.status, more_things: more_things} | |
end | |
end | |
defmodule SpeechGenerator do | |
def to_speech(%ApiResponse{} = api_response, to_spoken_time \\ &TimeUtils.to_spoken_time/1) do | |
speech_time = to_spoken_time.(api_response.time) | |
did_this_action = do_something_with(api_response) | |
"At #{speech_time}, I noticed that you #{did_this_action}" | |
end | |
end |
Here we have a 3-level-deep dependency graph. The ResponseHandler
calls out to HttpResponseParser
and SpeechGenerator
collaborating modules. Those two modules in turn call out to the TimezoneHelper
and TimeUtils
modules, respectively. While this approach is fine, each level of the system has some degree of coupling to other parts of the system. Is there another way to reduce the complexity here?
In this new approach, we reduce the responsibility surface area of module functions as much as possible, preferring that, as much as possible, lower-level modules merely acts on data and structs. Any coordination with external collaborating modules is delegated to an outside coordinator. Observe:
defmodule ResponseHandler do | |
# handle_response/1 becomes a coordinating function | |
def handle_response( | |
http_response, | |
get_timezone_from \\ &TimezoneHelper.timezone_from/1, | |
parse_http_response \\ &HttpResponseParser.parse/2, | |
to_spoken_time \\ &TimeUtils.to_spoken_time/1, | |
generate_speech \\ &SpeechGenerator.generate_speech/2 | |
) do | |
with timezone <- get_timezone_from.(http_response.headers["DATE"]), | |
parsed_api_response <- parse_http_response.(http_response, timezone), | |
speech_time <- to_spoken_time.(parsed_api_response.time) do | |
do_more_work(parsed_api_response) | |
|> generate_speech.(parsed_api_response, speech_time) | |
end | |
end | |
end | |
defmodule HttpResponseParser do | |
# Note how the prior call to TimezoneHelper.timezone_from/1 is pushed "up" to the coordinating function | |
def parse(http_response, timezone) do | |
more_things = do_more_parsing() | |
%ApiResponse{timezone: timezone, status: http_response.status, more_things: more_things} | |
end | |
end | |
defmodule SpeechGenerator do | |
# Note how the prior call to TimeUtils.to_spoken_time/1 is pushed "up" to the coordinating function | |
def to_speech(%ApiResponse{} = api_response, speech_time) do | |
did_this_action = do_something_with(api_response) | |
"At #{speech_time}, I noticed that you #{did_this_action}" | |
end | |
end |
The next approach is a simple one, and it’s mainly aimed at easing the ergonomics of having to wire through a handful of function references in the argument body.
Observe how difficult it would be to work with a function signature like this:
defmodule ResponseHandler do | |
# handle_response/1 becomes a coordinating function | |
def handle_response( | |
http_response, | |
get_timezone_from \\ &TimezoneHelper.timezone_from/1, | |
parse_http_response \\ &HttpResponseParser.parse/2, | |
to_spoken_time \\ &TimeUtils.to_spoken_time/1, | |
generate_speech \\ &SpeechGenerator.generate_speech/2 | |
) do | |
do_something_with_dependency("arg", parse_http_response) | |
|> do_something_else_with_dependency("arg2", get_timezone_from, to_spoken_time) | |
end | |
end |
Oh my goodness. Imagine any collaborating functions within this function declaration that then in turn need to have each specific function wired through its own function declaration. It’s hairy enough to make refactoring a miserable experience.
What if we instead grouped this scattering of functions and grouped them all into a map called dependencies
?
defmodule ResponseHandler do | |
# handle_response/1 becomes a coordinating function | |
def handle_response( | |
http_response, | |
dependencies \\ default_dependencies() | |
) do | |
do_something_with_dependency("arg", dependencies) | |
|> do_something_else_with_dependency("arg2", dependencies) | |
end | |
def default_dependencies() do | |
%{ | |
get_timezone_from: &TimezoneHelper.timezone_from/1, | |
parse_http_response: &HttpResponseParser.parse/2, | |
to_spoken_time: &TimeUtils.to_spoken_time/1, | |
generate_speech: &SpeechGenerator.generate_speech/2 | |
} | |
end | |
end |
Then in downstream collaborating functions, we simply route the entire dependencies
map as a final argument.
Tests are simple to write too:
test "ResponseHandler.handle_response/1 does stuff" do | |
fake_dependencies = %{ | |
get_timezone_from: fn _arg -> "America/Los_Angeles" end, | |
# more function doubles here … | |
} | |
ResponseHandler.handle_response(http_response, fake_dependencies) | |
end |
Here we get into more powerful forms of the Elixir language. For dependencies that represent a cohesive capability, we can use Protocols to represent an action that otherwise would have been a collaborating function. Let’s go back to one of our original examples:
defmodule ResponseHandler do | |
def handle_response( | |
http_response, | |
parse_http_response \\ &HttpResponseParser.parse/1, | |
generate_speech \\ &SpeechGenerator.generate/1 | |
) do | |
parse_http_response.(http_response) | |
|> do_more_work() | |
|> generate_speech.() | |
end | |
end |
What if we re-thought our dependencies in terms of protocols? We begin by reformulating our dependency in terms of the conforming data structure we wish our protocol to provide, in this case HttpParsing
and SpeechGeneration
:
defprotocol HttpParsing do | |
def parse_http_response(t, http_response) | |
end | |
defprotocol SpeechGeneration do | |
def generate_speech(t, response) | |
end |
Then we create a new struct that will provide the type to dispatch this new protocol on:
defmodule ResponseHandler.Dependencies do | |
defstruct [] | |
end |
Now we implement both protocols for this new struct:
defimpl HttpParsing, for: ResponseHandler.Dependencies do | |
def parse_http_response(_t, http_response) do | |
HttpResponseParser.parse(http_response) | |
end | |
end | |
defimpl SpeechGeneration, for: ResponseHandler.Dependencies do | |
def generate_speech(_t, options) do | |
SpeechGenerator.generate(options) | |
end | |
end |
Let’s turn back to our original implementating call site and see what it might look like with a data structure that implements our new HttpParsing
and SpeechGeneration
protocols:
defmodule ResponseHandler do | |
def handle_response( | |
http_response, | |
dependencies \\ %ResponseHandler.Dependencies{} | |
) do | |
response = HttpParsing.parse_http_response(dependencies, http_response) | |
|> do_more_work() | |
SpeechGeneration.generate_speech(dependencies, response) | |
end | |
end |
Since the ResponseHandler.Dependencies
struct has an implementation of both the HttpParsing
and SpeechGeneration
protocols, it goes ahead and supplies the real methods at runtime. In our tests, however, we can define the following inline in our test:
defmodule FakeDependencies do | |
defstruct [] | |
end | |
defimpl HttpParsing, for: FakeDependencies do | |
def parse_http_response(_t, http_response) do | |
%{ … } # fake response | |
end | |
end | |
defimpl SpeechGeneration, for: FakeDependencies do | |
def generate_speech(_t, options) do | |
%{ … } # fake data | |
end | |
end | |
test "does the thing" do | |
response = ResponseHandler.handle_response(http_response, %FakeDependencies{}) | |
# assertions… | |
end |
There we have it – four alternative lightweight approaches to dependency injection in Elixir, spanning from the most naive to the most powerful; from the simplest to the most complex.
Which one works for your code? The answer, as it usually does, is that it depends on your use case. I encourage the reader to always do the simplest thing that could possibly work. Start by doing simple function passing in the arguments list.
When you find yourself needing more and more function callbacks in your argument list, ask yourself if you can defer these calls to an outside caller.
If you must use all these functions, then consider consolidating them into a simple data structure and passing the structure around.
Finally, leverage Elixir’s protocol system if you want some really powerful compile-time checks against your collection of dependencies.
Of course, don’t forget that you may also find it appropriate to model your dependencies as Behaviours and mock function calls using Mox.
What do you think? What’s worked for you? Let us know on Twitter!
Many thanks to Carbon Five coworkers Hannah Howard, Craig Lyons, Erin Swenson-Healey and Will Ockelmann-Wagner for conversations around and input on this post.
Andrew is a design-minded developer who loves making applications that matter.