Integrating with an external API is almost a guarantee in any modern web app. To effectively test such integration, you need to stub it out. A good stub should be easy to create and consistently up-to-date with actual, current API responses. In this post, we’ll outline a testing strategy using stubs for an external API.
Our example app is a stock Rails 3.2 app. Let’s imagine that we’re assigned the following story:
This story involves integrating content from an external site. Hacker News doesn’t have an API, but they do have an RSS feed that we can use instead.
A typical implementation of our story will look something like this:
Postmodel for recent posts.
Postmodel asks a Hacker News library for recent posts.
To test this we’ll:
Postmodel’s specs. Since we already have an integration test, there’s no need to perform another mini-integration test between our domain model and its collaborator.
vcrto record the actual Hacker News HTTP request and response. This will give us the ability in the future to change our HTTP client without breaking our specs.
Our request spec uses capybara to simulate a homepage request. We tell
vcr to record any HTTP requests by using its RSpec metadata support.
The first time this spec is run,
vcr will record the actual Hacker News HTTP request and response and save it to the filesystem. Future spec runs will then reuse this recorded HTTP interaction. Since we don’t know the latest posts on Hacker News at the time this spec is first run, we only specify an expected DOM structure.
This failing spec guides us to the following controller and view implementation.
The second run of the request spec goes further than the first, moving us to the domain model.
Instead of programming our controllers and views to the interface of our Hacker News library, we’ll introduce a domain model. The domain model is where we’ll consolidate our app’s business logic. It also gives us another level of indirection, allowing us in the future to change how we get Hacker News content without having to change our controllers and views.
Our domain model won’t directly make any HTTP requests. Instead, it will collaborate with a separate Hacker News library that will be responsible for making HTTP requests and parsing RSS. This library acts as an adapter layer, allowing our domain model to define its relationship to the outside world in its own terms. With an integration test already in place, we can use mocking to test our domain model in isolation.
Our domain model is now specified but a re-run of our request spec reminds us we still have work to do. Mocking allowed us to implement our domain model by faking its collaborator, but we still have to implement that collaborator. Whenever you mock, you must have a higher-level test that actually tests the full integration between all your objects.
We can now drop back down and spec out the Hacker News library.
Our Hacker News library will be responsible for getting the latest posts from Hacker News. Like in our request spec, we’ll use
vcr to record the actual HTTP request and response to Hacker News. This keeps our spec accurate. Future runs of this spec will then reuse the recorded HTTP interaction. This keeps our spec fast. Stubbing at the HTTP level instead of mocking out our HTTP client, gives us the ability to change our HTTP client later (granted it’s one of
vcr‘s supported HTTP clients).
typhoeus and Ruby 1.9.3’s RSS library work together to request and parse the Hacker News RSS feed.
With our library specified, we can move back up to our request spec, which should now be passing.
In our request and library specs, we used
vcr to record an actual HTTP request to Hacker News and then in future spec runs reused that recorded HTTP interaction. The specs were accurate and fast but, if the Hacker News RSS feed changes, our specs still continue to pass. We need to keep our stubs up-to-date.
vcr can be configured to automatically re-record “out-of-date” HTTP interactions.
vcr stores a timestamp alongside every recorded HTTP interaction. The above
default_cassette_options configuration tells
vcr to re-record all HTTP interactions that are older than 1 week.
Depending on an external API in your test suite can easily lead to tests that sometimes pass and sometimes fail. To make your tests run consistently, stub out external APIs. Ensure your stubs are accurate by creating them from actual API responses. And finally, remember to routinely verify that your stubs are up-to-date with the current state of the external API.