Better Mocking in Ruby

In TDD we write tests to discover the interface of the object we’re testing. Mock objects extend TDD by discovering an object’s collaborators. [1] By mocking an object’s collaborators we can truly test an object in isolation. Changes in the implementation of its collaborators, but not their interface, will not cause a test in which those collaborators are mocked out to fail. This results in a more robust test suite in which a single failure doesn’t result in rippling failures throughout the suite.

Here’s a few tips I’ve learned that are helpful when mocking.

Never Mock Without a Failing Integration Test

This is the most important rule of mocking. When you test using mocks you are replacing an object’s collaborators with fakes. If the interface of those collaborators change, your test that mocks those collaborators could pass even though its incorrect, a false positive. In order to avoid such a situation you have to start with a failing integration test.

This could be a Cucumber feature, an RSpec request spec, or maybe even a controller spec, as long as it adheres to one rule: it tests all the objects with real collaborators i.e. nothing is mocked out. [2] Once this test is written you can then drop down through the layers of your application testing each object in isolation with mocking. The integration test is your safety net to catch failures between your objects due to interface changes.

Only Mock Types That You Own

Never mock code you didn’t write such as frameworks and 3rd party libraries. You want to actually exercise this code to see it work as advertised and you’re integrating with it correctly. Let’s take a look at a feature that we’ll implement using some 3rd party code: “User search uses google.com to perform the search”

After drilling down from a failing integration test to our model, our first test might be something along the lines of: [3]

describe User do

  describe '.search' do
    it 'searches Google for the given query' do
      query = 'foo'
      results = []

      HTTParty.
        should_receive(:get).
        with('http://www.google.com',
             :query => {
               :q => query,
               : output => 'json'
             }).
        and_return(results)

      User.search query
    end
  end

end

Here we’re mocking HTTParty, a type we don’t own. Unfortunately this prevents us from refactoring User.search to use another HTTP library. How we can fix this? By mocking roles not types.

Mock Roles Not Types

Let’s rewrite this by not mocking HTTParty and instead introduce a role specific to our domain.

describe User do

  describe '.search' do
    it 'searches for the given query' do
      query = 'foo'
      results = []

      searcher = double 'searcher'
      searcher.
        should_receive(:search).
        with(query).
        and_return(results)

      User.searcher = searcher

      User.search query
    end
  end

end

Here we introduced a searcher role with a generic API consisting of a search method that takes a query. We also renamed the spec to be more generic by not mentioning Google, an implementation detail. We keep the original spec as our high-level integration spec.

By introducing a domain specific role, we’ve removed any mention of implementation and now have a more generic domain model. If we want to refactor User.search to use a different HTTP library we configure User with a different implementation of the searcher interface.

Our new test did require us to add a setter to our User class in order to configure its searcher dependency.

class User < ActiveRecord::Base

  class_attribute :searcher

  def self.search(query)
    searcher.search query
  end

end

Testing 3rd Party Integration

Now that we’ve replaced our 3rd party dependency with a domain specific role how do we test our new role implementations? We’ll continue to follow our rule of not mocking types we don’t own and instead test them with mini-integration tests. This is ok because it’s the only way to ensure 3rd party behavior and integration.

Here’s a non-mocked test for one implementation of our searcher role that we’ll call Google:

describe Google do

  describe '.search' do
    before do
      query = 'foo'
      body = [{ 'url' => 'http://foo.com'    },
              { 'url' => 'http://foobar.com' }]
      stub_request(:get, 'www.google.com').
        with(:query => {
               :q => query,
               : output => 'json'
             }).
        to_return(:headers => { 'Content-Type' =>'application/json' },
                  :body => body.to_json)

      @results = Google.search query
    end

    it 'searches google for the given query' do
      @results.should have(2).results
    end
  end

end

This test doesn’t mock any 3rd party code, giving us freedom to refactor in the future. However we do use webmock to stub out a low level HTTP request but we don’t specify how this request is actually made so we can still refactor how we end up making this request.

Here’s one possible implementation using HTTParty:

class Google

  def self.search(query)
    HTTParty.get 'http://www.google.com', 
                :query =>{ :q => query, : output => 'json' }
  end

end

The Tradeoffs of Mocking

Most of the backlash against mocking results from failure to follow the above suggestions. This makes refactoring without breaking a test very difficult. Following the above techniques can give us a more robust and refactorable test suite. Unfortunately this is achieved via additional layers of indirection using roles and dependency injection.

Is this flexibility/complexity necessary? Dependency injection never caught on in Ruby, even using simple setters with default arguments. What do we risk if we don’t mock? We risk every test essentially becoming an integration test, which gives us a brittle and slow test suite. Is this a bad thing?

So when do we mock? It depends. In the end mocking just becomes yet another tradeoff we have to consider when testing software.


[1] Mocks are also useful for removing external dependencies, simulating error conditions, etc.
[2] Actually some mocking is ok in integration tests e.g. you may mock to eliminate a dependency on a 3rd party API in order to have more consistent and faster test runs.
[3] All mocking examples use RSpec’s built in mocking and stubbing framework.

About Jared Carroll

After a short stint in the fashion industry Jared found his true calling at Carbon Five. Yes... he looks like a serial killer in this photo. But really he is as gentle as a flower.
This entry was posted in Process, Web. Bookmark the permalink.

11 Responses to Better Mocking in Ruby

  1. Pingback: Better Mocking in Ruby

  2. Avdi Grimm says:

    Bang-on. This is all very good advice. Love the emphasis on mocking roles instead of types!

  3. Do you find that you often have to refactor your tests when you refactor your code?

    I often find myself changing my interfaces as I code and discover new requirements, which means I have to go back and re-write all the places that I mocked out that code.

    Recently, I’ve been doing pure integration testing, and avoided unit testing and mocking. This allows me to drastically refactor my code and data model, and still have a passing test suite.

    I understand that if you don’t change your interfaces, then mocking can be a tool to test objects in isolation, but I often find myself changing my interfaces frequently as part of refactoring and new feature development.

    Lastly, I’d like to mention that I heartily agree that if these guidelines are followed, mocking is far less brittle and more useful.

    -Nick

  4. Christian Bradley says:

    Solid article Jared. Always look forward to these.

  5. Jared Carroll says:

    @nick – Good point about how refactoring often leads to interface changes so mocking won’t buy you much. Developing test-first can help design that beautiful interface the first time around but requirements do change and interfaces are rarely set in stone so yeah I agree that refactoring implementation often leads to refactoring interface.

    Interesting all-integration testing approach, I’ve never tried that across the board. Factories too make ignoring mocking altogether more tempting because often test setup is the most intensive step of testing.

    @avdi, @christian thanks!

  6. Michael Wynholds says:

    I have definitely found myself recently replacing tests using mocks with more full-stack style integration tests (in my controller specs, for example). I think mocking is very much an art form, or at least you need to be aware of the trade offs before you go head first in to it. I am going to try these tips to see if they give me the love.

  7. Walter Yu says:

    Excellent write-up, I am currently repairing the broken tests in one of my applications and this will be helpful. Keep up the good work!

  8. Rob Pak says:

    Great write-up Jared, really good advice. Thanks!

  9. Vito Botta says:

    Great advice, something to keep in mind. Had this kind of issue with HTTParty/RestClient, and I had to update several specs

  10. Pingback: Delicious Bookmarks for July 24th from 08:57 to 09:33 « Lâmôlabs

  11. Filipe Giusti says:

    I agree with the tips, if you allow me a resume of them would be:

    1. Do integration test.
    2. Use adapters for all 3rd party code.
    3. Do not unit test those adapters.

    Another tradeoff that I think it’s missing is the amount of LOC for testing vs for the “actual application” (I know, I know tests are part of the application, but you understood me).

    These days I’m mostly writing only integration tests. Let’s see if speed is a concern as these applications grow.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>