assert_changes and assert_no_changes in Ruby

Update: This code and documentation is now available on github: http://github.com/ndp/assert_changes/tree/master

The Problem

On our work on gobalto.com, we spend time to have good fixture data for our tests– data that can represent all the important application states that our tests require. As a result, our tests are very dependent on the data. It’s important that someone doesn’t inadvertantly change it in subtle ways. This has led us to write not only asserts at the end of tests, but pre-conditions as well. For example,

    ...
    inotech = companies(:inotech)
    assert inotech.services.public.include?(categories(:a))
    assert inotech.services.public.include?(categories(:b))
    assert inotech.services.public.include?(categories(:c))

    post :edit_services_dialog, :id=>inotech.id,
                                          :service_category_id=>categories(:a).id
    inotech.reload

    assert inotech.services.public.include?(categories(:a))
    assert !inotech.services.public.include?(categories(:b))
    assert !inotech.services.public.include?(categories(:c))

Although the pre-conditions were introduced to guard against accidentally changing fixture data (or just to figure out what’s going on), we don’t necessarily delete them. They provide stronger tests. And in some ways, leaving in pre-conditions make the code more readable by providing documentation of your assumptions to the readers. The code above would be hard to follow without the clarifying pre-conditions. What do we expect to change and what stays the same?

Unfortunately all these asserts make the tests twice as long. And there’s a subtle readability problem: there’s no relationship between the corresponding pre- and post-conditions. In the example above, you have to scan carefully to see that the three assertions are repeated, but negated (b and c only). The reader must mentally put pieces together. And it’s not DRY. Using local variables doesn’t help much.

The Solution

I was inspired by a nice little test helper called assert_difference. It takes a string to evaluate and a block to execute. It’s useful for checking on state changes– especially database changes– during a test:

assert_difference('Company.count', -1) do
    Company.delete_one
end

(Without this method, we rely on a count of all database records, or merely look for specific ones. The former approach leads to brittle tests, and the latter to incomplete assertions.)

A limitation of assert_difference is that it only deals with integers. What if it were generalized? Here goes:

    i = true
    assert_changes 'i' => false do   # read as: i changes to false
      i = false
    end

The string passed to assert changes is evaluated in the block context, both before and after the block is run. So this block asserts that i becomes false (and by deduction, starts out as true). It executes asserts on both ends, just like we want.

Of course sometimes you want to be explicit about a state change, so you can specify both the starting and ending values using an array:

    o.answer = 'yes'
    assert_changes 'o.answer' => ['yes','no'] do
      o.answer = 'no'
    end

To handle the original example, you can pass multiple pre/post conditions of arbitrary complexity and they are all evaluated before and after the block is executed:

      assert_changes 'post(:a).status' => [:preview, :published],
                            'comment(:c).status' => [:preview, :deleted] do
        ...
      end

Finally, I added support for a :no_change symbol. Now I can re-write my original problem in a clearer form:

    assert_changes
        'inotech.services.public.include?(categories(:a))' => [true, :no_change],
            'inotech.services.public.include?(categories(:b))'=>false,
            'inotech.services.public.include?(categories(:c))'=>false do
      post :edit_services_dialog, :id=>inotech.id, :service_category_id=>categories(:a).id
      inotech.reload
    end

For completeness, I added assert_no_changes, with slightly extended parameter possibilities:

    i,j = 'hello','hi'
    assert_no_changes 'i' do ...  # i (before) == i (after)
    assert_no_changes 'i'=>'hello' do ... # i == 'hello' before and after
    assert_no_changes ['i','j'] do ... # neither i or j change
    assert_no_changes 'i'=>'hello','j'=>'hi' do # or be explicit

Update: This code and documentation is now available on github: http://github.com/ndp/assert_changes/tree/master

Thoughts? Comments?