Start Testing Your Migrations. (Right Now)

Posted on by in Process

Migrations are a necessary part of any Rails project. At some point, you need to modify the structure of your database, normalize data that exists within those tables, or refactor the architecture entirely. The migrations for these tasks can vary in complexity from simple “add_column” statements to extreme SQL with nested joins and subselects.

To be blunt, you can fark your data pretty easily by fat-fingering a migration. So how do we avoid this?

In our daily routine as Agile developers, we follow a simple workflow to avoid these situations. You write a test before you touch a single piece of code, then watch it fail, then write code and repeat until your tests pass. So why not start writing tests for your migrations?

Suggested Workflow

I currently use RSpec for testing. We have a default structure in our spec folder for models, controllers, etc. Let’s try to come up with a pattern for migrations that mimics the existing behavior. To make it more straightforward, let’s also use a real-world example: Adding a column to the “users” table called “favorite_color”:

  1. Create your migrations folder for specs (if it doesn’t exist):

    mkdir spec/migrations

  2. Create an empty migration:

    rails g migration AddFavoriteColorToUsers
    # => db/migrate/20110127192508_add_favorite_color_to_users.rb

  3. Get the current migration version (you will use this in your test):

    rails runner 'puts ActiveRecord::Migrator.current_version'
    # => 20110126212851

  4. Create your migration spec:

    touch spec/migrations/add_favorite_color_to_users_spec.rb

    • Load the required files:
              load 'spec/spec_helper.rb'
              load 'db/migrate/20110127192508_add_favorite_color_to_users.rb'
            
    • Setup your spec:
              describe AddFavoriteColorToUsers do
                before do
                  @my_migration_version = '20110127192508'
                  @previous_migration_version = '20110126212851'
                end
                pending "describe up"
                pending "describe down"
              end
            
    • Describe your “up” migration:
              describe ".up" do
                before do
                  ActiveRecord::Migrator.migrate @previous_migration_version
                  puts "Testing up migration for #{@my_migration_version} - resetting to #{ActiveRecord::Migrator.current_version}"
                end
      
                it "adds the 'favorite_color' column to the users table" do
                  expect { 
                    AddFavoriteColorToUsers.up 
                    User.reset_column_information
                  }.to change { User.columns }
                  User.columns.map(&:name).should include("favorite_color")
                end
              end
            
    • Describe your “down” migration:
              describe ".down" do                                                                                                                
                before do
                  ActiveRecord::Migrator.migrate @my_migration_version
                  puts "Testing down migration for #{@my_migration_version} - resetting to #{ActiveRecord::Migrator.current_version}"
                end
      
                it "removes the 'favorite_color' column from the users table" do
                  expect {
                    AddFavoriteColorToUsers.down
                    User.reset_column_information
                  }.to change { User.columns }
                  User.columns.map(&:name).should_not include("favorite_color")
                end
              end
            
  5. Get on TDD with Migrations! (Red,Green,Refactor)

Of course, you’ll need to dig into ActiveRecord internals to know how to write your tests. Then again, there’s nothing wrong with a deeper understanding of the framework we all use every day.

At this point, it may seem pointless to write a spec for this migration. Perhaps you’re right. All we’re doing is adding and removing a column from the database. But migrations aren’t always so straightforward. Sometimes your migrations execute advanced SQL for data normalization. Sometimes, they do things like persisting tertiary relationship ids to local columns, deleting stale records based off of nested selects and multiple joins, or removing duplicated records that have made their way into your production database.

Let’s dig in a little deeper, and see if I can’t convince you to start testing your migrations, right now.

Example Scenario: Delete Stale Records

Let’s take the following scenario and look at a migration that is just dying for a test.

Delete Alerts for Comments on Soft-Deleted Blog Post

Here, we have a blog application written in Rails (how original of me) with the following properties:

  • As an Author I can create a Blog Post
  • As a Reader I can comment on that Blog Post
  • As an Author I receive an Alert when someone comments on my Blog Post
  • As an Administrator I can “Soft Delete” any Blog Post

The bug we find in this application stems from the soft deletion of the blog post and their associated alerts. Users are seeing alerts for comments on soft-deleted blog posts after we introduced this change. We’ve written some new code to ensure that alerts on “deleted” posts are destroyed, but we also need to write a migration to remove the stale alerts from the database. This is made a little more complicated by the fact that the “alerts” table backs a polymorphic STI “Alert” model.

The Test (RSpec and Mocha)

load 'spec/spec_helper.rb'
load 'db/migrate/20110127220314_delete_alerts_for_deleted_posts.rb'


describe DeleteAlertsForDeletedPosts do
  before do
    @my_version = "20110127220314"
    @previous_version = "20110126212851"
    
    Alert.delete_all
    Post.delete_all
  end

  describe ".up" do
    before do
      ActiveRecord::Migrator.migrate @previous_version
    end

    context "given some deleted posts and some active posts" do
      before do
        5.times { Factory(:soft_deleted_post) }
        5.times { Factory(:post) }

        @deleted_posts = Post.where :deleted => true
        @active_posts = Post.where :deleted => false

        @deleted_posts.should have(5).items
        @active_posts.should have(5).items
      end

      context "when there are alerts for all the posts" do
        before do
          @deleted_posts.concat(@active_posts).each do |post|
            Factory :alert, :alertable => post
          end
        end

        it "deletes alerts for the deleted posts" do
          DeleteAlertsForDeletedPosts.up
          Post.where(:deleted => true).map(&:alerts).flatten.should be_empty
        end

        it "does not delete alerts for the active posts" do
          DeleteAlertsForDeletedPosts.up 
          Post.where(:deleted => false).map(&:alerts).flatten.should_not be_empty
        end
      end
    end
  end

  describe ".down" do
    it "does nothing" do
      true
    end
  end

  after(:all) do
    # Return to latest state
    ActiveRecord::Migrator.migrate "db/migrate"
  end
end

The Migration

 class DeleteAlertsForDeletedPosts < ActiveRecord::Migration
  def self.up
    execute <<-EOS
    DELETE FROM alerts
     WHERE alerts.alertable_type = 'Post'
     AND alerts.alertable_id IN (
      SELECT id FROM posts WHERE deleted = 1
     )
    EOS
  end

  def self.down
    puts "Previous migration removed stale alerts, there is nothing to do in this .down"
  end
end

Before I started testing my migrations with specs, I would do this by starting a Rails console and manually running each of these scenarios, then dropping back to the shell to run my migration, then back to the console to check the results. Perhaps you’re OK with doing things that way. To me, I find it wasteful and prone to error. By having the setup and teardown in my test’s “before” and “after” blocks, I can quickly make changes and verify the success or failure of my migration.

Plus, I’ve really grown to like the colors RED and GREEN.

Summary

Others may argue that migrations are not Ruby code and don’t need to be tested. Some will say that migrations are so straightforward that they don’t need to be tested. Both of these are valid points, and perhaps this is a bit of overkill. I am still experimenting with this process and trying to find out where these tests belong. I still have some questions about how to approach utilizing these tests in the full lifecycle of an application. But I’m a perfectionist, and I’ve seen cases where untested migrations cause very unexpected results. To me, migrations that can munge data on a production system should have some level of testing. There should be a flag (RED) that shows us developers that our migration doesn’t do what we expected it to do. I can’t think of a better solution for this than to start testing your migrations. Right now.

Let me know what you think. Comments, questions, suggestions are all appreciated. Keep in mind that I am an opinionated developer and my views do not necessarily always reflect those of Carbon Five.