How do I test Rails migrations?

I want to test that certain conditions hold after running a migration I've written. What's the current best way to do that?

To make this concrete: I made a migration that adds a column to a model, and gives it a default value. But I forgot to update all the pre-existing instances of that model to have that default value for the new column. None of my existing tests will catch that, because they all start with a fresh database and add new data, which will have the default. But if I push to production, I know things will break, and I want my tests to tell me that.

I've found http://spin.atomicobject.com/2007/02/27/migration-testing-in-rails/, but haven't tried it. It's very old. Is that the state-of-the-art?

Peter Marklund has an example gist of testing a migration here: https://gist.github.com/700194 (in rspec).

Note migrations have changed since his example to use instance methods instead of class methods.

Here's a summary:

1. Create a migration as usual
2. Create a file to put your migration test in. Suggestions: test/unit/import_legacy_devices_migration_test.rb or spec/migrations/import_legacy_devices_migration_spec.rb NOTE: you probably need to explicitly load the migration file as rails will probably not load it for you. Something like this should do: require File.join(Rails.root, 'db', 'migrate', '20101110154036_import_legacy_devices')
3. Migrations are (like everything in ruby), just a class. Test the up and down methods. If your logic is complex, I suggest refactoring out bits of logic to smaller methods that will be easier to test.
4. Before calling up, set up some some data as it would be before your migration, and assert that it's state is what you expect afterward.

I hope this helps.

UPDATE: Since posting this, I posted on my blog an example migration test.

UPDATE: Here's an idea for testing migrations even after they've been run in development.

EDIT: I've updated my proof-of-concept to a full spec file using the contrived example from my blog post.

# spec/migrations/add_email_at_utc_hour_to_users_spec.rb
require 'spec_helper'

migration_file_name = Dir[Rails.root.join('db/migrate/*_add_email_at_utc_hour_to_users.rb')].first
require migration_file_name


describe AddEmailAtUtcHourToUsers do

  # This is clearly not very safe or pretty code, and there may be a
  # rails api that handles this. I am just going for a proof of concept here.
  def migration_has_been_run?(version)
    table_name = ActiveRecord::Migrator.schema_migrations_table_name
    query = "SELECT version FROM %s WHERE version = '%s'" % [table_name, version]
    ActiveRecord::Base.connection.execute(query).any?
  end

  let(:migration) { AddEmailAtUtcHourToUsers.new }


  before do
    # You could hard-code the migration number, or find it from the filename...
    if migration_has_been_run?('20120425063641')
      # If this migration has already been in our current database, run down first
      migration.down
    end
  end


  describe '#up' do
    before { migration.up; User.reset_column_information }

    it 'adds the email_at_utc_hour column' do
      User.columns_hash.should have_key('email_at_utc_hour')
    end
  end
end

I just create an instance of the class, then call up or down on on it.

For example:

require Rails.root.join(
  'db',
  'migrate',
  '20170516191414_create_identities_ad_accounts_from_ad_account_identity'
)

describe CreateIdentitiesAdAccountsFromAdAccountIdentity do
  subject(:migration) { described_class.new }

  it 'properly creates identities_ad_accounts from ad account identities' do
    create_list :ad_account, 3, identity_id: create(:identity).id

    expect { suppress_output { migration.up } }
      .to change { IdentitiesAdAccount.count }.from(0).to(3)
  end
end

> I made a migration that adds a column to a model, and gives it a default value. But I forgot to update all the pre-existing instances of that model to have that default value for the new column.

Based on this statement, you are just trying to test that an "old" model, has the default, correct?

Theoretically you are testing if rails works. I.e., "Does rails set a default value to a newly added column"

Adding a column and setting a default value will be there in the "old" records of your database.

So, you don't need to update the other records to reflect the default setting, then. In theory there is nothing to test, as rails has tested that for you. Lastly, the reason to use defaults is so that you don't have to update the previous instances to use that default, right?

> Note: This answer might not actually target the question above. I am writing this for viewers who are here for knowing how to write tests for migrations in Rails.

This is how I did it

---

Step 1

You need to configure RSpec to use DatabaseCleaner

# spec/support/db_cleaner.rb
RSpec.configure do |config|
  config.around(:each) do |example|
    unless example.metadata[:manual_cleaning]
      DatabaseCleaner.strategy = :transaction
      DatabaseCleaner.cleaning { example.run }
    else
      example.run
    end
  end
end

This will run all your examples in transaction mode which is super fast. And, also you need to run migration tests in truncation mode because you need to make actual database hits.

>Note: You might not need to do as above if you are using truncation as strategy for DatabaseCleaner.

---

Step 2

Now, you can choose whether you want transaction for that example or group of example using manual_cleaning clause like below.

# spec/migrations/add_shipping_time_settings_spec.rb
require 'spec_helper'
require_relative  '../../db/migrate/20200505100506_add_shipping_time_settings.rb'

describe AddShippingTimeSettings, manual_cleaning: true do
  before do
    DatabaseCleaner.strategy = :truncation
    DatabaseCleaner.clean # Cleaning DB manually before suite
  end

  describe '#up' do
    context 'default values in database' do
      before do
        AddShippingTimeSettings.new.up
      end

      it 'creates required settings with default values' do
        data = Setting.where(code: AddShippingTimeSettings::SHIPPING_TIMES)
        expect(data.count).to eq(AddShippingTimeSettings::SHIPPING_TIMES.count)
        expect(data.map(&:value).uniq).to eq(['7'])
      end
    end
  end

  describe '#down' do
    context 'Clean Up' do
      before do
        AddShippingTimeSettings.new.up
        AddShippingTimeSettings.new.down
      end

      it 'cleans up the mess' do
        data = Setting.where(code: AddShippingTimeSettings::SHIPPING_TIMES)
        expect(data.count).to eq(0)
      end
    end
  end
end

I don't know Rails, but I think the approach is the same independently from the tooling I use the following approach:

• make sure deployed versions of database scripts are apropiatly tagged/labeled in Version Control
• based on that you need at least three scripts: a script that creates the old version from scratch (1), a script that creates the new version from scratch (2) and a script that creates the new version from the old version (3).
• create two db instances/schemata. In one run script 2, in the other run script 1 followed by script 3
• compare the results in the two databases, using sql queries against the data dictionary.

For testing also the effect on actual data, load test data into the databases after executing script 2 and between 1 and 3. Again run sql queries, compare the results

This is maybe not the most Railsy answer; Constrain your database.

If you had declared your column not null (null: false in rails migrations) the database wouldn't let you forget to provide a default value.

Relational databases are really good at enforcing constraints. If you get in the habit of adding them you can guarantee the quality of your data.

Imagine if you add a presence validation after some data already exists in production where that validation would fail. First, the validation won't run until the user tries to edit the data and when it does it may not be clear to the user what is causing the error because they may not be concerned with that particular value at this time. Second, your UI may expect that value to exist (after all your validation "guarantees" it) and you'll end up getting a page about an unexpected nil at 2AM. If you constrain the column as not null at the time you add the validation, the database will back-check all existing data and force you to fix it before the migration will complete.

While I use not null in this example the same holds true for a uniqueness validation and really anything else you can express with a constraint.

You could consider running isolated portions of your test suite with specific settings against copies of your production data (with e.g. something like yaml_db).

It's a bit meta, and if you know what the potential problems are with your new migrations you'd likely be better off just enhancing them to cover your specific needs, but it's possible.

describe 'some_migration' do
  it 'does certain things' do
    context = ActiveRecord::Base.connection.migration_context
    # The version right before some_migration
    version = 20201207234341
    # Rollback to right before some_migration
    context.down(version)

    set_up_some_data

    context.migrate
    # Or if you prefer:
    # context.forward(1)

    expect(certain_things).to be(true)
  end
end