Rails: add maintenance (read-only) mode to move everything off to a new server

On a recent larger migration, we wanted to move our whole app, including the database to a new server. The easiest way while keeping consistency was to put the app in maintenance mode, so no new data would be written to the (old) database. On the other hand, we have many read-only requests, so we could keep the app running for those, without any global maintenance page.

Simplest way: App wide downtime with a (styled) 503 page

Rails by default already provides html pages in public. The easiest way to make “maintenance mode” is to just kill your production puma processes, and let the webserver serve the 503 page. This is a global downtime, but it’s the easiest way to do it.

Better idea: Read-Only Mode via Rack Middleware

One way to address a Read-Only Mode, is to use a Rack-Middleware and only handle GET-requests. On the other hand, your GET requests might still change data (such as Analytics), or Cron-Jobs / Recurring Tasks in the background might still perform write operations. Furthermore, if you not only need to migrate the database, but also move the app, or perform larger operations, you might also want to check that no other write operations are performed - such as: Modification of files, performing API-calls, Mail-Deliveries, etc..

Our solution: patching Rails Core-Models

In our app, all database write operations are done via ActiveRecord and most API-calls and other side-effects are handled in ActiveJobs. So we decided to patch ActiveRecord and ActiveJob to raise an exception when trying to write data:

# config/initializers/read_only_mode.rb

module ArBaseReadonlyPatch
  extend ActiveSupport::Concern
  included do
    def readonly?
      ReadonlyMode.enabled?
    end
  end
end

ActiveSupport.on_load(:active_record) { include ArBaseReadonlyPatch }

module AjBaseReadonlyPatch
  extend ActiveSupport::Concern
  included do
    before_enqueue do
      if ReadonlyMode.enabled?
        raise ReadonlyMode::EnqueueStopped, 'Readonly mode is enabled'
      end
    end
  end
end
ActiveSupport.on_load(:active_job) { include AjBaseReadonlyPatch }

This initializer will patch both ActiveRecord and ActiveJob to raise an exception when trying to write data. This way, we can keep the app running, but no new data will be written.

Example implementation of a ReadonlyMode backed by Redis:

# app/models/readonly_mode.rb
module ReadonlyMode
  module_function
  class EnqueueStopped < StandardError; end

  def enabled? = client.get('readonly_mode') == 'true'
  def enable! = client.set('readonly_mode', 'true')
  def disable! = client.set('readonly_mode', 'false')

  def client
    @client ||=
      if ENV['REDIS_PORT_6379_TCP_PORT']
        Redis.new(host: ENV['REDIS_PORT_6379_TCP_ADDR'], port: ENV['REDIS_PORT_6379_TCP_PORT'], db: 0)
      else
        Redis.new
      end
  end
end

Notifying users about the maintenance

When you put your app in maintenance mode, you might want to notify your users about it. You can do this by adding a global flash message in your application layout:

# set notification in ApplicationController
class ApplicationController < ActionController::Base
  def check_readonly_mode
    if ReadonlyMode.enabled?
      flash.now[:alert] = t('readonly.identity_notice')
    end
  end

  rescue_from ReadonlyMode::EnqueueStopped do
    flash[:alert] = t('readonly.identity_notice')
    redirect_to root_path
  end

  rescue_from ActiveRecord::ReadOnlyRecord do
    flash[:alert] = t('readonly.identity_notice')
    redirect_to root_path
  end
end

# use notification in appropriate signed in controllers
class UsersController < ApplicationController
  before_action :check_readonly_mode
  ...
end

Thoughts and other considerations

• Ahoy: That analytics Gem creates visits and potentially events on every request. You might want to patch that as well (or disable it during).
• SNS / SQS: If you receive notifications via webhook, you might want to write those out to a file, archive it to replay them later - In our case, we just wrote them into a directoy instead of processing during ReadonlyMode and notified us via Exception Tracking about it.
• Incoming Emails via ActionMailbox or MailRoom. Depending on the provider, you might want to disable delivery before or - like SNS - write them out to a file with notification to you.
• Drain your ActiveJob-Queue and stop the workers
• If you use Cronjobs, disable those as well