Deploying Rails with Ansible with Dresden-Weekly toolbox (PostgreSQL, RVM, Sidekiq, Passenger)

In the past I used both Ansible and Capistrano to deploy my own Rails apps (as well as at my company); Ansible, for installing the provisioning, user accounts, and key and secret management, and Capistrano for each individual deployment.

Dresden Weekly Logo At Dresden-Weekly, a regularly meetup in Dresden, other developers faced similar deployment situations. Thus, we decided to throw our solutions together and create a suite of Rails deployment recipes (Ansible roles, as you will). As of today, Ansible-Rails consists of than 30 individual roles that are all part of the same repository and Ansible Galaxy role. All these roles are intended to be used together, as they use the same pool of variables.

Ansible Logo Now, I want to show you on how to use those roles for a complete deployment of Podfilter.de. I will use bare Ansible without an intermediate virtual environment, like Vagrant Ansible remote. If you are on Windows or can’t install recent versions of Ansible, this might be interesting for you.

With Ansible, we are going to provide:

PostgreSQL
Sidekiq Background workers as Userjobs
Redis (as needed by Sidekiq)
correct Ruby version with RVM and dependencies
imagemagick for image upload resizings
Logrotate for app logs
daily database dumps
Nginx + Passenger (without SSL, see bottom for explanation)

Installation

Make sure to have Ansible installed in a non-ancient version, like 1.9:

$ ansible --version
ansible 1.9.3

Add a Rolefile.yml and add the roles:

- src: "https://github.com/dresden-weekly/ansible-rails"
  version: "develop"
  name: "dresden-weekly.Rails"
- src: "debops.users"
  version: "v0.1.0"
- src: 'https://github.com/ANXS/postgresql.git'
  name: 'ANXS.postgresql'
  version: 'master'
- name: 'geerlingguy.redis'
  version: 'master'
  src: 'https://github.com/geerlingguy/ansible-role-redis.git'

For this example, I will install the most recent version of dresden-weekly.rails, as well as debops.user, Redis, and Postgresql roles.

Install the specified roles:

$ ansible-galaxy install -r Rolefile.yml --ignore-errors

Inventory & Provisioning

Next, add the server information to your inventory file (any file in your project directory), e.g. production:

[myapp]
some.fqdn-host.com ansible_ssh_host=12.23.34.45 ansible_ssh_user=root

[apps:children]
myapp

I recommend using a group or host for each app. Thus, you can use the same deployment script for all your apps and use a separate configuration for each app as a group_vars or host_vars file.

Here is a provisioning script:

# provision.yml
- name: Install Rails
  hosts: apps
  sudo: yes
  tags:
    - provisioning
  vars_files:
    - group_vars/pubkeys.yml
  vars:
    database_backup_base_dir: "/backup/databases"
  pre_tasks:
		# I've had problems with locale and postgresql, this might help
    - locale_gen: name=de_DE.UTF-8 state=present
    - locale_gen: name=en_US.UTF-8 state=present
  roles:
    - role: debops.users
      users_list:
      - name: ""
        comment: "Application user"
        sshkeys:
        - ""
    - role: ANXS.postgresql
      postgresql_databases:
        - name: ''
      postgresql_users:
        - name: ''
          role_attr_flags: CREATEDB,SUPERUSER
    - role: geerlingguy.redis
      when: install_redis
    - dresden-weekly.Rails/ruby/postgresql
    - dresden-weekly.Rails/ruby/rvm
		# imagemagick + rmagick dependencies + pngquant jpegoptim
    - dresden-weekly.Rails/ruby/imagemagick
    - dresden-weekly.Rails/rails/folders
    - dresden-weekly.Rails/rails/logrotate
		# set environment variables and profile script that changes to the app folder on login
    - role: dresden-weekly.Rails/user/profile
      rails_user_name: ""
      rails_user_bashrc_lines:
        - "cd  || true"
        - "cd  || true"
		# install nginx + configure app-site and enable it
    - dresden-weekly.Rails/nginx/passenger
		# simple daily crontab dump of pg database - sufficient for many app sizes
    - dresden-weekly.Rails/database/backup
		# Sidekiq + Upstart userjobs
    - role: dresden-weekly.Rails/rails/jobs/sidekiq
      when: 'rails_sidekiq_enabled is defined and rails_sidekiq_enabled == true'
    - role: dresden-weekly.Rails/upstart/userjobs
      users:
        - ""
      when: 'rails_sidekiq_enabled is defined and rails_sidekiq_enabled == true'

I’ve referenced an additional file, group_vars/pubkeys, where I store various SSH pubkeys. The file looks like this:

# group_vars/pubkeys.yml
pubkeys:
  stefan: 'ssh-rsa AAAAB3Nza........... user@host'
  somebody_else: 'ssh-rsa AAAAB3Nza........... user@host'
  deploy: 'ssh-rsa AAAAB3Nza........... user@host'

Additionally, you need the individual app config, e.g. group_vars/myapp.yml:

# group_vars/myapp.yml
ruby_version: '2.2.3'
# where and whom to deploy
app_name: myapp
app_user: ''
app_path: '/home//app'
# what ode to use
git_url: 'https://github.com/zealot128/podfilter.git'
git_branch: 'master'
rails_env: production
# nginx settings
app_domain: 'myapp.de'
app_hosts: 'myapp.de www.myapp.de'
# on which host to run migrations
rails_primary_node: ''
rails_database_name: '_'
rails_database_user: ''
postgresql_version: '9.4'
postgresql_locale: 'de_DE.UTF-8'
# add additional folders to symlink, like uploads
rails_deploy_custom_shared_folders: [ 'public/uploads' ]
# Sidekiq with 5 workers
rails_sidekiq_enabled: true
sidekiq_configuration_concurrency: 5
install_redis: true
# enable cronjobs with whenever
whenever: true
# global user env variables: You can either use those or the config/secrets.yml for
# getting configuration and secrets into your app
rails_user_env:
  SIDEKIQ_WEB_USER: admin
  SECRET_TOKEN: 'rake secret output here'
  TWITTER_CONSUMER_SECRET: key
  TWITTER_CONSUMER_KEY: secret
  SIDEKIQ_WEB_PASSWORD: password
  RAILS_ENV: ''

# by default, only a couple of files/folders are copied over, you might
#  whant to add additional
rails_deploy_custom_archive:
- bin
- Gemfile.lock
#- private
#- engines

# automatically backup database daily
database_backup_name: ""
# provide secret files that are copied over on each deploy
rails_provisioned_files:
  - file: config/secrets.yml
    yaml:
      production:
        http_username: admin
        http_password: somepassword
        secret_key_base: 'tip: run rake secret in a rails app to get a good secret'
        twitter_consumer_key: "somekey"
        twitter_consumer_secret: "somesecret"
        twitter_access_token: "somekey"
        twitter_access_token_secret: "somesecret"

  - file: config/database.yml
    yaml:
      production:
        adapter: postgresql
        database: ''
        encoding: UTF8
        pool: 30
# You might want to symlink additional files
# rails_shared_files:
#   - db/production.sqlite3

Run it!

$ ansible-playbook -i production provision.yml

You only need to run the provision.yml once or when you change variables, like Passenger/Nginx options (more runs should not be harmful though)..

Run an individual deploy

Now everything is set up. You can deploy the app code. This task is intended to be run every time you want to deploy app changes. Here a deployment playbook:

# deploy.yml
- name: 'Deploy app'
  hosts: apps
  sudo: yes
  tags: deploy
  sudo_user: ''
  vars:
    profile: '/bin/bash -lc -- '
  roles:
    - dresden-weekly.Rails/rails/folders
    - dresden-weekly.Rails/rails/create-release
    - role: dresden-weekly.Rails/rails/jobs/sidekiq/restart
      when: 'rails_sidekiq_enabled is defined and rails_sidekiq_enabled == true'
    - dresden-weekly.Rails/rails/tasks/bundle
    - dresden-weekly.Rails/rails/tasks/compile-assets
    - dresden-weekly.Rails/rails/tasks/migrate-database
    - dresden-weekly.Rails/rails/update-current
    - role: dresden-weekly.Rails/rails/tasks/whenever
      when: whenever
    - dresden-weekly.Rails/rails/cleanup-old-releases

Yeah, that’s it. Run it to deploy the code:

$ ansible-playbook -i production deploy.yml

This will deploy the app similar like Capistrano:

Create folders app/releases app/shared app/repo
Checkout Code to app/repo, export code to releases/RELEASE_ID
bundle install --deployment to shared/bundle
Gracefully shutdown any existing Sidekiq workers (Tell sidekiq to not accept any further work)
compile the assets rake assets:precompile
run migrations
change the current symlink
update crontab through whenever
remove everything but the most recent 5 releases
Handlers in the end:
- touch tmp/restart.txt and trigger Passenger restart
- restart Sidekiq workers

Feel free to browse the other roles and each role’s different configuration option. Also check out the example deployment repositories.

If you miss some essential roles or configuration options, feel free to add issues (or Pull Requests ;-D). For example, the current Nginx Passenger role does not support SSL, as I use a proxy for my own setup und don’t need SSL on the app server.