Rails/ActiveStorage: Client-Side encryption with Amazon S3 & dynamic service switching

Recently, we implemented client-side encryption for ActiveStorage - Which was a customer’s requirement in light of FISA and Schrems II GDPR rulings. In summary, relying only on Amazon’s KMS is not enough, so we implemented Client-Side encryption. Unfortunately, there are not a lot of guides out, that explain, how to best do this with ActiveStorage. So here is our take on it.

First, the resources I started with:

Andrew Kane’s blog post - Active Storage S3 Client-Side Encryption
Preview for Amazon S3 Client-Side Encrypted Active Storage files
AWS Ruby SDK which explains also ho
AWS guide on Client-Side Encryption which explains also how the Envelope Encryption works.

The code from Ankane is a good starting point, but missed a few things, like:

Previews
Direct Uploads
Support linking via proxy
Implement 100% of the Service API to be compatible with ActiveStorage - e.g. was missing downloads and chunked downloads
Variants
Implements the new AWS::S3::EncryptionV2 interface

Which we implemented (except previews - not tested). For our own convenience, we created a Gem that wraps the modified service:

pludoni/active_storage_client_side_encrypted

You can add the Gem, or just download the main service file: encrypted_s3_service.rb You will need Rails 7 because there are a lot of changes in ActiveStorage, that are not backwards-compatible. Also, it added tracked variants, which makes encrypting them too feasible.

AWS-SDK supports many different key formats. For starters, we only use a static 32-byte-length (256bit) key. That is the format, our adapter supports for now. Future implementations could also implement Public/Private key encryption.

Add the service:

encrypted_amazon:
  service: EncryptedS3  # <---- Important
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: <%= Rails.application.credentials.dig(:aws, :region) %>
  bucket: <%= Rails.application.credentials.dig(:aws, :bucket) %>
  # Static Encryption Key: 32 bytes - must be 32 bytes = 256bit length.
  # mark as base64 to encode all ascii characters
  # generate with: Base64.strict_encode64(OpenSSL::Cipher.new("AES-256-ECB").random_key)

  encryption_key: "base64:randomGiberish"

Then, either use it as a default service and you are down. Or use it for a specific attachment:

class MyModel < ApplicationRecord
  has_one_attached :contract_pdf, service: :encrypted_amazon
end

Change the service dynamically based on the tenant

In our case, we enabled that feature on a customer/tenant basis, so we are switching out the service dynamically:

# Tenant, User whatever
class Organisation < ApplicationRecord
  def with_s3_settings(&block)
    tenant_before = Current.organisation_tenant
    Current.organisation_tenant = self
    before = Rails.application.config.active_storage.service
    if self.encryption_enabled?
      if Rails.application.config.active_storage.service_configurations['encrypted_amazon']
        Rails.application.config.active_storage.service = :encrypted_amazon
        ActiveStorage::Blob.service = ActiveStorage::Blob.services.fetch(:encrypted_amazon)
      else
        Rails.logger.warn "ActiveStorage service #{setting} not configured"
      end
    end
    yield
  ensure
    Current.organisation_tenant = tenant_before
    Rails.application.config.active_storage.service = before
    ActiveStorage::Blob.service = ActiveStorage::Blob.services.fetch(before)
  end
end

Then, you can switch the service in a before_action or similar.

class ApplicationController < ActionController::Base
  around_action :set_active_storage_service

  # sign in

  def set_active_storage_service
    if Current.organisation_tenant
      Current.organisation_tenant.with_s3_settings do
        yield
      end
    else
      yield
    end
  end
end

If you also want to enable Direct Uploads with client-side encryption dynamically, you need to “patch” the ActiveStorage controller:

# config/initalizers/active_storage_direct_upload_patch.rb
module ASDirectUploadPatch
  def blob_args
    service_name = params[:service_name].presence
    super.merge(service_name: service_name)
  end
end

Rails.application.reloader.to_prepare do
  ActiveStorage::DirectUploadsController.prepend ASDirectUploadPatch
rescue Aws::Errors::MissingRegionError
end

# In the views/forms call the direct-upload-path like this:
<%= f.file_field :file, multiple: true, "data-direct-upload-url" => rails_direct_uploads_url(service_name: 'encrypted_amazon') %>

Rails didn’t implement it, because it is not safe, and the URL can be manipulated and different services chosen. But in our case, we have further tenant checks in place. Otherwise, you might want to pass a signed parameter instead of the plain service name instead.

Active Job

If you are using ActiveJobs that are creating or modifying blobs, it might be necessary to wrap them with the current tenant and switch out the service during each process:

class ApplicationJob < ActiveJob::Base
  queue_as :default

  # Serialize the whole Current object, so it can be passed to the job
  def serialize
    super.merge(current: Current.serialize)
  end

  # Load the Current metadata
  def deserialize(job_data)
    if job_data['current']
      Current.deserialize(job_data['current'])
    end
    super(job_data)
  end

  around_perform :set_active_storage_service

  # Call our method from above and switch out the service
  def set_active_storage_service
    if Current.organisation_tenant
      Current.organisation_tenant.with_s3_settings do
        yield
      end
    else
      yield
    end
  end
end

Impact on server load

The encryption is done by the “client”, which in this case is your server, that’s the one, holding the keys, so it also must do the work! Your web-app server (Puma etc.) must also handle ALL file uploads and downloads, so make sure you have enough extra capacity. That’s why in our case, we only selectively enable it for customers that need it, and also only for sensitive data. For example, we do not encrypt company logos, banners, custom CSS etc.

Feel free to try out the Gem. Let me know if you have any questions or suggestions via Mastodon ruby.social/@zealot128. Also, contributions to the Gem code are welcome.