blog.thms.uk

Moving Mastodon to Docker Compose

Now that I understand docker a lot better, I figured it’s finally time to tackle the migration of my mastodon server to docker.

This post outlines how I’ve done the migration. Turns out it’s really quite straightforward!

Inventory current setup

Firstly, we take stock of what we have: take a note of all the versions of our software dependencies, so we can point our docker compose files at the same versions. You can check all of these at the bottom of /admin/dashboard. For me these were:

Get docker compose file

Now let’s get the docker compose file from the official source. (I run glitch-soc, so I got it from https://github.com/glitch-soc/mastodon/blob/main/docker-compose.yml. If you run stock mastodon get yours from https://github.com/mastodon/mastodon/blob/main/docker-compose.yml.)

Then make some changes:

  1. Un-comment the es service.
  2. Point es, db, redis, web, streaming and sidekiq to the correct versions.
  3. In the web service, un-comment depends_on: es

Migrate config

One thing that’s probably a bit controversial, but I do like to keep as much of my config as possible in the docker-compose.yml file. This allows me to add the file to git so I can version control it, and quickly see whatever might change.

In order to do this I’ll add a shared yaml anchor, with all the non-secret environment variables, and then reference this within each of the mastodon services:

x-mastodon-env: &mastodon-env
  LOCAL_DOMAIN: thms.uk
  WEB_DOMAIN: mstdn.thms.uk
  [...]

services:
  web:
    image: ghcr.io/glitch-soc/mastodon:v4.6.0-rc.1
    restart: always
    env_file: .env.production          # secrets only
    environment:
      <<: *mastodon-env
    [..]

  streaming:
    image: ghcr.io/glitch-soc/mastodon-streaming:v4.6.0-rc.1
    restart: always
    env_file: .env.production          # secrets only
    environment:
      <<: *mastodon-env
    [..]

  sidekiq:
    image: ghcr.io/glitch-soc/mastodon:v4.6.0-rc.1
    restart: always
    env_file: .env.production          # secrets only
    environment:
      <<: *mastodon-env
    [..]

My .env.production file is now really short:

SECRET_KEY_BASE=
OTP_SECRET=
VAPID_PRIVATE_KEY=
AWS_SECRET_ACCESS_KEY=
SMTP_PASSWORD=
DEEPL_API_KEY=
ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=
ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=
ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=

All of these values are simply copied from my production .env.production file.

Pull images so they are ready

We’ll pull the images, so they are ready to use when we want to migrate data. This just shortens our downtime slightly:

docker compose pull

Migrate data

Migrating the data is the same as it’s always been really. I host my media on Backblaze B2, so I don’t need to worry about this.

I also won’t bother migrating the elasticsearch index: this can simply be recreated afterwards.

As such, I need to migrate my SQL and redis databases.

We’ll firstly shut down the old system so we get a consistent state:

sudo systemctl stop mastodon-*

Then we make a postgres dump:

sudo -u mastodon pg_dump -Fc -p 5433 -d mastodon_production > mastodon.dump

And make sure we have an up-to-date redis snapshot:

redis-cli SAVE

On the new docker host we’ll now copy both files across:

scp old-vm:~/mastodon.dump ~/mastodon.dump
mkdir ./redis
scp old-vm:/var/lib/redis/dump.rdb ./redis/dump.rdb

We need to adjust permissions on the redis directory:

sudo chown -R 999:999 ./redis

And bring the redis service up:

docker compose up -d redis
docker compose logs redis

In the logs look for “DB loaded from disk”.

That’s redis ready. To import the postgres DB, firstly bring up the db service, and create the user and database:

docker compose up -d db
docker compose exec -T db psql -U postgres -c "CREATE ROLE mastodon LOGIN CREATEDB;"
docker compose exec -T db psql -U postgres -c "CREATE DATABASE mastodon_production OWNER mastodon;"

Then run the pg_restore command:

docker compose exec -T db pg_restore -U mastodon -d mastodon_production --no-owner < ~/mastodon.dump

Bring up new server

Now that this is all done, we just bring up the remaining services:

docker compose up -d

And finally we re-deploy the search index:

docker compose run --rm web nice -n 19 bin/tootctl search deploy

Wrapping up

That’s the migration done: mastodon is now running entirely on docker, with the bulk of my config version-controlled in a single docker-compose.yml. Before tearing down the old VM, give everything a once-over: check the web UI loads, federation is working, and the search index has finished deploying. Once you’re happy, the old systemd setup can go.

Postscript

I have since published the git repo which I use to track my docker compose files on Forgejo: https://code.thms.uk/michael/mastodon-compose.