infra-vintage/mastodon

pub.solar mastodon

https://mastodon.pub.solar

Upgrading Mastodon

This section assumes you edited docker-compose.yml and bumped the mastodon docker image version tag

# check current trailing number of mastodon containers
# current_container_index
docker ps | grep -E "sidekiq|streaming|web"

# make a DB backup and copy it to your local machine
docker exec -it matrix-postgres bash

# in shell in matrix-postgres container:
pg_dump -U mastodon -d mastodon_production -W -f /root/mastodon_db_backup-$(date +%F).sql
exit

# copy backup to local machine
docker cp matrix-postgres:/root/mastodon_db_backup-$(date +%F).sql .

# download new mastodon docker images
docker-compose --project-name blue-mastodon pull

# run pre-update migrations
docker-compose run \
  --rm \
  -e SKIP_POST_DEPLOYMENT_MIGRATIONS=true \
  web \
    rails db:migrate

# create new containers with new mastodon version
docker-compose --project-name blue-mastodon up \
  --detach \
  --scale web=2 \
  --scale streaming=2 \
  --scale sidekiq=2 \
  --no-recreate

# stop containers with old mastodon version
docker stop \
  blue-mastodon_web_($current_container_index - 1) \
  blue-mastodon_streaming_($current_container_index - 1) \
  blue-mastodon_sidekiq_($current_container_index - 1)

# run post-deployment migrations
docker-compose run --rm web rails db:migrate

# clean up containers with old mastodon version
docker rm \
  blue-mastodon_web_($current_container_index - 1) \
  blue-mastodon_streaming_($current_container_index - 1) \
  blue-mastodon_sidekiq_($current_container_index - 1)

Upgrading Caddy

mkdir -p certificates/acme-v02.api.letsencrypt.org-directory
docker cp --archive blue-mastodon_caddy_2:/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/files.pub.solar ./certificates/acme-v02.api.letsencrypt.org-directory/
docker cp --archive blue-mastodon_caddy_2:/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/mastodon.pub.solar ./certificates/acme-v02.api.letsencrypt.org-directory/

docker-compose --project-name blue-mastodon up \
  --detach \
  --scale caddy=2 \
  --no-recreate \
  --no-start

docker cp --archive ./backups/certificates blue-mastodon_caddy_3:/data/caddy/certificates
docker start blue-mastodon_caddy_3

# Stop old caddy container
docker stop blue-mastodon_caddy_2

# Verify everything works fine, then remove the old caddy container
docker rm blue-mastodon_caddy_2

Upgrading Elasticsearch

Look for new releases on https://www.elastic.co/guide/en/elasticsearch/reference/7.17/es-release-notes.html and edit the docker image tag accordingly.

docker-compose --project-name blue-mastodon up \
  --detach \
  --scale elasticsearch=2 \
  --no-recreate

# Stop old elasticsearch container
docker stop blue-mastodon_elasticsearch_2

docker exec -it blue-mastodon_web_15 bash
tootctl search deploy

# Verify everything works fine, then remove the old caddy container
docker rm blue-mastodon_elasticsearch_2

Todos:

• implement automatic backups, they are only done manually during upgrades at the moment
• switch proxy from nginx-dehydrated to caddy - done