pub-solar/matrix-docker-ansible-deploy
5
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Make gzipped SQL dumps by default during --upgrade-postgres

Browse source
This commit is contained in:
Slavi Pantaleev 2019-10-05 11:41:59 +03:00
parent 29526e7bb1
commit e32aaacaa7
3 changed files with 11 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
CHANGELOG.md
View file

@ -11,6 +11,8 @@ For now, the playbook only uses that one database (`homeserver`) and that one si
However, in the future, additional components besides Synapse may also make use the Postgres database server. However, in the future, additional components besides Synapse may also make use the Postgres database server.
One such example is the [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge, which strongly encourages use of Postgres in its v1.0 release. We are yet to upgrade to it. One such example is the [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge, which strongly encourages use of Postgres in its v1.0 release. We are yet to upgrade to it.
Additionally, Postgres [upgrading](docs/maintenance-postgres.md#upgrading-postgresql) now uses gzipped dump files by default, to minimize disk space usage.
# 2019-10-04 # 2019-10-04

4
docs/maintenance-postgres.md
View file

@ -74,4 +74,8 @@ The auto-upgrade-backup directory stays around forever, until you **manually dec
As part of the upgrade, the database is dumped to `/tmp`, an upgraded and empty Postgres server is started, and then the dump is restored into the new server. As part of the upgrade, the database is dumped to `/tmp`, an upgraded and empty Postgres server is started, and then the dump is restored into the new server.
To use a different directory for the dump, pass some extra flags to the command above, like this: `--extra-vars="postgres_dump_dir=/directory/to/dump/here"` To use a different directory for the dump, pass some extra flags to the command above, like this: `--extra-vars="postgres_dump_dir=/directory/to/dump/here"`
To save disk space in `/tmp`, the dump file is gzipped on the fly at the expense of CPU usage.
If you have plenty of space in `/tmp` and would rather avoid gzipping, you can explicitly pass a dump filename which doesn't end in `.gz`.
Example: `--extra-vars="postgres_dump_name=matrix-postgres-dump.sql"`
**All databases, roles, etc. on the Postgres server are migrated**. **All databases, roles, etc. on the Postgres server are migrated**.

7
roles/matrix-postgres/tasks/upgrade_postgres.yml
View file

@ -7,7 +7,7 @@
- name: Set postgres_dump_name, if not provided - name: Set postgres_dump_name, if not provided
set_fact: set_fact:
postgres_dump_name: "matrix-postgres.out" postgres_dump_name: "matrix-postgres-dump.sql.gz"
when: "postgres_dump_name|default('') == ''" when: "postgres_dump_name|default('') == ''"
- name: Set postgres_auto_upgrade_backup_data_path, if not provided - name: Set postgres_auto_upgrade_backup_data_path, if not provided
@ -81,7 +81,9 @@
--entrypoint=/bin/sh --entrypoint=/bin/sh
-v {{ postgres_dump_dir }}:/out -v {{ postgres_dump_dir }}:/out
{{ matrix_postgres_detected_version_corresponding_docker_image }} {{ matrix_postgres_detected_version_corresponding_docker_image }}
-c "pg_dumpall -h matrix-postgres > /out/{{ postgres_dump_name }}" -c "pg_dumpall -h matrix-postgres
{{ '| gzip -c ' if postgres_dump_name.endswith('.gz') else '' }}
> /out/{{ postgres_dump_name }}"
- name: Ensure matrix-postgres is stopped - name: Ensure matrix-postgres is stopped
service: service:
@ -125,6 +127,7 @@
-v {{ postgres_dump_dir }}:/in:ro -v {{ postgres_dump_dir }}:/in:ro
{{ matrix_postgres_docker_image_latest }} {{ matrix_postgres_docker_image_latest }}
-c "cat /in/{{ postgres_dump_name }} | -c "cat /in/{{ postgres_dump_name }} |
{{ 'gunzip |' if postgres_dump_name.endswith('.gz') else '' }}
grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' | grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' | grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres" psql -v ON_ERROR_STOP=1 -h matrix-postgres"