Fix Postgres database importing/upgrading conflicts

We were running into conflicts, because having initialized
the roles (users) and databases, trying to import leads to
errors (role XXX already exists, etc.).

We were previously ignoring the Synapse database (`homeserver`)
when upgrading/importing, because that one gets created by default
whenever the container starts.

For our additional databases, it's a similar situation now.
It's not created by default as soon as Postgres starts with an empty
database, but rather we create it as part of running the playbook.

So we either need to skip those role/database creation statements
while upgrading/importing, or to avoid creating the additional database
and rely on the import for that. I've gone for the former, because
it's already similar to what we were doing and it's simpler
(it lets `setup_postgres.yml` be the same in all scenarios).
This commit is contained in:
Slavi Pantaleev 2020-12-14 22:28:20 +02:00
parent 2a502db239
commit dd797ba6a7
4 changed files with 34 additions and 4 deletions

View file

@ -1100,6 +1100,20 @@ matrix_postgres_additional_databases: |
}] if (matrix_dimension_enabled and matrix_dimension_database_engine == 'postgres' and matrix_dimension_database_hostname == 'matrix-postgres') else []) }] if (matrix_dimension_enabled and matrix_dimension_database_engine == 'postgres' and matrix_dimension_database_hostname == 'matrix-postgres') else [])
}} }}
matrix_postgres_import_roles_to_ignore: |
{{
[matrix_postgres_connection_username]
+
matrix_postgres_additional_databases|map(attribute='username')
}}
matrix_postgres_import_databases_to_ignore: |
{{
[matrix_postgres_db_name]
+
matrix_postgres_additional_databases|map(attribute='name')
}}
###################################################################### ######################################################################
# #
# /matrix-postgres # /matrix-postgres

View file

@ -43,6 +43,22 @@ matrix_postgres_container_postgres_bind_port: ""
# password: some_password # password: some_password
matrix_postgres_additional_databases: [] matrix_postgres_additional_databases: []
# A list of roles/users to avoid creating when importing (or upgrading) the database.
# If a dump file contains the roles and they've also been created beforehand (see `matrix_postgres_additional_databases`),
# importing would fail.
# We either need to not create them or to ignore the `CREATE ROLE` statements in the dump.
matrix_postgres_import_roles_to_ignore: [matrix_postgres_connection_username]
matrix_postgres_import_roles_ignore_regex: "^CREATE ROLE ({{ matrix_postgres_import_roles_to_ignore|join('|') }});"
# A list of databases to avoid creating when importing (or upgrading) the database.
# If a dump file contains the databases and they've also been created beforehand (see `matrix_postgres_additional_databases`),
# importing would fail.
# We either need to not create them or to ignore the `CREATE DATABASE` statements in the dump.
matrix_postgres_import_databases_to_ignore: [matrix_postgres_db_name]
matrix_postgres_import_databases_ignore_regex: "^CREATE DATABASE ({{ matrix_postgres_import_databases_to_ignore|join('|') }})\\s"
# The number of seconds to wait after starting `matrix-postgres.service` # The number of seconds to wait after starting `matrix-postgres.service`
# and before trying to run queries for creating additional databases/users against it. # and before trying to run queries for creating additional databases/users against it.
# #

View file

@ -74,8 +74,8 @@
{{ matrix_postgres_docker_image_latest }} {{ matrix_postgres_docker_image_latest }}
-c "cat /{{ server_path_postgres_dump|basename }} | -c "cat /{{ server_path_postgres_dump|basename }} |
{{ 'gunzip |' if server_path_postgres_dump.endswith('.gz') else '' }} {{ 'gunzip |' if server_path_postgres_dump.endswith('.gz') else '' }}
grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' | grep -vE '{{ matrix_postgres_import_roles_ignore_regex }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' | grep -vE '{{ matrix_postgres_import_databases_ignore_regex }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres" psql -v ON_ERROR_STOP=1 -h matrix-postgres"
# This is a hack. # This is a hack.

View file

@ -135,8 +135,8 @@
{{ matrix_postgres_docker_image_latest }} {{ matrix_postgres_docker_image_latest }}
-c "cat /in/{{ postgres_dump_name }} | -c "cat /in/{{ postgres_dump_name }} |
{{ 'gunzip |' if postgres_dump_name.endswith('.gz') else '' }} {{ 'gunzip |' if postgres_dump_name.endswith('.gz') else '' }}
grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' | grep -vE '{{ matrix_postgres_import_roles_ignore_regex }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' | grep -vE '{{ matrix_postgres_import_databases_ignore_regex }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres" psql -v ON_ERROR_STOP=1 -h matrix-postgres"
# This is a hack. # This is a hack.