Reorganize Postgres access (breaking change)

In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
This commit is contained in:
Slavi Pantaleev 2021-01-22 12:23:00 +02:00
parent 703f1b1a04
commit 95346f3117
9 changed files with 139 additions and 21 deletions

View file

@ -1,3 +1,85 @@
# 2021-01-22
## (Breaking Change) Postgres changes that require manual intervention
We've made a lot of changes to our Postgres setup and some manual action is required (described below). Sorry about the hassle.
**TLDR**: people running an [external Postgres server](docs/configuring-playbook-external-postgres.md) don't need to change anything for now. Everyone else (the common/default case) is affected and manual intervention is required.
### Why?
- we had a default Postgres password (`matrix_postgres_connection_password: synapse-password`), which we think is **not ideal for security anymore**. We now ask you to generate/provide a strong password yourself. Postgres is normally not exposed outside the container network, making it relatively secure, but still:
- by tweaking the configuration, you may end up intentionally or unintentionally exposing your Postgres server to the local network (or even publicly), while still using the default default credentials (`synapse` + `synapse-password`)
- we can't be sure we trust all these services (bridges, etc). Some of them may try to talk to or attack `matrix-postgres` using the default credentials (`synapse` + `synapse-password`)
- you may have other containers running on the same Docker network, which may try to talk to or attack `matrix-postgres` using the default credentials (`synapse` + `synapse-password`)
- our Postgres usage **was overly-focused on Synapse** (default username of `synapse` and default/main database of `homeserver`). Additional homeserver options are likely coming in the future ([Dendrite](https://matrix.org/docs/projects/server/dendrite), [Conduit](https://matrix.org/docs/projects/server/conduit), [The Construct](https://matrix.org/docs/projects/server/construct)), so being too focused on `matrix-synapse` is not great. From now on, Synapse is just another component of this playbook, which happens to have an *additional database* (called `synapse`) on the Postgres server.
- we try to reorganize things a bit, to make the playbook even friendlier to people running an [external Postgres server](docs/configuring-playbook-external-postgres.md). Work on this will proceed in the future.
So, this is some **effort to improve security** and to **prepare for a brighter future of having more homeserver options** than just Synapse.
### What has really changed?
- the default superuser Postgres username is now `matrix` (used to be `synapse`)
- the default Postgres database is now `matrix` (used to be `homeserver`)
- Synapse's database is now `synapse` (used to be `homeserver`). This is now just another "additional database" that the playbook manages for you
- Synapse's user called `synapse` is just a regular user that can only use the `synapse` database (not a superuser anymore)
### What do I do if I'm using the integrated Postgres server (default)?
By default, the playbook runs an integrated Postgres server for you in a container (`matrix-postgres`). Unless you've explicitly configured an [external Postgres server](docs/configuring-playbook-external-postgres.md), these steps are meant for you.
To migrate to the new setup, expect a few minutes of downtime, while you follow these steps:
1. Generate a strong password to be used for your superuser Postgres user (called `matrix`). You can use `pwgen -s 64 1` to generate it, or some other tool.
2. Update your playbook's `inventory/host_vars/matrix.DOMAIN/vars.yml` file, adding a line like this:
```yaml
matrix_postgres_connection_password: YOUR_POSTGRES_PASSWORD_HERE
```
.. where `YOUR_POSTGRES_PASSWORD_HERE` is to be replaced with the password you generated during step #1.
3. Stop all services: `ansible-playbook -i inventory/hosts setup.yml --tags=stop`
4. Log in to the server via SSH. The next commands will be performed there.
5. Start the Postgres database server: `systemctl start matrix-postgres`
6. Open a Postgres shell: `/usr/local/bin/matrix-postgres-cli`
7. Execute the following query, while making sure to **change the password inside**:
```sql
CREATE ROLE matrix LOGIN SUPERUSER PASSWORD 'YOUR_POSTGRES_PASSWORD_HERE';
```
.. where `YOUR_POSTGRES_PASSWORD_HERE` is to be replaced with the password you generated during step #1.
8. Execute the following queries as you see them (no modifications necessary, so you can just paste them):
```sql
CREATE DATABASE matrix OWNER matrix;
ALTER DATABASE postgres OWNER TO matrix;
ALTER DATABASE template0 OWNER TO matrix;
ALTER DATABASE template1 OWNER TO matrix;
\c matrix;
ALTER DATABASE homeserver RENAME TO synapse;
ALTER ROLE synapse NOSUPERUSER NOCREATEDB NOCREATEROLE;
\quit
```
You may need to press *Enter* after pasting the lines above.
1. Re-run the playbook normally: `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`
### What do I do if I'm using an external Postgres server?
If you've explicitly configured an [external Postgres server](docs/configuring-playbook-external-postgres.md), there are changes that you need to do at this time.
The fact that we've renamed Synapse's database from `homeserver` to `synapse` (in our defaults) should not affect you, as you're already explicitly defining `matrix_synapse_database_database` (if you've followed our guide, that is). If you're not explicitly defining this variable, you may wish to do so (`matrix_synapse_database_database: homeserver`), to avoid the new `synapse` default and keep things as they were.
# 2021-01-20 # 2021-01-20
## (Breaking Change) The mautrix-facebook bridge now requires a Postgres database ## (Breaking Change) The mautrix-facebook bridge now requires a Postgres database

View file

@ -4,6 +4,7 @@ By default, this playbook would set up a PostgreSQL database server on your mach
If that's alright, you can skip this. If that's alright, you can skip this.
If you'd like to use an external PostgreSQL server that you manage, you can edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`). If you'd like to use an external PostgreSQL server that you manage, you can edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`).
It should be something like this: It should be something like this:
```yaml ```yaml

View file

@ -19,6 +19,17 @@ You can use the `/usr/local/bin/matrix-postgres-cli` tool to get interactive ter
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available. If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
By default, this tool puts you in the `matrix` database, which contains nothing.
To see the available databases, run `\list` (or just `\l`).
To change to another database (for example `synapse`), run `\connect synapse` (or just `\c synapse`).
You can then proceed to write queries. Example: `SELECT COUNT(*) FROM users;`
**Be careful**. Modifying the database directly (especially as services are running) is dangerous and may lead to irreversible database corruption.
When in doubt, consider [making a backup](#backing-up-postgresql).
## Vacuuming PostgreSQL ## Vacuuming PostgreSQL

View file

@ -4,7 +4,7 @@
# Note: this playbook does not touch the server referenced here. # Note: this playbook does not touch the server referenced here.
# Installation happens on another server ("matrix.<matrix-domain>"). # Installation happens on another server ("matrix.<matrix-domain>").
# #
# If you've deployed using the wrong domain, you'll have to run the Uninstalling step, # If you've deployed using the wrong domain, you'll have to run the Uninstalling step,
# because you can't change the Domain after deployment. # because you can't change the Domain after deployment.
# #
# Example value: example.com # Example value: example.com
@ -18,12 +18,18 @@ matrix_domain: YOUR_BARE_DOMAIN_NAME_HERE
# you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`). # you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`).
# #
# Example value: someone@example.com # Example value: someone@example.com
matrix_ssl_lets_encrypt_support_email: YOUR_EMAIL_ADDRESS_HERE matrix_ssl_lets_encrypt_support_email: ''
# A shared secret (between Coturn and Synapse) used for authentication. # A shared secret (between Coturn and Synapse) used for authentication.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`). # You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
matrix_coturn_turn_static_auth_secret: "" matrix_coturn_turn_static_auth_secret: ''
# A secret used to protect access keys issued by the server. # A secret used to protect access keys issued by the server.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`). # You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
matrix_synapse_macaroon_secret_key: "" matrix_synapse_macaroon_secret_key: ''
# A Postgres password to used for the superuser Postgres user (called `matrix` by default).
#
# The playbook creates additional Postgres users and databases (one for each enabled service)
# using this superuser account.
matrix_postgres_connection_password: ''

View file

@ -1021,16 +1021,20 @@ matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matr
matrix_postgres_enabled: true matrix_postgres_enabled: true
matrix_postgres_connection_hostname: "matrix-postgres" # We unset this if internal Postgres disabled, which will cascade to some other variables
matrix_postgres_connection_username: "synapse" # and tell users they need to set it (either here or in those variables).
# Please note that the max length of the password is 99 characters matrix_postgres_connection_hostname: "{{ 'matrix-postgres' if matrix_postgres_enabled else '' }}"
matrix_postgres_connection_password: "synapse-password"
matrix_postgres_db_name: "homeserver"
matrix_postgres_pgloader_container_image_self_build: "{{ matrix_architecture != 'amd64' }}" matrix_postgres_pgloader_container_image_self_build: "{{ matrix_architecture != 'amd64' }}"
matrix_postgres_additional_databases: | matrix_postgres_additional_databases: |
{{ {{
([{
'name': matrix_synapse_database_database,
'username': matrix_synapse_database_user,
'password': matrix_synapse_database_password,
}] if (matrix_synapse_enabled and matrix_synapse_database_database != matrix_postgres_db_name and matrix_synapse_database_host == 'matrix-postgres') else [])
+
([{ ([{
'name': matrix_ma1sd_database_name, 'name': matrix_ma1sd_database_name,
'username': matrix_ma1sd_database_username, 'username': matrix_ma1sd_database_username,
@ -1243,10 +1247,7 @@ matrix_synapse_container_metrics_api_host_bind_port: "{{ '127.0.0.1:9100' if (ma
# For exposing the Synapse Manhole port (plain HTTP) to the local host. # For exposing the Synapse Manhole port (plain HTTP) to the local host.
matrix_synapse_container_manhole_api_host_bind_port: "{{ '127.0.0.1:9000' if matrix_synapse_manhole_enabled else '' }}" matrix_synapse_container_manhole_api_host_bind_port: "{{ '127.0.0.1:9000' if matrix_synapse_manhole_enabled else '' }}"
matrix_synapse_database_host: "{{ matrix_postgres_connection_hostname }}" matrix_synapse_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'synapse.db') | to_uuid }}"
matrix_synapse_database_user: "{{ matrix_postgres_connection_username }}"
matrix_synapse_database_password: "{{ matrix_postgres_connection_password }}"
matrix_synapse_database_database: "{{ matrix_postgres_db_name }}"
# We do not enable TLS in Synapse by default. # We do not enable TLS in Synapse by default.
# TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse. # TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse.

View file

@ -1,9 +1,10 @@
matrix_postgres_enabled: true matrix_postgres_enabled: true
matrix_postgres_connection_hostname: "" matrix_postgres_connection_hostname: "matrix-postgres"
matrix_postgres_connection_username: "" matrix_postgres_connection_port: 5432
matrix_postgres_connection_username: "matrix"
matrix_postgres_connection_password: "" matrix_postgres_connection_password: ""
matrix_postgres_db_name: "" matrix_postgres_db_name: "matrix"
matrix_postgres_base_path: "{{ matrix_base_data_path }}/postgres" matrix_postgres_base_path: "{{ matrix_base_data_path }}/postgres"
matrix_postgres_data_path: "{{ matrix_postgres_base_path }}/data" matrix_postgres_data_path: "{{ matrix_postgres_base_path }}/data"

View file

@ -6,17 +6,29 @@
The `matrix_postgres_use_external` variable defined in your configuration is not used by this playbook anymore! The `matrix_postgres_use_external` variable defined in your configuration is not used by this playbook anymore!
You'll need to adapt to the new way of using an external Postgres server. You'll need to adapt to the new way of using an external Postgres server.
It's a combination of `matrix_postgres_enabled: false` and specifying Postgres connection It's a combination of `matrix_postgres_enabled: false` and specifying Postgres connection
details in a few `matrix_synapse_database_` variables. details in a few `matrix_postgres_connection_` variables.
See the "Using an external PostgreSQL server (optional)" documentation page. See the "Using an external PostgreSQL server (optional)" documentation page.
when: "'matrix_postgres_use_external' in vars" when: "'matrix_postgres_use_external' in vars"
# This is separate (from the other required variables below),
# because we'd like to have a friendlier message for our existing users.
- name: Fail if matrix_postgres_connection_password not defined
fail:
msg: >-
The playbook no longer has a default Postgres password defined in the `matrix_postgres_connection_password` variable, among lots of other Postgres changes.
You need to perform multiple manual steps to resolve this.
See our changelog for more details:
https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#breaking-change-postgres-changes-that-require-manual-intervention
when: "matrix_postgres_connection_password == ''"
- name: Fail if required Postgres settings not defined - name: Fail if required Postgres settings not defined
fail: fail:
msg: > msg: >-
You need to define a required configuration setting (`{{ item }}`). You need to define a required configuration setting (`{{ item }}`).
when: "vars[item] == ''" when: "vars[item] == ''"
with_items: with_items:
- "matrix_postgres_connection_hostname" - "matrix_postgres_connection_hostname"
- "matrix_postgres_connection_port"
- "matrix_postgres_connection_username" - "matrix_postgres_connection_username"
- "matrix_postgres_connection_password" - "matrix_postgres_connection_password"
- "matrix_postgres_db_name" - "matrix_postgres_db_name"

View file

@ -294,10 +294,10 @@ matrix_synapse_manhole_enabled: false
matrix_synapse_sentry_dsn: "" matrix_synapse_sentry_dsn: ""
# Postgres database information # Postgres database information
matrix_synapse_database_host: "" matrix_synapse_database_host: "matrix-postgres"
matrix_synapse_database_user: "" matrix_synapse_database_user: "synapse"
matrix_synapse_database_password: "" matrix_synapse_database_password: ""
matrix_synapse_database_database: "" matrix_synapse_database_database: "synapse"
matrix_synapse_turn_uris: [] matrix_synapse_turn_uris: []
matrix_synapse_turn_shared_secret: "" matrix_synapse_turn_shared_secret: ""

View file

@ -7,6 +7,10 @@
when: "vars[item] == ''" when: "vars[item] == ''"
with_items: with_items:
- "matrix_synapse_macaroon_secret_key" - "matrix_synapse_macaroon_secret_key"
- "matrix_synapse_database_host"
- "matrix_synapse_database_user"
- "matrix_synapse_database_password"
- "matrix_synapse_database_database"
- name: (Deprecation) Catch and report renamed settings - name: (Deprecation) Catch and report renamed settings
fail: fail: