Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen, preparing for a future where it's just one-of-many homeserver software options. We also no longer have a default Postgres superuser password, which improves security. The changelog explains more as to why this was done and how to proceed from here.
This commit is contained in:
parent
703f1b1a04
commit
95346f3117
82
CHANGELOG.md
82
CHANGELOG.md
|
@ -1,3 +1,85 @@
|
||||||
|
# 2021-01-22
|
||||||
|
|
||||||
|
## (Breaking Change) Postgres changes that require manual intervention
|
||||||
|
|
||||||
|
We've made a lot of changes to our Postgres setup and some manual action is required (described below). Sorry about the hassle.
|
||||||
|
|
||||||
|
**TLDR**: people running an [external Postgres server](docs/configuring-playbook-external-postgres.md) don't need to change anything for now. Everyone else (the common/default case) is affected and manual intervention is required.
|
||||||
|
|
||||||
|
### Why?
|
||||||
|
|
||||||
|
- we had a default Postgres password (`matrix_postgres_connection_password: synapse-password`), which we think is **not ideal for security anymore**. We now ask you to generate/provide a strong password yourself. Postgres is normally not exposed outside the container network, making it relatively secure, but still:
|
||||||
|
- by tweaking the configuration, you may end up intentionally or unintentionally exposing your Postgres server to the local network (or even publicly), while still using the default default credentials (`synapse` + `synapse-password`)
|
||||||
|
- we can't be sure we trust all these services (bridges, etc). Some of them may try to talk to or attack `matrix-postgres` using the default credentials (`synapse` + `synapse-password`)
|
||||||
|
- you may have other containers running on the same Docker network, which may try to talk to or attack `matrix-postgres` using the default credentials (`synapse` + `synapse-password`)
|
||||||
|
- our Postgres usage **was overly-focused on Synapse** (default username of `synapse` and default/main database of `homeserver`). Additional homeserver options are likely coming in the future ([Dendrite](https://matrix.org/docs/projects/server/dendrite), [Conduit](https://matrix.org/docs/projects/server/conduit), [The Construct](https://matrix.org/docs/projects/server/construct)), so being too focused on `matrix-synapse` is not great. From now on, Synapse is just another component of this playbook, which happens to have an *additional database* (called `synapse`) on the Postgres server.
|
||||||
|
- we try to reorganize things a bit, to make the playbook even friendlier to people running an [external Postgres server](docs/configuring-playbook-external-postgres.md). Work on this will proceed in the future.
|
||||||
|
|
||||||
|
So, this is some **effort to improve security** and to **prepare for a brighter future of having more homeserver options** than just Synapse.
|
||||||
|
|
||||||
|
### What has really changed?
|
||||||
|
|
||||||
|
- the default superuser Postgres username is now `matrix` (used to be `synapse`)
|
||||||
|
- the default Postgres database is now `matrix` (used to be `homeserver`)
|
||||||
|
- Synapse's database is now `synapse` (used to be `homeserver`). This is now just another "additional database" that the playbook manages for you
|
||||||
|
- Synapse's user called `synapse` is just a regular user that can only use the `synapse` database (not a superuser anymore)
|
||||||
|
|
||||||
|
### What do I do if I'm using the integrated Postgres server (default)?
|
||||||
|
|
||||||
|
By default, the playbook runs an integrated Postgres server for you in a container (`matrix-postgres`). Unless you've explicitly configured an [external Postgres server](docs/configuring-playbook-external-postgres.md), these steps are meant for you.
|
||||||
|
|
||||||
|
To migrate to the new setup, expect a few minutes of downtime, while you follow these steps:
|
||||||
|
|
||||||
|
1. Generate a strong password to be used for your superuser Postgres user (called `matrix`). You can use `pwgen -s 64 1` to generate it, or some other tool.
|
||||||
|
|
||||||
|
2. Update your playbook's `inventory/host_vars/matrix.DOMAIN/vars.yml` file, adding a line like this:
|
||||||
|
```yaml
|
||||||
|
matrix_postgres_connection_password: YOUR_POSTGRES_PASSWORD_HERE
|
||||||
|
```
|
||||||
|
|
||||||
|
.. where `YOUR_POSTGRES_PASSWORD_HERE` is to be replaced with the password you generated during step #1.
|
||||||
|
|
||||||
|
3. Stop all services: `ansible-playbook -i inventory/hosts setup.yml --tags=stop`
|
||||||
|
4. Log in to the server via SSH. The next commands will be performed there.
|
||||||
|
5. Start the Postgres database server: `systemctl start matrix-postgres`
|
||||||
|
6. Open a Postgres shell: `/usr/local/bin/matrix-postgres-cli`
|
||||||
|
7. Execute the following query, while making sure to **change the password inside**:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE ROLE matrix LOGIN SUPERUSER PASSWORD 'YOUR_POSTGRES_PASSWORD_HERE';
|
||||||
|
```
|
||||||
|
|
||||||
|
.. where `YOUR_POSTGRES_PASSWORD_HERE` is to be replaced with the password you generated during step #1.
|
||||||
|
|
||||||
|
8. Execute the following queries as you see them (no modifications necessary, so you can just paste them):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE DATABASE matrix OWNER matrix;
|
||||||
|
|
||||||
|
ALTER DATABASE postgres OWNER TO matrix;
|
||||||
|
ALTER DATABASE template0 OWNER TO matrix;
|
||||||
|
ALTER DATABASE template1 OWNER TO matrix;
|
||||||
|
|
||||||
|
\c matrix;
|
||||||
|
|
||||||
|
ALTER DATABASE homeserver RENAME TO synapse;
|
||||||
|
|
||||||
|
ALTER ROLE synapse NOSUPERUSER NOCREATEDB NOCREATEROLE;
|
||||||
|
|
||||||
|
\quit
|
||||||
|
```
|
||||||
|
|
||||||
|
You may need to press *Enter* after pasting the lines above.
|
||||||
|
|
||||||
|
1. Re-run the playbook normally: `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`
|
||||||
|
|
||||||
|
### What do I do if I'm using an external Postgres server?
|
||||||
|
|
||||||
|
If you've explicitly configured an [external Postgres server](docs/configuring-playbook-external-postgres.md), there are changes that you need to do at this time.
|
||||||
|
|
||||||
|
The fact that we've renamed Synapse's database from `homeserver` to `synapse` (in our defaults) should not affect you, as you're already explicitly defining `matrix_synapse_database_database` (if you've followed our guide, that is). If you're not explicitly defining this variable, you may wish to do so (`matrix_synapse_database_database: homeserver`), to avoid the new `synapse` default and keep things as they were.
|
||||||
|
|
||||||
|
|
||||||
# 2021-01-20
|
# 2021-01-20
|
||||||
|
|
||||||
## (Breaking Change) The mautrix-facebook bridge now requires a Postgres database
|
## (Breaking Change) The mautrix-facebook bridge now requires a Postgres database
|
||||||
|
|
|
@ -4,6 +4,7 @@ By default, this playbook would set up a PostgreSQL database server on your mach
|
||||||
If that's alright, you can skip this.
|
If that's alright, you can skip this.
|
||||||
|
|
||||||
If you'd like to use an external PostgreSQL server that you manage, you can edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`).
|
If you'd like to use an external PostgreSQL server that you manage, you can edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`).
|
||||||
|
|
||||||
It should be something like this:
|
It should be something like this:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
|
|
@ -19,6 +19,17 @@ You can use the `/usr/local/bin/matrix-postgres-cli` tool to get interactive ter
|
||||||
|
|
||||||
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
|
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
|
||||||
|
|
||||||
|
By default, this tool puts you in the `matrix` database, which contains nothing.
|
||||||
|
|
||||||
|
To see the available databases, run `\list` (or just `\l`).
|
||||||
|
|
||||||
|
To change to another database (for example `synapse`), run `\connect synapse` (or just `\c synapse`).
|
||||||
|
|
||||||
|
You can then proceed to write queries. Example: `SELECT COUNT(*) FROM users;`
|
||||||
|
|
||||||
|
**Be careful**. Modifying the database directly (especially as services are running) is dangerous and may lead to irreversible database corruption.
|
||||||
|
When in doubt, consider [making a backup](#backing-up-postgresql).
|
||||||
|
|
||||||
|
|
||||||
## Vacuuming PostgreSQL
|
## Vacuuming PostgreSQL
|
||||||
|
|
||||||
|
|
|
@ -4,7 +4,7 @@
|
||||||
# Note: this playbook does not touch the server referenced here.
|
# Note: this playbook does not touch the server referenced here.
|
||||||
# Installation happens on another server ("matrix.<matrix-domain>").
|
# Installation happens on another server ("matrix.<matrix-domain>").
|
||||||
#
|
#
|
||||||
# If you've deployed using the wrong domain, you'll have to run the Uninstalling step,
|
# If you've deployed using the wrong domain, you'll have to run the Uninstalling step,
|
||||||
# because you can't change the Domain after deployment.
|
# because you can't change the Domain after deployment.
|
||||||
#
|
#
|
||||||
# Example value: example.com
|
# Example value: example.com
|
||||||
|
@ -18,12 +18,18 @@ matrix_domain: YOUR_BARE_DOMAIN_NAME_HERE
|
||||||
# you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`).
|
# you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`).
|
||||||
#
|
#
|
||||||
# Example value: someone@example.com
|
# Example value: someone@example.com
|
||||||
matrix_ssl_lets_encrypt_support_email: YOUR_EMAIL_ADDRESS_HERE
|
matrix_ssl_lets_encrypt_support_email: ''
|
||||||
|
|
||||||
# A shared secret (between Coturn and Synapse) used for authentication.
|
# A shared secret (between Coturn and Synapse) used for authentication.
|
||||||
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
|
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
|
||||||
matrix_coturn_turn_static_auth_secret: ""
|
matrix_coturn_turn_static_auth_secret: ''
|
||||||
|
|
||||||
# A secret used to protect access keys issued by the server.
|
# A secret used to protect access keys issued by the server.
|
||||||
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
|
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
|
||||||
matrix_synapse_macaroon_secret_key: ""
|
matrix_synapse_macaroon_secret_key: ''
|
||||||
|
|
||||||
|
# A Postgres password to used for the superuser Postgres user (called `matrix` by default).
|
||||||
|
#
|
||||||
|
# The playbook creates additional Postgres users and databases (one for each enabled service)
|
||||||
|
# using this superuser account.
|
||||||
|
matrix_postgres_connection_password: ''
|
||||||
|
|
|
@ -1021,16 +1021,20 @@ matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matr
|
||||||
|
|
||||||
matrix_postgres_enabled: true
|
matrix_postgres_enabled: true
|
||||||
|
|
||||||
matrix_postgres_connection_hostname: "matrix-postgres"
|
# We unset this if internal Postgres disabled, which will cascade to some other variables
|
||||||
matrix_postgres_connection_username: "synapse"
|
# and tell users they need to set it (either here or in those variables).
|
||||||
# Please note that the max length of the password is 99 characters
|
matrix_postgres_connection_hostname: "{{ 'matrix-postgres' if matrix_postgres_enabled else '' }}"
|
||||||
matrix_postgres_connection_password: "synapse-password"
|
|
||||||
matrix_postgres_db_name: "homeserver"
|
|
||||||
|
|
||||||
matrix_postgres_pgloader_container_image_self_build: "{{ matrix_architecture != 'amd64' }}"
|
matrix_postgres_pgloader_container_image_self_build: "{{ matrix_architecture != 'amd64' }}"
|
||||||
|
|
||||||
matrix_postgres_additional_databases: |
|
matrix_postgres_additional_databases: |
|
||||||
{{
|
{{
|
||||||
|
([{
|
||||||
|
'name': matrix_synapse_database_database,
|
||||||
|
'username': matrix_synapse_database_user,
|
||||||
|
'password': matrix_synapse_database_password,
|
||||||
|
}] if (matrix_synapse_enabled and matrix_synapse_database_database != matrix_postgres_db_name and matrix_synapse_database_host == 'matrix-postgres') else [])
|
||||||
|
+
|
||||||
([{
|
([{
|
||||||
'name': matrix_ma1sd_database_name,
|
'name': matrix_ma1sd_database_name,
|
||||||
'username': matrix_ma1sd_database_username,
|
'username': matrix_ma1sd_database_username,
|
||||||
|
@ -1243,10 +1247,7 @@ matrix_synapse_container_metrics_api_host_bind_port: "{{ '127.0.0.1:9100' if (ma
|
||||||
# For exposing the Synapse Manhole port (plain HTTP) to the local host.
|
# For exposing the Synapse Manhole port (plain HTTP) to the local host.
|
||||||
matrix_synapse_container_manhole_api_host_bind_port: "{{ '127.0.0.1:9000' if matrix_synapse_manhole_enabled else '' }}"
|
matrix_synapse_container_manhole_api_host_bind_port: "{{ '127.0.0.1:9000' if matrix_synapse_manhole_enabled else '' }}"
|
||||||
|
|
||||||
matrix_synapse_database_host: "{{ matrix_postgres_connection_hostname }}"
|
matrix_synapse_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'synapse.db') | to_uuid }}"
|
||||||
matrix_synapse_database_user: "{{ matrix_postgres_connection_username }}"
|
|
||||||
matrix_synapse_database_password: "{{ matrix_postgres_connection_password }}"
|
|
||||||
matrix_synapse_database_database: "{{ matrix_postgres_db_name }}"
|
|
||||||
|
|
||||||
# We do not enable TLS in Synapse by default.
|
# We do not enable TLS in Synapse by default.
|
||||||
# TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse.
|
# TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse.
|
||||||
|
|
|
@ -1,9 +1,10 @@
|
||||||
matrix_postgres_enabled: true
|
matrix_postgres_enabled: true
|
||||||
|
|
||||||
matrix_postgres_connection_hostname: ""
|
matrix_postgres_connection_hostname: "matrix-postgres"
|
||||||
matrix_postgres_connection_username: ""
|
matrix_postgres_connection_port: 5432
|
||||||
|
matrix_postgres_connection_username: "matrix"
|
||||||
matrix_postgres_connection_password: ""
|
matrix_postgres_connection_password: ""
|
||||||
matrix_postgres_db_name: ""
|
matrix_postgres_db_name: "matrix"
|
||||||
|
|
||||||
matrix_postgres_base_path: "{{ matrix_base_data_path }}/postgres"
|
matrix_postgres_base_path: "{{ matrix_base_data_path }}/postgres"
|
||||||
matrix_postgres_data_path: "{{ matrix_postgres_base_path }}/data"
|
matrix_postgres_data_path: "{{ matrix_postgres_base_path }}/data"
|
||||||
|
|
|
@ -6,17 +6,29 @@
|
||||||
The `matrix_postgres_use_external` variable defined in your configuration is not used by this playbook anymore!
|
The `matrix_postgres_use_external` variable defined in your configuration is not used by this playbook anymore!
|
||||||
You'll need to adapt to the new way of using an external Postgres server.
|
You'll need to adapt to the new way of using an external Postgres server.
|
||||||
It's a combination of `matrix_postgres_enabled: false` and specifying Postgres connection
|
It's a combination of `matrix_postgres_enabled: false` and specifying Postgres connection
|
||||||
details in a few `matrix_synapse_database_` variables.
|
details in a few `matrix_postgres_connection_` variables.
|
||||||
See the "Using an external PostgreSQL server (optional)" documentation page.
|
See the "Using an external PostgreSQL server (optional)" documentation page.
|
||||||
when: "'matrix_postgres_use_external' in vars"
|
when: "'matrix_postgres_use_external' in vars"
|
||||||
|
|
||||||
|
# This is separate (from the other required variables below),
|
||||||
|
# because we'd like to have a friendlier message for our existing users.
|
||||||
|
- name: Fail if matrix_postgres_connection_password not defined
|
||||||
|
fail:
|
||||||
|
msg: >-
|
||||||
|
The playbook no longer has a default Postgres password defined in the `matrix_postgres_connection_password` variable, among lots of other Postgres changes.
|
||||||
|
You need to perform multiple manual steps to resolve this.
|
||||||
|
See our changelog for more details:
|
||||||
|
https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#breaking-change-postgres-changes-that-require-manual-intervention
|
||||||
|
when: "matrix_postgres_connection_password == ''"
|
||||||
|
|
||||||
- name: Fail if required Postgres settings not defined
|
- name: Fail if required Postgres settings not defined
|
||||||
fail:
|
fail:
|
||||||
msg: >
|
msg: >-
|
||||||
You need to define a required configuration setting (`{{ item }}`).
|
You need to define a required configuration setting (`{{ item }}`).
|
||||||
when: "vars[item] == ''"
|
when: "vars[item] == ''"
|
||||||
with_items:
|
with_items:
|
||||||
- "matrix_postgres_connection_hostname"
|
- "matrix_postgres_connection_hostname"
|
||||||
|
- "matrix_postgres_connection_port"
|
||||||
- "matrix_postgres_connection_username"
|
- "matrix_postgres_connection_username"
|
||||||
- "matrix_postgres_connection_password"
|
- "matrix_postgres_connection_password"
|
||||||
- "matrix_postgres_db_name"
|
- "matrix_postgres_db_name"
|
||||||
|
|
|
@ -294,10 +294,10 @@ matrix_synapse_manhole_enabled: false
|
||||||
matrix_synapse_sentry_dsn: ""
|
matrix_synapse_sentry_dsn: ""
|
||||||
|
|
||||||
# Postgres database information
|
# Postgres database information
|
||||||
matrix_synapse_database_host: ""
|
matrix_synapse_database_host: "matrix-postgres"
|
||||||
matrix_synapse_database_user: ""
|
matrix_synapse_database_user: "synapse"
|
||||||
matrix_synapse_database_password: ""
|
matrix_synapse_database_password: ""
|
||||||
matrix_synapse_database_database: ""
|
matrix_synapse_database_database: "synapse"
|
||||||
|
|
||||||
matrix_synapse_turn_uris: []
|
matrix_synapse_turn_uris: []
|
||||||
matrix_synapse_turn_shared_secret: ""
|
matrix_synapse_turn_shared_secret: ""
|
||||||
|
|
|
@ -7,6 +7,10 @@
|
||||||
when: "vars[item] == ''"
|
when: "vars[item] == ''"
|
||||||
with_items:
|
with_items:
|
||||||
- "matrix_synapse_macaroon_secret_key"
|
- "matrix_synapse_macaroon_secret_key"
|
||||||
|
- "matrix_synapse_database_host"
|
||||||
|
- "matrix_synapse_database_user"
|
||||||
|
- "matrix_synapse_database_password"
|
||||||
|
- "matrix_synapse_database_database"
|
||||||
|
|
||||||
- name: (Deprecation) Catch and report renamed settings
|
- name: (Deprecation) Catch and report renamed settings
|
||||||
fail:
|
fail:
|
||||||
|
|
Loading…
Reference in a new issue