pub-solar/matrix-docker-ansible-deploy
5
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Replace custom/matrix-postgres-backup role with galaxy/com.devture.ansible.role.postgres_backup

Browse source 
This role is usable on its own and it's not tied to Matrix, so
extracting it out into an independent role that we install via
ansible-galaxy makes sense.

This also fixes the confusion from the other day, where
`matrix_postgres_*` had to be renamed to `devture_postgres_*`
(unless it was about `matrix_postgres_backup_*`).
We now can safely say that ALL `matrix_postgres_*` variables need to be
renamed.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2305
This commit is contained in:
Slavi Pantaleev 2022-11-30 10:59:25 +02:00
parent a365e54f4d
commit 4eed49f931
13 changed files with 69 additions and 259 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
CHANGELOG.md
View file

@ -1,3 +1,12 @@
# 2022-11-30
## matrix-postgres-backup has been replaced by the com.devture.ansible.role.postgres_backup external role
Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup).
You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `devture_postgres_backup` prefix.
# 2022-11-28
## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role

17
docs/configuring-playbook-postgres-backup.md
View file

@ -1,6 +1,6 @@
# Setting up postgres backup (optional)
The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you.
The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you via the [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup) Ansible role.
For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead.
@ -10,7 +10,7 @@ For a more complete backup solution (one that includes not only Postgres, but al
Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
```yaml
matrix_postgres_backup_enabled: true
devture_postgres_backup_enabled: true
```
Refer to the table below for additional configuration variables and their default values.
@ -18,12 +18,13 @@ Refer to the table below for additional configuration variables and their defaul
| Name | Default value | Description |
| :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups|
|`devture_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`devture_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`devture_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`devture_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`devture_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`devture_postgres_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `devture_postgres_data_path` |
|`devture_postgres_data_path` | `"{{ devture_postgres_base_path }}/data"` | Storage path for postgres-backup database backups |
## Installing

59
group_vars/matrix_servers
View file

@ -289,7 +289,7 @@ devture_systemd_service_manager_services_list_auto: |
+
([{'name': (devture_postgres_identifier + '.service'), 'priority': 500, 'groups': ['matrix', 'postgres']}] if devture_postgres_enabled else [])
+
([{'name': 'matrix-postgres-backup.service', 'priority': 3000, 'groups': ['matrix', 'backup', 'postgres-backup']}] if matrix_postgres_backup_enabled else [])
([{'name': (devture_postgres_backup_identifier + '.service'), 'priority': 3000, 'groups': ['matrix', 'backup', 'postgres-backup']}] if devture_postgres_backup_enabled else [])
+
([{'name': 'matrix-prometheus.service', 'priority': 4000, 'groups': ['matrix', 'monitoring', 'prometheus', 'prometheus-core']}] if matrix_prometheus_enabled else [])
+
@ -2241,6 +2241,7 @@ matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matr
#
######################################################################
########################################################################
# #
# com.devture.ansible.role.postgres #
@ -2504,6 +2505,42 @@ devture_postgres_managed_databases_auto: |
# #
########################################################################
########################################################################
# #
# com.devture.ansible.role.postgres_backup #
# #
########################################################################
devture_postgres_backup_enabled: false
devture_postgres_backup_identifier: matrix-postgres-backup
devture_postgres_backup_architecture: "{{ matrix_architecture }}"
devture_postgres_backup_base_path: "{{ matrix_base_data_path }}/postgres-backup"
devture_postgres_backup_container_network: "{{ matrix_docker_network }}"
devture_postgres_backup_uid: "{{ matrix_user_uid }}"
devture_postgres_backup_gid: "{{ matrix_user_gid }}"
devture_postgres_backup_connection_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}"
devture_postgres_backup_connection_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}"
devture_postgres_backup_connection_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}"
devture_postgres_backup_connection_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}"
devture_postgres_backup_postgres_data_path: "{{ devture_postgres_data_path if devture_postgres_enabled else '' }}"
devture_postgres_backup_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}"
########################################################################
# #
# /com.devture.ansible.role.postgres_backup #
# #
########################################################################
######################################################################
#
# matrix-sygnal
@ -2965,26 +3002,6 @@ matrix_registration_database_password: "{{ '%s' | format(matrix_homeserver_gener
#
######################################################################
######################################################################
#
# matrix-postgres-backup
#
######################################################################
matrix_postgres_backup_connection_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}"
matrix_postgres_backup_connection_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}"
matrix_postgres_backup_connection_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}"
matrix_postgres_backup_connection_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}"
matrix_postgres_backup_postgres_data_path: "{{ devture_postgres_data_path if devture_postgres_enabled else '' }}"
matrix_postgres_backup_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}"
######################################################################
#
# /matrix-postgres-backup
#
######################################################################
######################################################################
#

5
playbooks/matrix.yml
View file

@ -97,8 +97,11 @@
- custom/matrix-nginx-proxy
- custom/matrix-coturn
- custom/matrix-aux
- custom/matrix-postgres-backup
- role: galaxy/com.devture.ansible.role.postgres_backup
- custom/matrix-backup-borg
- custom/matrix-user-creator
- custom/matrix-common-after

3
requirements.yml
View file

@ -21,6 +21,9 @@
- src: git+https://github.com/devture/com.devture.ansible.role.postgres.git
version: 381feb7f671c60e8875298102a7c9b41242979ee
- src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git
version: 77b1f9ae1aafa31c9078178c1036bf744c99d08b
- src: git+https://github.com/devture/com.devture.ansible.role.systemd_service_manager.git
version: 6ccb88ac5fc27e1e70afcd48278ade4b564a9096

45
roles/custom/matrix-postgres-backup/defaults/main.yml
View file

@ -1,45 +0,0 @@
---
# Project source code URL: https://github.com/prodrigestivill/docker-postgres-backup-local
matrix_postgres_backup_enabled: false
matrix_postgres_backup_connection_hostname: ''
matrix_postgres_backup_connection_port: 5432
matrix_postgres_backup_connection_username: "matrix"
matrix_postgres_backup_connection_password: ""
matrix_postgres_backup_extra_opts: "-Z9 --schema=public --blobs"
matrix_postgres_backup_schedule: "@daily"
matrix_postgres_backup_keep_days: 7
matrix_postgres_backup_keep_weeks: 4
matrix_postgres_backup_keep_months: 12
matrix_postgres_backup_healthcheck_port: "8080"
matrix_postgres_backup_databases: []
matrix_postgres_backup_path: "{{ matrix_base_data_path }}/postgres-backup"
# Specifies where the Postgres data is.
# We use this to autodetect the Postgres version during playbook runtime (by parsing the `PG_VERSION` file contained there).
# You can leave this empty to prevent auto-detection.
matrix_postgres_backup_postgres_data_path: ""
matrix_postgres_backup_architecture: amd64
# matrix_postgres_backup_docker_image_distro controls whether we use Alpine-based images (`-alpine`) or the normal Debian-based images.
# Alpine-based Postgres images are smaller and we usually prefer them, but they don't work on ARM32 (tested on a Raspberry Pi 3 running Raspbian 10.7).
# On ARM32, `-alpine` images fail with the following error:
# > LOG: startup process (PID 37) was terminated by signal 11: Segmentation fault
matrix_postgres_backup_docker_image_distro: "{{ 'alpine' if matrix_postgres_backup_architecture in ['amd64', 'arm64'] else 'debian' }}"
matrix_postgres_backup_docker_image_v9: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:9.6-{{ matrix_postgres_backup_docker_image_distro }}-2aa03d1"
matrix_postgres_backup_docker_image_v10: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:10-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v11: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:11-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v12: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:12-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v13: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:13-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v14: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:14-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v15: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:15-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_latest: "{{ matrix_postgres_backup_docker_image_v15 }}"
# This variable is assigned at runtime. Overriding its value has no effect.
matrix_postgres_backup_docker_image_to_use: '{{ matrix_postgres_backup_docker_image_latest }}'
matrix_postgres_backup_docker_image_force_pull: "{{ matrix_postgres_backup_docker_image_to_use.endswith(':latest') }}"

20
roles/custom/matrix-postgres-backup/tasks/main.yml
View file

@ -1,20 +0,0 @@
---
- block:
- when: matrix_postgres_backup_enabled | bool
ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_config.yml"
- when: matrix_postgres_backup_enabled | bool
ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_install.yml"
tags:
- setup-all
- setup-postgres-backup
- install-all
- install-postgres-backup
- block:
- when: not matrix_postgres_backup_enabled | bool
ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
tags:
- setup-all
- setup-postgres-backup

51
roles/custom/matrix-postgres-backup/tasks/setup_install.yml
View file

@ -1,51 +0,0 @@
---
- ansible.builtin.include_role:
name: galaxy/com.devture.ansible.role.postgres
tasks_from: detect_existing_postgres_version
when: 'matrix_postgres_backup_postgres_data_path != ""'
# If we have found an existing version (installed from before), we use its corresponding Docker image.
# If not, we install using the latest Postgres.
#
# Upgrading is supposed to be performed separately and explicitly (see `upgrade_postgres.yml`).
- ansible.builtin.set_fact:
matrix_postgres_backup_docker_image_to_use: "{{ matrix_postgres_backup_docker_image_latest if matrix_postgres_backup_detected_version_corresponding_docker_image | default('') == '' else matrix_postgres_backup_detected_version_corresponding_docker_image }}"
- name: Ensure postgres backup Docker image is pulled
community.docker.docker_image:
name: "{{ matrix_postgres_backup_docker_image_to_use }}"
source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
force_source: "{{ matrix_postgres_backup_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_postgres_backup_docker_image_force_pull }}"
register: result
retries: "{{ devture_playbook_help_container_retries_count }}"
delay: "{{ devture_playbook_help_container_retries_delay }}"
until: result is not failed
- name: Ensure Postgres backup paths exist
ansible.builtin.file:
path: "{{ item }}"
state: directory
mode: 0700
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
with_items:
- "{{ matrix_postgres_backup_path }}"
- name: Ensure Postgres environment variables file created
ansible.builtin.template:
src: "{{ role_path }}/templates/{{ item }}.j2"
dest: "{{ matrix_postgres_backup_path }}/{{ item }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
mode: 0640
with_items:
- "env-postgres-backup"
- name: Ensure matrix-postgres-backup.service installed
ansible.builtin.template:
src: "{{ role_path }}/templates/systemd/matrix-postgres-backup.service.j2"
dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
mode: 0644

39
roles/custom/matrix-postgres-backup/tasks/setup_uninstall.yml
View file

@ -1,39 +0,0 @@
---
- name: Check existence of matrix-postgres-backup service
ansible.builtin.stat:
path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
register: matrix_postgres_backup_service_stat
- when: matrix_postgres_backup_service_stat.stat.exists | bool
block:
- name: Ensure matrix-postgres-backup is stopped
ansible.builtin.service:
name: matrix-postgres-backup
state: stopped
enabled: false
daemon_reload: true
- name: Ensure matrix-postgres-backup.service doesn't exist
ansible.builtin.file:
path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
state: absent
- name: Check existence of matrix-postgres-backup backup path
ansible.builtin.stat:
path: "{{ matrix_postgres_backup_path }}"
register: matrix_postgres_backup_path_stat
# We just want to notify the user. Deleting data is too destructive.
- name: Inject warning if matrix-postgres backup data remains
ansible.builtin.set_fact:
devture_playbook_runtime_messages_list: |
{{
devture_playbook_runtime_messages_list | default([])
+
[
"NOTE: You are not using the local backup service to backup the PostgreSQL database, but some old data remains from before in `{{ matrix_postgres_backup_path }}`. Feel free to delete it."
]
}}
when: matrix_postgres_backup_path_stat.stat.exists | bool

27
roles/custom/matrix-postgres-backup/tasks/validate_config.yml
View file

@ -1,27 +0,0 @@
---
- name: Fail if required postgres-backup settings not defined
ansible.builtin.fail:
msg: >-
You need to define a required configuration setting (`{{ item }}`).
when: "vars[item] == ''"
with_items:
- "matrix_postgres_backup_connection_hostname"
- "matrix_postgres_backup_connection_username"
- "matrix_postgres_backup_connection_password"
- "matrix_postgres_backup_connection_port"
- "matrix_postgres_backup_schedule"
- "matrix_postgres_backup_keep_days"
- "matrix_postgres_backup_keep_weeks"
- "matrix_postgres_backup_keep_months"
- "matrix_postgres_backup_path"
- "matrix_postgres_backup_databases"
- name: (Deprecation) Catch and report renamed settings
ansible.builtin.fail:
msg: >-
Your configuration contains a variable, which now has a different name.
Please change your configuration to rename the variable (`{{ item.old }}` -> `{{ item.new }}`).
when: "item.old in vars"
with_items:
- {'old': 'matrix_postgres_backup_docker_image_suffix', 'new': 'matrix_postgres_backup_docker_image_distro'}

12
roles/custom/matrix-postgres-backup/templates/env-postgres-backup.j2
View file

@ -1,12 +0,0 @@
#jinja2: lstrip_blocks: "True"
POSTGRES_USER={{ matrix_postgres_backup_connection_username }}
POSTGRES_PASSWORD={{ matrix_postgres_backup_connection_password }}
POSTGRES_HOST={{ matrix_postgres_backup_connection_hostname }}
POSTGRES_DB={{ matrix_postgres_backup_databases|join(', ') }}
POSTGRES_EXTRA_OPTS={{ matrix_postgres_backup_extra_opts }}
SCHEDULE={{ matrix_postgres_backup_schedule }}
BACKUP_KEEP_DAYS={{ matrix_postgres_backup_keep_days }}
BACKUP_KEEP_WEEKS={{ matrix_postgres_backup_keep_weeks }}
BACKUP_KEEP_MONTHS={{ matrix_postgres_backup_keep_months }}
HEALTHCHECK_PORT={{ matrix_postgres_backup_healthcheck_port }}
POSTGRES_PORT={{ matrix_postgres_backup_connection_port }}

31
roles/custom/matrix-postgres-backup/templates/systemd/matrix-postgres-backup.service.j2
View file

@ -1,31 +0,0 @@
#jinja2: lstrip_blocks: "True"
[Unit]
Description=Automatic Backup of Matrix Postgres server
After=docker.service
Requires=docker.service
DefaultDependencies=no
[Service]
Type=simple
Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} stop matrix-postgres-backup
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-postgres-backup 2>/dev/null || true'
ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-postgres-backup \
--log-driver=none \
--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
--cap-drop=ALL \
--read-only \
--network={{ matrix_docker_network }} \
--env-file={{ matrix_postgres_backup_path }}/env-postgres-backup \
--mount type=bind,src={{ matrix_postgres_backup_path }},dst=/backups \
{{ matrix_postgres_backup_docker_image_to_use }}
ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} stop matrix-postgres-backup
ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-postgres-backup 2>/dev/null || true'
Restart=always
RestartSec=30
SyslogIdentifier=matrix-postgres-backup
[Install]
WantedBy=multi-user.target

10
roles/custom/matrix_playbook_migration/tasks/validate_config.yml
View file

@ -41,8 +41,10 @@
The matrix-postgres role in the playbook has been replaced with the com.devture.ansible.role.postgres role (https://github.com/devture/com.devture.ansible.role.postgres).
The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all matrix-postgres variables (`matrix_postgres_*` -> `devture_postgres_*`).
Note that `matrix_postgres_backup_*` variables (used by the `matrix-postgres-backup` role) need to remain as they are for now. Do not rename those!
The matrix-postgres-backup role in the playbook has been replaced with the com.devture.ansible.role.postgres_backup role (https://github.com/devture/com.devture.ansible.role.postgres_backup).
The new role is pretty much the same, but uses differently named variables.
The following variables in your configuration need to be renamed: {{ vars | dict2items | rejectattr('key', 'match', 'matrix_postgres_backup_') | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | rejectattr('key', 'match', 'matrix_postgres_backup_') | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"
Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `devture_postgres_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"