Merge branch 'master' into pub.solar

2021-04-07 23:10:22 +02:00 · 2021-04-07 23:10:22 +02:00 · 9f45a11f84
parent 1e7c5abbf3 4830b7d830
commit 9f45a11f84
56 changed files with 1337 additions and 208 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,3 +1,21 @@
+# 2021-04-05
+
+## Automated local Postgres backup support
+
+Thanks to [foxcris](https://github.com/foxcris), the playbook can now make automated local Postgres backups on a fixed schedule using [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local).
+
+Additional details are available in [Setting up postgres backup](docs/configuring-playbook-postgres-backup.md).
+
+
+
+# 2021-04-03
+
+## Mjolnir moderation tool (bot) support
+
+Thanks to [Aaron Raimist](https://github.com/aaronraimist), the playbook can now install and configure the [Mjolnir](https://github.com/matrix-org/mjolnir) moderation tool (bot).
+
+Additional details are available in [Setting up Mjolnir](docs/configuring-playbook-bot-mjolnir.md).
+

 # 2021-03-20

--- a/README.md
+++ b/README.md
@ -93,6 +93,8 @@ Using this playbook, you can get the following services configured on your serve

 - (optional) [Go-NEB](https://github.com/matrix-org/go-neb) multi functional bot written in Go - see [docs/configuring-playbook-bot-go-neb.md](docs/configuring-playbook-bot-go-neb.md) for setup documentation

+- (optional) [Mjolnir](https://github.com/matrix-org/mjolnir), a moderation tool for Matrix - see [docs/configuring-playbook-bot-mjolnir.md](docs/configuring-playbook-bot-mjolnir.md) for setup documentation
+
 - (optional) [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin), a web UI tool for administrating users and rooms on your Matrix server - see [docs/configuring-playbook-synapse-admin.md](docs/configuring-playbook-synapse-admin.md) for setup documentation

 - (optional) [matrix-registration](https://github.com/ZerataX/matrix-registration), a simple python application to have a token based matrix registration - see [docs/configuring-playbook-matrix-registration.md](docs/configuring-playbook-matrix-registration.md) for setup documentation
--- a/docs/configuring-awx-system.md
+++ b/docs/configuring-awx-system.md
@ -19,7 +19,7 @@ The following repositories allow you to copy and use this setup:

 [Create AWX System](https://gitlab.com/GoMatrixHosting/create-awx-system) - Creates and configures the AWX system for you.

-[Ansible Create Delete Subscription Membership](https://gitlab.com/GoMatrixHosting/ansible-create-delete-subscription-membership) - Used by the AWX system to create memberships and subscriptions.
+[Ansible Create Delete Subscription Membership](https://gitlab.com/GoMatrixHosting/ansible-create-delete-subscription-membership) - Used by the AWX system to create memberships and subscriptions. Also includes other administrative playbooks for updates, backups and restoring servers.

 [Ansible Provision Server](https://gitlab.com/GoMatrixHosting/ansible-provision-server) - Used by AWX members to perform initial configuration of their DigitalOcean or On-Premises server.

--- a/docs/configuring-playbook-bot-mjolnir.md
+++ b/docs/configuring-playbook-bot-mjolnir.md
@ -0,0 +1,121 @@
+# Setting up Mjolnir (optional)
+
+The playbook can install and configure the [Mjolnir](https://github.com/matrix-org/mjolnir) moderation bot for you.
+
+See the project's [documentation](https://github.com/matrix-org/mjolnir) to learn what it does and why it might be useful to you.
+
+Note: the playbook does not currently support the Mjolnir Synapse module. The playbook does support another antispam module, see [Setting up Synapse Simple Antispam](configuring-playbook-synapse-simple-antispam.md).
+
+
+## 1. Register the bot account
+
+The playbook does not automatically create users for you. The bot requires an access token to be able to connect to your homeserver.
+
+You **need to register the bot user manually** before setting up the bot.
+
+Choose a strong password for the bot. You can generate a good password with a command like this: `pwgen -s 64 1`.
+
+You can use the playbook to [register a new user](registering-users.md):
+
+```
+ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=bot.mjolnir password=PASSWORD_FOR_THE_BOT admin=no' --tags=register-user
+```
+
+If you would like Mjolnir to be able to deactivate users, move aliases, shutdown rooms, etc then it must be a server admin so you need to change `admin=no` to `admin=yes` in the command above.
+
+
+## 2. Get an access token
+
+If you use curl, you can get an access token like this:
+
+```
+curl -X POST --header 'Content-Type: application/json' -d '{
+    "identifier": { "type": "m.id.user", "user": "bot.mjolnir" },
+    "password": "PASSWORD_FOR_THE_BOT",
+    "type": "m.login.password"
+}' 'https://matrix.DOMAIN/_matrix/client/r0/login'
+```
+
+Alternatively, you can use a full-featured client (such as Element) to log in and get the access token from there (note: don't log out from the client as that will invalidate the token).
+
+
+## 3. Make sure the account is free from rate limiting
+
+You will need to prevent Synapse from rate limiting the bot's account. This is not an optional step. If you do not do this step Mjolnir will crash. [Currently there is no Synapse config option for this](https://github.com/matrix-org/synapse/issues/6286) so you have to manually edit the Synapse database. Manually editing the Synapse database is rarely a good idea but in this case it is required. Please ask for help if you are uncomfortable with these steps.
+
+1. Copy the statement below into a text editor. 
+
+	```
+	INSERT INTO ratelimit_override VALUES ("@bot.mjolnir:DOMAIN", 0, 0);
+	```
+
+1. Change the username (`@bot.mjolnir:DOMAIN`) to the username you used when you registered the bot's account. You must change `DOMAIN` to your server's domain.
+
+1. Get a database terminal by following these steps: [maintenance-postgres.md#getting-a-database-terminal](maintenance-postgres.md#getting-a-database-terminal)
+
+1. Connect to Synapse's database by typing `\connect synapse` into the database terminal
+
+1. Paste in the `INSERT INTO` command that you edited and press enter.
+
+You can run `SELECT * FROM ratelimit_override;` to see if it worked. If the output looks like this:
+
+```
+      user_id          | messages_per_second | burst_count
+-----------------------+---------------------+-------------
+ @bot.mjolnir:raim.ist |                   0 |           0`
+```
+then you did it correctly.
+
+
+## 4. Create a management room
+
+Using your own account, create a new invite only room that you will use to manage the bot. This is the room where you will see the status of the bot and where you will send commands to the bot, such as the command to ban a user from another room. Anyone in this room can control the bot so it is important that you only invite trusted users to this room. The room must be unencrypted since the playbook does not support installing Pantalaimon yet.
+
+Once you have created the room you need to copy the room ID so you can tell the bot to use that room. In Element you can do this by going to the room's settings, clicking Advanced, and then coping the internal room ID. The room ID will look something like `!QvgVuKq0ha8glOLGMG:DOMAIN`.
+
+Finally invite the `@bot.mjolnir:DOMAIN` account you created earlier into the room.
+
+
+## 5. Adjusting the playbook configuration
+
+Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file (adapt to your needs):
+
+You must replace `ACCESS_TOKEN_FROM_STEP_2_GOES_HERE` and `ROOM_ID_FROM_STEP_4_GOES_HERE` with the your own values.
+
+```yaml
+matrix_bot_mjolnir_enabled: true
+
+matrix_bot_mjolnir_access_token: "ACCESS_TOKEN_FROM_STEP_2_GOES_HERE"
+
+matrix_bot_mjolnir_management_room: "ROOM_ID_FROM_STEP_4_GOES_HERE"
+```
+
+
+## 6. Installing
+
+After configuring the playbook, run the [installation](installing.md) command:
+
+```
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+```
+
+
+## Usage
+
+You can refer to the upstream [documentation](https://github.com/matrix-org/mjolnir) for additional ways to use and configure mjolnir. Check out their [quickstart guide](https://github.com/matrix-org/mjolnir#quickstart-guide) for some basic commands you can give to the bot.
+
+You can configure additional options by adding the `matrix_bot_mjolnir_configuration_extension_yaml` variable to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file.
+
+For example to change mjolnir's `recordIgnoredInvites` option to `true` you would add the following to your `vars.yml` file.
+
+```yaml
+matrix_bot_mjolnir_configuration_extension_yaml: |
+  # Your custom YAML configuration goes here.
+  # This configuration extends the default starting configuration (`matrix_bot_mjolnir_configuration_yaml`).
+  #
+  # You can override individual variables from the default configuration, or introduce new ones.
+  #
+  # If you need something more special, you can take full control by
+  # completely redefining `matrix_bot_mjolnir_configuration_yaml`.
+  recordIgnoredInvites: true
+```
--- a/docs/configuring-playbook-postgres-backup.md
+++ b/docs/configuring-playbook-postgres-backup.md
@ -0,0 +1,32 @@
+# Setting up postgres backup (optional)
+
+The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you.
+
+## Adjusting the playbook configuration
+
+Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
+
+```yaml
+matrix_postgres_backup_enabled: true
+```
+
+Refer to the table below for additional configuration variables and their default values.
+
+
+| Name                              | Default value                | Description                                                      |
+| :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
+|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
+|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
+|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
+|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
+|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
+|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups|
+
+
+## Installing
+
+After configuring the playbook, run the [installation](installing.md) command again:
+
+```
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+```
--- a/docs/configuring-playbook-prometheus-grafana.md
+++ b/docs/configuring-playbook-prometheus-grafana.md
@ -17,11 +17,10 @@ matrix_grafana_anonymous_access: false

 # This has no relation to your Matrix user id. It can be any username you'd like.
 # Changing the username subsequently won't work.
-matrix_grafana_default_admin_user: some_username_chosen_by_you
+matrix_grafana_default_admin_user: "some_username_chosen_by_you"

-# Passwords containing special characters may be troublesome.
 # Changing the password subsequently won't work.
-matrix_grafana_default_admin_password: some_strong_password_chosen_by_you
+matrix_grafana_default_admin_password: "some_strong_password_chosen_by_you"
 ```

 By default, a [Grafana](https://grafana.com/) web user-interface will be available at `https://stats.<your-domain>`.
--- a/docs/configuring-playbook.md
+++ b/docs/configuring-playbook.md
@ -129,9 +129,11 @@ When you're done with all the configuration you'd like to do, continue with [Ins

 ### Bots

- [Setting up matrix-reminder-bot](configuring-playbook-bot-matrix-reminder-bot.md) (optional)
+- [Setting up matrix-reminder-bot](configuring-playbook-bot-matrix-reminder-bot.md) - a bot to remind you about stuff (optional)

- [Setting up Go-NEB](configuring-playbook-bot-go-neb.md) (optional)
+- [Setting up Go-NEB](configuring-playbook-bot-go-neb.md) - an extensible multifunctional bot (optional)
+
+- [Setting up Mjolnir](configuring-playbook-bot-mjolnir.md) - a moderation tool/bot (optional)


 ### Other specialized services
--- a/docs/container-images.md
+++ b/docs/container-images.md
@ -90,6 +90,8 @@ These services are not part of our default installation, but can be enabled by [

 - [matrixdotorg/go-neb](https://hub.docker.com/r/matrixdotorg/go-neb) - the [Go-NEB](https://github.com/matrix-org/go-neb) bot (optional)

+- [matrixdotorg/mjolnir](https://hub.docker.com/r/matrixdotorg/mjolnir) - the [mjolnir](https://github.com/matrix-org/mjolnir) moderation bot (optional)
+
 - [awesometechnologies/synapse-admin](https://hub.docker.com/r/awesometechnologies/synapse-admin) - the [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin) web UI tool for administrating users and rooms on your Matrix server (optional)

 - [prom/prometheus](https://hub.docker.com/r/prom/prometheus/) - [Prometheus](https://github.com/prometheus/prometheus/) is a systems and service monitoring system
--- a/docs/maintenance-postgres.md
+++ b/docs/maintenance-postgres.md
@ -12,6 +12,7 @@ Table of contents:

 - [Upgrading PostgreSQL](#upgrading-postgresql), for upgrading to new major versions of PostgreSQL. Such **manual upgrades are sometimes required**.

+- [Tuning PostgreSQL](#tuning-postgresql) to make it run faster

 ## Getting a database terminal

@ -48,7 +49,9 @@ ansible-playbook -i inventory/hosts setup.yml --tags=run-postgres-vacuum,start

 ## Backing up PostgreSQL

-To make a back up of the current PostgreSQL database, make sure it's running and then execute a command like this on the server:
+To automatically make Postgres database backups on a fixed schedule, see [Setting up postgres backup](configuring-playbook-postgres-backup.md).
+
+To make a one off back up of the current PostgreSQL database, make sure it's running and then execute a command like this on the server:

 ```bash
 /usr/bin/docker exec \
@ -90,3 +93,43 @@ If you have plenty of space in `/tmp` and would rather avoid gzipping, you can e
 Example: `--extra-vars="postgres_dump_name=matrix-postgres-dump.sql"`

 **All databases, roles, etc. on the Postgres server are migrated**.
+
+
+## Tuning PostgreSQL
+
+PostgreSQL can be tuned to make it run faster. This is done by passing extra arguments to Postgres with the `matrix_postgres_process_extra_arguments` variable. You should use a website like https://pgtune.leopard.in.ua/ or information from https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server to determine what Postgres settings you should change.
+
+### Here are some examples:
+
+These are not recommended values and they may not work well for you. This is just to give you an idea of some of the options that can be set. If you are an experienced PostgreSQL admin feel free to update this documentation with better examples.
+
+Here is an example config for a small 2 core server with 4GB of RAM and SSD storage:
+```
+matrix_postgres_process_extra_arguments: [
+  "-c 'shared_buffers=128MB'",
+  "-c 'effective_cache_size=2304MB'",
+  "-c 'effective_io_concurrency=100'",
+  "-c 'random_page_cost=2.0'",
+  "-c 'min_wal_size=500MB'",
+]
+```
+
+Here is an example config for a large 6 core server with 24GB of RAM:
+```
+matrix_postgres_process_extra_arguments: [
+  "-c max_connections=40",
+  "-c shared_buffers=1536MB",
+  "-c checkpoint_completion_target=0.7",
+  "-c wal_buffers=16MB",
+  "-c default_statistics_target=100",
+  "-c random_page_cost=1.1",
+  "-c effective_io_concurrency=100",
+  "-c work_mem=2621kB",
+  "-c min_wal_size=1GB",
+  "-c max_wal_size=4GB",
+  "-c max_worker_processes=6",
+  "-c max_parallel_workers_per_gather=3",
+  "-c max_parallel_workers=6",
+  "-c max_parallel_maintenance_workers=3",
+]
+```
--- a/docs/maintenance-synapse.md
+++ b/docs/maintenance-synapse.md
@ -6,13 +6,11 @@ Table of contents:

 - [Purging old data with the Purge History API](#purging-old-data-with-the-purge-history-api), for when you wish to delete in-use (but old) data from the Synapse database

- [Synapse maintenance](#synapse-maintenance)
-	- [Purging old data with the Purge History API](#purging-old-data-with-the-purge-history-api)
 - [Compressing state with rust-synapse-compress-state](#compressing-state-with-rust-synapse-compress-state)
-	- [Browse and manipulate the database](#browse-and-manipulate-the-database)

 - [Browse and manipulate the database](#browse-and-manipulate-the-database), for when you really need to take matters into your own hands

+- [Make Synapse faster](#make-synapse-faster)

 ## Purging old data with the Purge History API

@ -73,3 +71,13 @@ docker run --rm --publish 1799:8080 --link matrix-postgres --net matrix adminer
 You should then be able to browse the adminer database administration GUI at http://localhost:1799/ after entering your DB credentials (found in the `host_vars` or on the server in `{{matrix_synapse_config_dir_path}}/homeserver.yaml` under `database.args`)

 ⚠️ Be **very careful** with this, there is **no undo** for impromptu DB operations.
+
+## Make Synapse faster
+
+Synapse's presence feature which tracks which users are online and which are offline can use a lot of processing power. You can disable presence by adding `matrix_synapse_use_presence: false` to your `vars.yml` file.
+
+Tuning Synapse's cache factor can help reduce RAM usage. [See the upstream documentation](https://github.com/matrix-org/synapse#help-synapse-is-slow-and-eats-all-my-ram-cpu) for more information on what value to set the cache factor to. Use the variable `matrix_synapse_caches_global_factor` to set the cache factor.
+
+Tuning your PostgreSQL database will also make Synapse run significantly faster. See [maintenance-postgres.md##tuning-postgresql](maintenance-postgres.md##tuning-postgresql).
+
+See also [How do I optimize this setup for a low-power server?](faq.md#how-do-i-optimize-this-setup-for-a-low-power-server).
--- a/group_vars/matrix_servers
+++ b/group_vars/matrix_servers
@ -815,6 +815,33 @@ matrix_bot_go_neb_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_en
 ######################################################################


+######################################################################
+#
+# matrix-bot-mjolnir
+#
+######################################################################
+
+# We don't enable bots by default.
+matrix_bot_mjolnir_enabled: false
+
+matrix_bot_mjolnir_systemd_required_services_list: |
+  {{
+    ['docker.service']
+    +
+    (['matrix-synapse.service'] if matrix_synapse_enabled else [])
+    +
+    (['matrix-postgres.service'] if matrix_postgres_enabled else [])
+    +
+    (['matrix-nginx-proxy.service'] if matrix_nginx_proxy_enabled else [])
+  }}
+
+######################################################################
+#
+# /matrix-bot-mjolnir
+#
+######################################################################
+
+
 ######################################################################
 #
 # matrix-corporal
@ -1650,7 +1677,7 @@ matrix_prometheus_node_exporter_enabled: false
 # Normally, matrix-nginx-proxy is enabled and nginx can reach Prometheus Node Exporter over the container network.
 # If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose
 # Prometheus' HTTP port to the local host.
-matrix_prometheus_node_exporter_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9100' }}"
+matrix_prometheus_node_exporter_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9200' }}"

 ######################################################################
 #
@ -1747,3 +1774,32 @@ matrix_registration_database_password: "{{ matrix_synapse_macaroon_secret_key |
 # /matrix-registration
 #
 ######################################################################
+
+######################################################################
+#
+# matrix-postgres-backup
+#
+######################################################################
+
+matrix_postgres_backup_connection_hostname: "{{ matrix_postgres_connection_hostname }}"
+matrix_postgres_backup_connection_port: "{{ matrix_postgres_connection_port }}"
+matrix_postgres_backup_connection_username: "{{ matrix_postgres_connection_username }}"
+matrix_postgres_backup_connection_password: "{{ matrix_postgres_connection_password }}"
+
+matrix_postgres_backup_postgres_data_path: "{{ matrix_postgres_data_path if matrix_postgres_enabled else '' }}"
+
+# the default matrix synapse databse is not always part of the matrix_postgres_additional_databases variable thus we have to add it if the default database is used
+matrix_postgres_backup_databases: |
+  {{
+    (([{
+        'name': matrix_synapse_database_database
+    }] if (matrix_synapse_enabled and matrix_synapse_database_database == matrix_postgres_db_name and matrix_synapse_database_host == 'matrix-postgres') else [])
+    +
+    matrix_postgres_additional_databases)|map(attribute='name')|list
+  }}
+
+######################################################################
+#
+# /matrix-postgres-backup
+#
+######################################################################
--- a/roles/matrix-awx/surveys/backup_server.json.j2
+++ b/roles/matrix-awx/surveys/backup_server.json.j2
@ -5,7 +5,7 @@
    {
      "question_name": "Enable Backup",
      "question_description": "Set if remote backup is enabled or not. If enabled a daily backup of your server will be sent to the backup server located in {{ backup_server_location }}.",
-      "required": true,
+      "required": false,
      "min": null,
      "max": null,
      "default": "{{ matrix_awx_backup_enabled | string | lower }}",
--- a/roles/matrix-awx/surveys/configure_dimension.json.j2
+++ b/roles/matrix-awx/surveys/configure_dimension.json.j2
@ -0,0 +1,30 @@
+{
+  "name": "Configure Dimension",
+  "description": "",
+  "spec": [
+    {
+      "question_name": "Enable Dimension",
+      "question_description": "Enables the Dimension integration server, before doing this you need to create a CNAME record for 'dimension.{{ matrix_domain }}' that points to 'matrix.{{ matrix_domain }}'.",
+      "required": false,
+      "min": null,
+      "max": null,
+      "default": "{{ matrix_dimension_enabled | string | lower }}",
+      "choices": "true\nfalse",
+      "new_question": true,
+      "variable": "matrix_dimension_enabled",
+      "type": "multiplechoice"
+    },
+    {
+      "question_name": "Dimension Users",
+      "question_description": "Here you can list the user accounts that will be able to configure Dimension. Entries must be seperated with newlines and must be a complete Matrix ID. For example: '@dimension:{{ matrix_domain }}'",
+      "required": false,
+      "min": 0,
+      "max": 65536,
+      "default": {{ ext_dimension_users_raw_final | to_json }},
+      "choices": "",
+      "new_question": true,
+      "variable": "ext_dimension_users_raw",
+      "type": "textarea"
+    }
+  ]
+}
--- a/roles/matrix-awx/tasks/backup_server.yml
+++ b/roles/matrix-awx/tasks/backup_server.yml
@ -5,27 +5,24 @@
    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# AWX Settings'
+    insertafter: '# AWX Settings Start'
  with_dict:
    'matrix_awx_backup_enabled': '{{ matrix_awx_backup_enabled }}'
-
- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
+  tags: use-survey

 - name: Save new 'Backup Server' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
  template:
    src: 'roles/matrix-awx/surveys/backup_server.json.j2'
    dest: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/backup_server.json'
+  tags: use-survey

 - name: Copy new 'Backup Server' survey.json to target machine
  copy:
    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/backup_server.json'
    dest: '/matrix/awx/backup_server.json'
    mode: '0660'
+  tags: use-survey

 - name: Collect AWX admin token the hard way!
  delegate_to: 127.0.0.1
@ -33,6 +30,7 @@
      curl -sku {{ tower_username }}:{{ tower_password }} -H "Content-Type: application/json" -X POST -d '{"description":"Tower CLI", "application":null, "scope":"write"}' https://{{ tower_host }}/api/v2/users/1/personal_tokens/ | jq '.token' | sed -r 's/\"//g'
  register: tower_token
  no_log: True
+  tags: use-survey

 - name: Recreate 'Backup Server' job template
  delegate_to: 127.0.0.1
@ -41,7 +39,7 @@
    description: "Performs a backup of the entire service to a remote location."
    extra_vars: "{{ lookup('file', '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/extra_vars.json') }}"
    job_type: run
-    job_tags: "backup-server"
+    job_tags: "backup-server,use-survey"
    inventory: "{{ member_id }}"
    project: "{{ member_id }} - Matrix Docker Ansible Deploy"
    playbook: setup.yml
@ -54,6 +52,23 @@
    tower_host: "https://{{ tower_host }}"
    tower_oauthtoken: "{{ tower_token.stdout }}"
    validate_certs: yes
+  tags: use-survey
+
+- name: Run export.sh if this job template is run by the client
+  command: /bin/sh /root/export.sh
+  tags: use-survey
+
+- name: Include vars in matrix_vars.yml
+  include_vars:
+    file: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+  no_log: True
+
+- name: Copy new 'matrix_vars.yml' to target machine
+  copy:
+    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    dest: '/matrix/awx/matrix_vars.yml'
+    mode: '0660'
+  tags: use-survey

 - name: Perform the borg backup
  command: borgmatic
--- a/roles/matrix-awx/tasks/cache_matrix_variables.yml
+++ b/roles/matrix-awx/tasks/cache_matrix_variables.yml
@ -0,0 +1,12 @@
+
+- name: Collect current datetime
+  set_fact:
+    awx_datetime: "{{ lookup('pipe', 'date +%Y-%m-%d_%H:%M') }}"
+
+- name: Create cached matrix_vars.yml file location
+  set_fact:
+    awx_cached_matrix_vars: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars_{{ awx_datetime }}.yml'
+
+- name: Create cached matrix_vars.yml
+  delegate_to: 127.0.0.1
+  shell: "cp /var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml {{ awx_cached_matrix_vars }}"
--- a/roles/matrix-awx/tasks/customise_website_access_export.yml
+++ b/roles/matrix-awx/tasks/customise_website_access_export.yml
@ -1,12 +1,11 @@

-
 - name: Enable index.html creation if user doesn't wish to customise base domain
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Base Domain Settings'
+    insertafter: '# Base Domain Settings Start'
  with_dict:
    'matrix_nginx_proxy_base_domain_homepage_enabled': 'true'
  when: customise_base_domain_website|bool == false
@ -14,10 +13,10 @@
 - name: Disable index.html creation to allow multi-file site if user does wish to customise base domain
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Base Domain Settings'
+    insertafter: '# Base Domain Settings Start'
  with_dict:
    'matrix_nginx_proxy_base_domain_homepage_enabled': 'false'
  when: customise_base_domain_website|bool == true
@ -25,27 +24,23 @@
 - name: Record custom 'Customise Website + Access Export' variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Custom Settings'
+    insertafter: '# Custom Settings Start'
  with_dict:
    'customise_base_domain_website': '{{ customise_base_domain_website }}'
    'sftp_auth_method': '"{{ sftp_auth_method }}"'
    'sftp_password': '"{{ sftp_password }}"'
    'sftp_public_key': '"{{ sftp_public_key }}"'

- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
-
 - name: Reload vars in matrix_vars.yml
  include_vars:
-    file: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    file: '{{ awx_cached_matrix_vars }}'
  no_log: True

+# ^ Is this even needed?
+
 - name: Save new 'Customise Website + Access Export' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
  template:
--- a/roles/matrix-awx/tasks/load_hosting_and_org_variables.yml
+++ b/roles/matrix-awx/tasks/load_hosting_and_org_variables.yml
@ -8,3 +8,4 @@
  include_vars:
    file: '/var/lib/awx/projects/hosting/hosting_vars.yml'
  no_log: True
+
--- a/roles/matrix-awx/tasks/load_matrix_variables.yml
+++ b/roles/matrix-awx/tasks/load_matrix_variables.yml
@ -1,5 +1,15 @@

- name: Include vars in matrix_vars.yml
+- name: Include new vars in matrix_vars.yml
  include_vars:
-    file: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    file: '{{ awx_cached_matrix_vars }}'
  no_log: True
+
+- name: If include_vars succeeds overwrite the old matrix_vars.yml 
+  delegate_to: 127.0.0.1
+  shell: "cp {{ awx_cached_matrix_vars }} /var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml && rm {{ awx_cached_matrix_vars }}"
+
+- name: Copy new 'matrix_vars.yml' to target machine
+  copy:
+    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    dest: '/matrix/awx/matrix_vars.yml'
+    mode: '0660'
--- a/roles/matrix-awx/tasks/main.yml
+++ b/roles/matrix-awx/tasks/main.yml
@ -26,6 +26,15 @@
  tags:
    - create-user

+# Import configs, media repo from /chroot/backup import
+- include_tasks: 
+    file: "import_awx.yml"
+    apply:
+      tags: import-awx
+  when: run_setup|bool and matrix_awx_enabled|bool
+  tags:
+    - import-awx
+    
 # Perform extra self-check functions
 - include_tasks: 
    file: "self_check.yml"
@ -35,14 +44,14 @@
  tags:
    - self-check

-# Import configs, media repo from /chroot/backup import
+# Create cached matrix_vars.yml file
 - include_tasks: 
-    file: "import_awx.yml"
+    file: "cache_matrix_variables.yml"
    apply:
-      tags: import-awx
+      tags: always
  when: run_setup|bool and matrix_awx_enabled|bool
  tags:
-    - import-awx
+    - always

 # Configure SFTP so user can upload a static website or access the servers export
 - include_tasks: 
@ -98,6 +107,15 @@
  tags:
    - setup-corporal

+# Additional playbook to set the variable file during Dimension configuration
+- include_tasks: 
+    file: "set_variables_dimension.yml"
+    apply:
+      tags: setup-dimension
+  when: run_setup|bool and matrix_awx_enabled|bool
+  tags:
+    - setup-dimension
+
 # Additional playbook to set the variable file during Synapse Admin configuration
 - include_tasks: 
    file: "set_variables_synapse_admin.yml"
@ -107,7 +125,7 @@
  tags:
    - setup-synapse-admin

-# Load newly formed matrix variables from tower volume
+# Load newly formed matrix variables from AWX volume
 - include_tasks: 
    file: "load_matrix_variables.yml"
    apply:
--- a/roles/matrix-awx/tasks/self_check.yml
+++ b/roles/matrix-awx/tasks/self_check.yml
@ -3,6 +3,7 @@
  apt:
    name:
      - sysstat
+      - curl
    state: present

 - name: Install prerequisite yum packages on AWX
@ -19,6 +20,17 @@
      - dnspython
    state: present

+- name: Calculate MAU value
+  shell: |
+    curl -s localhost:9000 | grep "^synapse_admin_mau_current "
+  register: mau_stat
+  no_log: True
+
+- name: Print MAU value
+  debug:
+    msg: "{{ mau_stat.stdout.split('\n') }}"
+  when: mau_stat is defined
+
 - name: Calculate CPU usage statistics
  shell: iostat -c
  register: cpu_usage_stat
--- a/roles/matrix-awx/tasks/set_variables_corporal.yml
+++ b/roles/matrix-awx/tasks/set_variables_corporal.yml
@ -2,20 +2,20 @@
 - name: Record Corporal Enabled/Disabled variable
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Corporal Settings'
+    insertafter: '# Corporal Settings Start'
  with_dict:
    'matrix_corporal_enabled': '{{ matrix_corporal_enabled }}'

 - name: Enable Shared Secret Auth if Corporal enabled
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Shared Secret Auth Settings'
+    insertafter: '# Shared Secret Auth Settings Start'
  with_dict:
    'matrix_synapse_ext_password_provider_shared_secret_auth_enabled': 'true'
  when: matrix_corporal_enabled|bool
@ -23,10 +23,10 @@
 - name: Disable Shared Secret Auth if Corporal disabled
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Shared Secret Auth Settings'
+    insertafter: '# Shared Secret Auth Settings Start'
  with_dict:
    'matrix_synapse_ext_password_provider_shared_secret_auth_enabled': 'false'
  when: not matrix_corporal_enabled|bool
@ -34,10 +34,10 @@
 - name: Enable Rest Auth Endpoint if Corporal enabled
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Extension'
+    insertafter: '# Synapse Extension Start'
  with_dict:
    'matrix_synapse_ext_password_provider_rest_auth_enabled': 'true'
  when: matrix_corporal_enabled|bool
@ -45,10 +45,10 @@
 - name: Disable Rest Auth Endpoint if Corporal disabled
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Extension'
+    insertafter: '# Synapse Extension Start'
  with_dict:
    'matrix_synapse_ext_password_provider_rest_auth_enabled': 'false'
  when: not matrix_corporal_enabled|bool
@ -56,10 +56,10 @@
 - name: Disable Corporal API if Simple Static File mode selected
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Corporal Settings'
+    insertafter: '# Corporal Settings Start'
  with_dict:
    'matrix_corporal_http_api_enabled': 'false'
  when: (matrix_corporal_policy_provider_mode == "Simple Static File") or (not matrix_corporal_enabled|bool)
@ -67,10 +67,10 @@
 - name: Enable Corporal API if Push/Pull mode delected
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Corporal Settings'
+    insertafter: '# Corporal Settings Start'
  with_dict:
    'matrix_corporal_http_api_enabled': 'true'
  when: (matrix_corporal_policy_provider_mode != "Simple Static File") and (matrix_corporal_enabled|bool)
@ -78,10 +78,10 @@
 - name: Record Corporal API Access Token if it's defined
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Corporal Settings'
+    insertafter: '# Corporal Settings Start'
  with_dict:
    'matrix_corporal_http_api_auth_token': '{{ matrix_corporal_http_api_auth_token }}'
  when: matrix_corporal_http_api_auth_token|length > 0
@ -89,8 +89,8 @@
 - name: Record 'Simple Static File' configuration variables in matrix_vars.yml
  delegate_to: 127.0.0.1
  blockinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    insertafter: "# Corporal Policy Provider Settings"
+    path: '{{ awx_cached_matrix_vars }}'
+    insertafter: "# Corporal Policy Provider Settings Start"
    block: |
      matrix_corporal_policy_provider_config: |
        {
@ -151,8 +151,8 @@
 - name: Record 'HTTP Pull Mode' configuration variables in matrix_vars.yml
  delegate_to: 127.0.0.1
  blockinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    insertafter: "# Corporal Policy Provider Settings"
+    path: '{{ awx_cached_matrix_vars }}'
+    insertafter: "# Corporal Policy Provider Settings Start"
    block: |
      matrix_corporal_policy_provider_config: |
        {
@ -168,8 +168,8 @@
 - name: Record 'HTTP Push Mode' configuration variables in matrix_vars.yml
  delegate_to: 127.0.0.1
  blockinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    insertafter: "# Corporal Policy Provider Settings"
+    path: '{{ awx_cached_matrix_vars }}'
+    insertafter: "# Corporal Policy Provider Settings Start"
    block: |
      matrix_corporal_policy_provider_config: |
        {
@ -181,7 +181,7 @@
 - name: Lower RateLimit if set to 'Normal'
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: '    address:\n        per_second: 50\n        burst_count: 300\n    account:\n        per_second: 0.17\n        burst_count: 300'
    replace: '    address:\n        per_second: 0.17\n        burst_count: 3\n    account:\n        per_second: 0.17\n        burst_count: 3'
  when: matrix_corporal_raise_ratelimits == "Normal"
@ -189,17 +189,11 @@
 - name: Raise RateLimit if set to 'Raised'
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: '    address:\n        per_second: 0.17\n        burst_count: 3\n    account:\n        per_second: 0.17\n        burst_count: 3'
    replace: '    address:\n        per_second: 50\n        burst_count: 300\n    account:\n        per_second: 0.17\n        burst_count: 300'
  when: matrix_corporal_raise_ratelimits == "Raised"

- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0664'
-
 - name: Save new 'Configure Corporal' survey.json to the AWX tower
  delegate_to: 127.0.0.1
  template:
--- a/roles/matrix-awx/tasks/set_variables_dimension.yml
+++ b/roles/matrix-awx/tasks/set_variables_dimension.yml
@ -0,0 +1,109 @@
+
+- name: Include vars in matrix_vars.yml
+  include_vars:
+    file: '{{ awx_cached_matrix_vars }}'
+  no_log: True
+
+- name: Install jq on remote machine
+  apt:
+    name: jq
+    state: present
+
+- name: Collect access token of Dimension user
+  shell: |
+    curl -X POST --header 'Content-Type: application/json' -d '{ "identifier": { "type": "m.id.user","user": "dimension" }, "password": "{{ matrix_awx_dimension_user_password }}", "type": "m.login.password"}' 'https://matrix.{{ matrix_domain }}/_matrix/client/r0/login' | jq -c '. | {access_token}' | sed 's/.*\":\"//' | sed 's/\"}//'
+  register: dimension_user_access_token
+
+- name: Record Synapse variables locally on AWX
+  delegate_to: 127.0.0.1
+  lineinfile:
+    path: '{{ awx_cached_matrix_vars }}'
+    regexp: "^#? *{{ item.key | regex_escape() }}:"
+    line: "{{ item.key }}: {{ item.value }}"
+    insertafter: '# Dimension Settings Start'
+  with_dict:
+    'matrix_dimension_enabled': '{{ matrix_dimension_enabled }}'
+    'matrix_dimension_access_token': '"{{ dimension_user_access_token.stdout }}"'
+  
+- name: Set final users list if users are defined
+  set_fact:
+    ext_dimension_users_raw_final: "{{ ext_dimension_users_raw }}"
+  when: ext_dimension_users_raw|length > 0
+
+- name: Set final users list if no users are defined
+  set_fact:
+    ext_dimension_users_raw_final: '@dimension:{{ matrix_domain }}'
+  when: ext_dimension_users_raw|length == 0
+
+- name: Remove Dimension Users
+  delegate_to: 127.0.0.1
+  replace:
+    path: '{{ awx_cached_matrix_vars }}'
+    regexp: '^  - .*\n'
+    after: 'matrix_dimension_admins:'
+    before: '# Dimension Settings End'
+
+- name: Set Dimension Users Header
+  delegate_to: 127.0.0.1
+  lineinfile:
+    path: '{{ awx_cached_matrix_vars }}'
+    insertbefore: '# Dimension Settings End'
+    line: "matrix_dimension_admins:"
+
+- name: Set Dimension Users
+  delegate_to: 127.0.0.1
+  lineinfile:
+    path: '{{ awx_cached_matrix_vars }}'
+    insertafter: '^matrix_dimension_admins:'
+    line: '  - "{{ item }}"'
+  with_items: "{{ ext_dimension_users_raw_final.splitlines() }}"
+
+- name: Record Dimension Custom variables locally on AWX
+  delegate_to: 127.0.0.1
+  lineinfile:
+    path: '{{ awx_cached_matrix_vars }}'
+    regexp: "^#? *{{ item.key | regex_escape() }}:"
+    line: "{{ item.key }}: {{ item.value }}"
+    insertafter: '# Custom Settings Start'
+  with_dict:
+    'ext_dimension_users_raw': '{{ ext_dimension_users_raw.splitlines() | to_json }}'
+
+- name: Save new 'Configure Dimension' survey.json to the AWX tower, template
+  delegate_to: 127.0.0.1
+  template:
+    src: 'roles/matrix-awx/surveys/configure_dimension.json.j2'
+    dest: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}//configure_dimension.json'
+
+- name: Copy new 'Configure Dimension' survey.json to target machine
+  copy:
+    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/configure_dimension.json'
+    dest:  '/matrix/awx/configure_dimension.json'
+    mode: '0660'
+
+- name: Collect AWX admin token the hard way!
+  delegate_to: 127.0.0.1
+  shell: |
+      curl -sku {{ tower_username }}:{{ tower_password }} -H "Content-Type: application/json" -X POST -d '{"description":"Tower CLI", "application":null, "scope":"write"}' https://{{ tower_host }}/api/v2/users/1/personal_tokens/ | jq '.token' | sed -r 's/\"//g'
+  register: tower_token
+  no_log: True
+
+- name: Recreate 'Configure Dimension' job template
+  delegate_to: 127.0.0.1
+  awx.awx.tower_job_template:
+    name: "{{ matrix_domain }} - 1 - Configure Dimension"
+    description: "Configure Dimension, the self-hosted integrations server."
+    extra_vars: "{{ lookup('file', '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/extra_vars.json') }}"
+    job_type: run
+    job_tags: "start,setup-all,setup-dimension"
+    inventory: "{{ member_id }}"
+    project: "{{ member_id }} - Matrix Docker Ansible Deploy"
+    playbook: setup.yml
+    credential: "{{ member_id }} - AWX SSH Key"
+    survey_enabled: true
+    survey_spec: "{{ lookup('file', '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/configure_dimension.json') }}"
+    become_enabled: yes
+    state: present
+    verbosity: 1
+    tower_host: "https://{{ tower_host }}"
+    tower_oauthtoken: "{{ tower_token.stdout }}"
+    validate_certs: yes
--- a/roles/matrix-awx/tasks/set_variables_element.yml
+++ b/roles/matrix-awx/tasks/set_variables_element.yml
@ -2,10 +2,10 @@
 - name: Record Element-Web variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Element Settings'
+    insertafter: '# Element Settings Start'
  with_dict:
    'matrix_client_element_enabled': '{{ matrix_client_element_enabled }}'
    'matrix_client_element_jitsi_preferredDomain': '{{ matrix_client_element_jitsi_preferredDomain }}'
@ -16,24 +16,18 @@

 - name: Set fact for 'https' string
  set_fact:
-    https_string: "https"
+    awx_https_string: "https"

 - name: Record Element-Web Background variable locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Element Settings'
+    insertafter: '# Element Settings Start'
  with_dict:
    'matrix_client_element_branding_welcomeBackgroundUrl': '{{ matrix_client_element_branding_welcomeBackgroundUrl }}'
-  when: (https_string in matrix_client_element_branding_welcomeBackgroundUrl) and ( matrix_client_element_branding_welcomeBackgroundUrl|length > 0 )
-
- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
+  when: (awx_https_string in matrix_client_element_branding_welcomeBackgroundUrl) and ( matrix_client_element_branding_welcomeBackgroundUrl|length > 0 )

 - name: Save new 'Configure Element' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
--- a/roles/matrix-awx/tasks/set_variables_jitsi.yml
+++ b/roles/matrix-awx/tasks/set_variables_jitsi.yml
@ -2,20 +2,14 @@
 - name: Record Jitsi variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Jitsi Settings'
+    insertafter: '# Jitsi Settings Start'
  with_dict:
    'matrix_jitsi_enabled': '{{ matrix_jitsi_enabled }}'
    'matrix_jitsi_web_config_defaultLanguage': '{{ matrix_jitsi_web_config_defaultLanguage }}'

- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
-
 - name: Save new 'Configure Jitsi' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
  template:
--- a/roles/matrix-awx/tasks/set_variables_ma1sd.yml
+++ b/roles/matrix-awx/tasks/set_variables_ma1sd.yml
@ -2,20 +2,20 @@
 - name: Record ma1sd variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# ma1sd Settings'
+    insertafter: '# ma1sd Settings Start'
  with_dict:
    'matrix_ma1sd_enabled': '{{ matrix_ma1sd_enabled }}'

 - name: Disable REST auth (matrix-corporal/ma1sd) if using internal auth
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Extension'
+    insertafter: '# Synapse Extension Start'
  with_dict:
    'matrix_synapse_ext_password_provider_rest_auth_enabled': 'false'
  when: ext_matrix_ma1sd_auth_store == 'Synapse Internal'
@ -23,10 +23,10 @@
 - name: Enable REST auth if using external LDAP/AD with ma1sd
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Extension'
+    insertafter: '# Synapse Extension Start'
  with_dict:
    'matrix_synapse_ext_password_provider_rest_auth_enabled': 'true'
    'matrix_synapse_ext_password_provider_rest_auth_endpoint': 'http://matrix-ma1sd:8090'
@ -40,7 +40,7 @@
 - name: Remove entire ma1sd configuration extension
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: '^.*\n'
    after: '# Start ma1sd Extension'
    before: '# End ma1sd Extension'
@ -48,14 +48,14 @@
 - name: Replace conjoined ma1sd configuration extension limiters
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: '^# Start ma1sd Extension# End ma1sd Extension'
    replace: '# Start ma1sd Extension\n# End ma1sd Extension'

 - name: Insert ma1sd configuration extension header if using external LDAP/AD with ma1sd
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    line: "matrix_ma1sd_configuration_extension_yaml: |"
    insertafter: '# Start ma1sd Extension'
  when: ext_matrix_ma1sd_auth_store == 'LDAP/AD'
@ -63,7 +63,7 @@
 - name: Set ma1sd configuration extension if using external LDAP/AD with ma1sd
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    insertbefore: '# End ma1sd Extension'
    line: '{{ item }}'
  with_items: "{{ ext_matrix_ma1sd_configuration_extension_yaml_parsed }}"
@ -72,20 +72,14 @@
 - name: Record ma1sd Custom variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertbefore: '# Custom Settings'
+    insertbefore: '# Custom Settings Start'
  with_dict:
    'ext_matrix_ma1sd_auth_store': '{{ ext_matrix_ma1sd_auth_store }}'
    'ext_matrix_ma1sd_configuration_extension_yaml': '{{ ext_matrix_ma1sd_configuration_extension_yaml.splitlines() | to_json }}'

- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
-
 - name: Save new 'Configure ma1sd' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
  template:
@ -126,4 +120,3 @@
    tower_oauthtoken: "{{ tower_token.stdout }}"
    validate_certs: yes

-# ^ This playbook isn't executing so the survey isn't being updated! :P
--- a/roles/matrix-awx/tasks/set_variables_synapse.yml
+++ b/roles/matrix-awx/tasks/set_variables_synapse.yml
@ -12,10 +12,10 @@
 - name: Record Synapse variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Settings'
+    insertafter: '# Synapse Settings Start'
  with_dict:
    'matrix_synapse_allow_public_rooms_over_federation': '{{ matrix_synapse_allow_public_rooms_over_federation }}'
    'matrix_synapse_enable_registration': '{{ matrix_synapse_enable_registration }}'
@ -29,7 +29,7 @@
 - name: Empty Synapse variable 'matrix_synapse_auto_join_rooms' locally on AWX, if raw inputs empty
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^matrix_synapse_auto_join_rooms: .*$"
    replace: "matrix_synapse_auto_join_rooms: []"
  when: matrix_synapse_auto_join_rooms_raw|length == 0
@ -43,10 +43,10 @@
 - name: Record Synapse variable 'matrix_synapse_auto_join_rooms' locally on AWX, if it's not blank
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Settings'
+    insertafter: '# Synapse Settings Start'
  with_dict:
    "matrix_synapse_auto_join_rooms": "{{ matrix_synapse_auto_join_rooms_array }}"
  when: matrix_synapse_auto_join_rooms_raw|length > 0
@ -54,10 +54,10 @@
 - name: Record Synapse Shared Secret if it's defined
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Settings'
+    insertafter: '# Synapse Settings Start'
  with_dict:
    'matrix_synapse_registration_shared_secret': '{{ matrix_synapse_registration_shared_secret }}'
  when: matrix_synapse_registration_shared_secret|length > 0
@ -65,10 +65,10 @@
 - name: Record registations_require_3pid extra variable if true
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "{{ item }}:"
    line: "{{ item }}"
-    insertbefore: '# End Synapse Extension'
+    insertbefore: '# Synapse Extension End'
  with_items:
    - "  registrations_require_3pid:"
    - "    - email"
@ -77,10 +77,10 @@
 - name: Remove registrations_require_3pid extra variable if false
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "{{ item }}:"
    line: "{{ item }}"
-    insertbefore: '# End Synapse Extension'
+    insertbefore: '# Synapse Extension End'
    state: absent
  with_items:
    - "  registrations_require_3pid:"
@ -90,10 +90,10 @@
 - name: Remove URL Languages
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: '^(?!.*\bemail\b)    - [a-zA-Z\-]{2,5}\n'
    after: '  url_preview_accept_language:'
-    before: '# End Synapse Extension'
+    before: '# Synapse Extension End'

 - name: Set URL languages default if raw inputs empty
  set_fact:
@ -108,7 +108,7 @@
 - name: Set URL languages if raw inputs empty
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    insertafter: '^  url_preview_accept_language:'
    line: "    - {{ ext_url_preview_accept_language_default }}"
  when: ext_url_preview_accept_language_raw|length == 0
@ -116,7 +116,7 @@
 - name: Set URL languages if raw inputs not empty
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    insertafter: '^  url_preview_accept_language:'
    line: "    - {{ item }}"
  with_items: "{{ ext_url_preview_accept_language_raw.splitlines() }}"
@ -125,22 +125,22 @@
 - name: Remove Federation Whitelisting 1
  delegate_to: 127.0.0.1
  replace:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: '^    - [a-z0-9]+\.[a-z0-9.]+\n'
    after: '  federation_domain_whitelist:'
-    before: '# End Synapse Extension'
+    before: '# Synapse Extension End'

 - name: Remove Federation Whitelisting 2
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    line: "  federation_domain_whitelist:"
    state: absent

 - name: Set Federation Whitelisting 1
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    insertafter: '^matrix_synapse_configuration_extension_yaml: \|'
    line: "  federation_domain_whitelist:"
  when: ext_federation_whitelist_raw|length > 0
@ -148,7 +148,7 @@
 - name: Set Federation Whitelisting 2
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    insertafter: '^  federation_domain_whitelist:'
    line: "    - {{ item }}"
  with_items: "{{ ext_federation_whitelist_raw.splitlines() }}"
@ -157,10 +157,10 @@
 - name: Record Synapse Custom variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertbefore: '# Custom Settings'
+    insertafter: '# Custom Settings Start'
  with_dict:
    'ext_federation_whitelist_raw': '{{ ext_federation_whitelist_raw.splitlines() | to_json }}'
    'ext_url_preview_accept_language_default': '{{ ext_url_preview_accept_language_default.splitlines() | to_json }}'
@ -176,21 +176,15 @@
 - name: Record Synapse Extension variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertbefore: '# End Synapse Extension'
+    insertbefore: '# Synapse Extension End'
  with_dict:
    '  enable_registration_captcha': '{{ ext_enable_registration_captcha }}'
    '  recaptcha_public_key': '{{ ext_recaptcha_public_key }}'
    '  recaptcha_private_key': '{{ ext_recaptcha_private_key }}'

- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
-
 - name: Save new 'Configure Synapse' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
  template:
--- a/roles/matrix-awx/tasks/set_variables_synapse_admin.yml
+++ b/roles/matrix-awx/tasks/set_variables_synapse_admin.yml
@ -2,19 +2,13 @@
 - name: Record Synapse Admin variables locally on AWX
  delegate_to: 127.0.0.1
  lineinfile:
-    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    path: '{{ awx_cached_matrix_vars }}'
    regexp: "^#? *{{ item.key | regex_escape() }}:"
    line: "{{ item.key }}: {{ item.value }}"
-    insertafter: '# Synapse Admin Settings'
+    insertafter: '# Synapse Admin Settings Start'
  with_dict:
    'matrix_synapse_admin_enabled': '{{ matrix_synapse_admin_enabled }}'

- name: Copy new 'matrix_vars.yml' to target machine
-  copy:
-    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
-    dest: '/matrix/awx/matrix_vars.yml'
-    mode: '0660'
-
 - name: Save new 'Configure Synapse Admin' survey.json to the AWX tower, template
  delegate_to: 127.0.0.1
  template:
--- a/roles/matrix-bot-mjolnir/defaults/main.yml
+++ b/roles/matrix-bot-mjolnir/defaults/main.yml
@ -0,0 +1,52 @@
+# A moderation tool for Matrix
+# See: https://github.com/matrix-org/mjolnir
+
+matrix_bot_mjolnir_enabled: true
+matrix_bot_mjolnir_version: "v0.1.17"
+matrix_bot_mjolnir_docker_image: "docker.io/matrixdotorg/mjolnir:{{ matrix_bot_mjolnir_version }}"
+matrix_bot_mjolnir_docker_image_force_pull: "{{ matrix_bot_mjolnir_docker_image.endswith(':latest') }}"
+
+matrix_bot_mjolnir_base_path: "{{ matrix_base_data_path }}/mjolnir"
+matrix_bot_mjolnir_config_path: "{{ matrix_bot_mjolnir_base_path }}/config"
+matrix_bot_mjolnir_data_path: "{{ matrix_bot_mjolnir_base_path }}/data"
+
+# A list of extra arguments to pass to the container
+matrix_bot_mjolnir_container_extra_arguments: []
+
+# List of systemd services that matrix-bot-mjolnir.service depends on
+matrix_bot_mjolnir_systemd_required_services_list: ['docker.service']
+
+# List of systemd services that matrix-bot-mjolnir.service wants
+matrix_bot_mjolnir_systemd_wanted_services_list: []
+
+# The access token for the bot user
+matrix_bot_mjolnir_access_token: ""
+
+# The room ID where people can use the bot. The bot has no access controls, so
+# anyone in this room can use the bot - secure your room!
+# This should be a room alias or room ID - not a matrix.to URL.
+# Note: Mjolnir is fairly verbose - expect a lot of messages from it.
+matrix_bot_mjolnir_management_room: ""
+
+# Default configuration template which covers the generic use case.
+# You can customize it by controlling the various variables inside it.
+#
+# For a more advanced customization, you can extend the default (see `matrix_bot_mjolnir_configuration_extension_yaml`)
+# or completely replace this variable with your own template.
+matrix_bot_mjolnir_configuration_yaml: "{{ lookup('template', 'templates/production.yaml.j2') }}"
+
+matrix_bot_mjolnir_configuration_extension_yaml: |
+  # Your custom YAML configuration goes here.
+  # This configuration extends the default starting configuration (`matrix_bot_mjolnir_configuration_yaml`).
+  #
+  # You can override individual variables from the default configuration, or introduce new ones.
+  #
+  # If you need something more special, you can take full control by
+  # completely redefining `matrix_bot_mjolnir_configuration_yaml`.
+
+matrix_bot_mjolnir_configuration_extension: "{{ matrix_bot_mjolnir_configuration_extension_yaml|from_yaml if matrix_bot_mjolnir_configuration_extension_yaml|from_yaml is mapping else {} }}"
+
+# Holds the final configuration (a combination of the default and its extension).
+# You most likely don't need to touch this variable. Instead, see `matrix_bot_mjolnir_configuration_yaml`.
+matrix_bot_mjolnir_configuration: "{{ matrix_bot_mjolnir_configuration_yaml|from_yaml|combine(matrix_bot_mjolnir_configuration_extension, recursive=True) }}"
+
--- a/roles/matrix-bot-mjolnir/tasks/init.yml
+++ b/roles/matrix-bot-mjolnir/tasks/init.yml
@ -0,0 +1,3 @@
+- set_fact:
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-bot-mjolnir.service'] }}"
+  when: matrix_bot_mjolnir_enabled|bool
--- a/roles/matrix-bot-mjolnir/tasks/main.yml
+++ b/roles/matrix-bot-mjolnir/tasks/main.yml
@ -0,0 +1,21 @@
+- import_tasks: "{{ role_path }}/tasks/init.yml"
+  tags:
+    - always
+
+- import_tasks: "{{ role_path }}/tasks/validate_config.yml"
+  when: "run_setup|bool and matrix_bot_mjolnir_enabled|bool"
+  tags:
+    - setup-all
+    - setup-bot-mjolnir
+
+- import_tasks: "{{ role_path }}/tasks/setup_install.yml"
+  when: "run_setup|bool and matrix_bot_mjolnir_enabled|bool"
+  tags:
+    - setup-all
+    - setup-bot-mjolnir
+
+- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
+  when: "run_setup|bool and not matrix_bot_mjolnir_enabled|bool"
+  tags:
+    - setup-all
+    - setup-bot-mjolnir
--- a/roles/matrix-bot-mjolnir/tasks/setup_install.yml
+++ b/roles/matrix-bot-mjolnir/tasks/setup_install.yml
@ -0,0 +1,50 @@
+---
+
+- set_fact:
+    matrix_bot_mjolnir_requires_restart: false
+
+- name: Ensure matrix-bot-mjolnir paths exist
+  file:
+    path: "{{ item.path }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - { path: "{{ matrix_bot_mjolnir_base_path }}", when: true }
+    - { path: "{{ matrix_bot_mjolnir_config_path }}", when: true }
+    - { path: "{{ matrix_bot_mjolnir_data_path }}", when: true }
+  when: "item.when|bool"
+
+- name: Ensure mjolnir image is pulled
+  docker_image:
+    name: "{{ matrix_bot_mjolnir_docker_image }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_bot_mjolnir_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_bot_mjolnir_docker_image_force_pull }}"
+
+- name: Ensure matrix-bot-mjolnir config installed
+  copy:
+    content: "{{ matrix_bot_mjolnir_configuration|to_nice_yaml }}"
+    dest: "{{ matrix_bot_mjolnir_config_path }}/production.yaml"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+
+- name: Ensure matrix-bot-mjolnir.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-bot-mjolnir.service.j2"
+    dest: "{{ matrix_systemd_path }}/matrix-bot-mjolnir.service"
+    mode: 0644
+  register: matrix_bot_mjolnir_systemd_service_result
+
+- name: Ensure systemd reloaded after matrix-bot-mjolnir.service installation
+  service:
+    daemon_reload: yes
+  when: "matrix_bot_mjolnir_systemd_service_result.changed|bool"
+
+- name: Ensure matrix-bot-mjolnir.service restarted, if necessary
+  service:
+    name: "matrix-bot-mjolnir.service"
+    state: restarted
+  when: "matrix_bot_mjolnir_requires_restart|bool"
--- a/roles/matrix-bot-mjolnir/tasks/setup_uninstall.yml
+++ b/roles/matrix-bot-mjolnir/tasks/setup_uninstall.yml
@ -0,0 +1,35 @@
+---
+
+- name: Check existence of matrix-bot-mjolnir service
+  stat:
+    path: "{{ matrix_systemd_path }}/matrix-bot-mjolnir.service"
+  register: matrix_bot_mjolnir_service_stat
+
+- name: Ensure matrix-bot-mjolnir is stopped
+  service:
+    name: matrix-bot-mjolnir
+    state: stopped
+    daemon_reload: yes
+  register: stopping_result
+  when: "matrix_bot_mjolnir_service_stat.stat.exists|bool"
+
+- name: Ensure matrix-bot-mjolnir.service doesn't exist
+  file:
+    path: "{{ matrix_systemd_path }}/matrix-bot-mjolnir.service"
+    state: absent
+  when: "matrix_bot_mjolnir_service_stat.stat.exists|bool"
+
+- name: Ensure systemd reloaded after matrix-bot-mjolnir.service removal
+  service:
+    daemon_reload: yes
+  when: "matrix_bot_mjolnir_service_stat.stat.exists|bool"
+
+- name: Ensure matrix-bot-mjolnir paths don't exist
+  file:
+    path: "{{ matrix_bot_mjolnir_base_path }}"
+    state: absent
+
+- name: Ensure mjolnir Docker image doesn't exist
+  docker_image:
+    name: "{{ matrix_bot_mjolnir_docker_image }}"
+    state: absent
--- a/roles/matrix-bot-mjolnir/tasks/validate_config.yml
+++ b/roles/matrix-bot-mjolnir/tasks/validate_config.yml
@ -0,0 +1,9 @@
+---
+
+- name: Fail if required variables are undefined
+  fail:
+    msg: "The `{{ item }}` variable must be defined and have a non-null value."
+  with_items:
+    - "matrix_bot_mjolnir_access_token"
+    - "matrix_bot_mjolnir_management_room"
+  when: "vars[item] == '' or vars[item] is none"
--- a/roles/matrix-bot-mjolnir/templates/production.yaml.j2
+++ b/roles/matrix-bot-mjolnir/templates/production.yaml.j2
@ -0,0 +1,162 @@
+# Where the homeserver is located (client-server URL). This should point at
+# pantalaimon if you're using that.
+homeserverUrl: "{{ matrix_homeserver_url }}"
+
+# The access token for the bot to use. Do not populate if using Pantalaimon.
+accessToken: "{{ matrix_bot_mjolnir_access_token }}"
+
+# Pantalaimon options (https://github.com/matrix-org/pantalaimon)
+#pantalaimon:
+#  # If true, accessToken above is ignored and the username/password below will be
+#  # used instead. The access token of the bot will be stored in the dataPath.
+#  use: false
+#
+#  # The username to login with.
+#  username: mjolnir
+#
+#  # The password to login with. Can be removed after the bot has logged in once and
+#  # stored the access token.
+#  password: your_password
+
+# The directory the bot should store various bits of information in
+dataPath: "/data"
+
+# If true (the default), only users in the `managementRoom` can invite the bot
+# to new rooms.
+autojoinOnlyIfManager: true
+
+# If `autojoinOnlyIfManager` is false, only the members in this group can invite
+# the bot to new rooms.
+#acceptInvitesFromGroup: '+example:example.org'
+
+# If the bot is invited to a room and it won't accept the invite (due to the
+# conditions above), report it to the management room. Defaults to disabled (no
+# reporting).
+recordIgnoredInvites: false
+
+# The room ID where people can use the bot. The bot has no access controls, so
+# anyone in this room can use the bot - secure your room!
+# This should be a room alias or room ID - not a matrix.to URL.
+# Note: Mjolnir is fairly verbose - expect a lot of messages from it.
+managementRoom: "{{ matrix_bot_mjolnir_management_room }}"
+
+# Set to false to make the management room a bit quieter.
+verboseLogging: false
+
+# The log level for the logs themselves. One of DEBUG, INFO, WARN, and ERROR.
+# This should be at INFO or DEBUG in order to get support for Mjolnir problems.
+logLevel: "INFO"
+
+# Set to false to disable synchronizing the ban lists on startup. If true, this
+# is the same as running !mjolnir sync immediately after startup.
+syncOnStartup: true
+
+# Set to false to prevent Mjolnir from checking its permissions on startup. This
+# is recommended to be left as "true" to catch room permission problems (state
+# resets, etc) before Mjolnir is needed.
+verifyPermissionsOnStartup: true
+
+# If true, Mjolnir won't actually ban users or apply server ACLs, but will
+# think it has. This is useful to see what it does in a scenario where the
+# bot might not be trusted fully, yet. Default false (do bans/ACLs).
+noop: false
+
+# Set to true to use /joined_members instead of /state to figure out who is
+# in the room. Using /state is preferred because it means that users are
+# banned when they are invited instead of just when they join, though if your
+# server struggles with /state requests then set this to true.
+fasterMembershipChecks: false
+
+# A case-insensitive list of ban reasons to automatically redact a user's
+# messages for. Typically this is useful to avoid having to type two commands
+# to the bot. Use asterisks to represent globs (ie: "spam*testing" would match
+# "spam for testing" as well as "spamtesting").
+automaticallyRedactForReasons:
+ - "spam"
+ - "advertising"
+
+# A list of rooms to protect (matrix.to URLs)
+#protectedRooms:
+#  - "https://matrix.to/#/#yourroom:example.org"
+
+# Set this option to true to protect every room the bot is joined to. Note that
+# this effectively makes the protectedRooms and associated commands useless because
+# the bot by nature must be joined to the room to protect it.
+#
+# Note: the management room is *excluded* from this condition. Add it to the
+# protected rooms to protect it.
+#
+# Note: ban list rooms the bot is watching but didn't create will not be protected.
+# Manually add these rooms to the protected rooms list if you want them protected.
+protectAllJoinedRooms: false
+
+# Misc options for command handling and commands
+commands:
+  # If true, Mjolnir will respond to commands like !help and !ban instead of
+  # requiring a prefix. This is useful if Mjolnir is the only bot running in
+  # your management room.
+  #
+  # Note that Mjolnir can be pinged by display name instead of having to use
+  # the !mjolnir prefix. For example, "my_moderator_bot: ban @spammer:example.org"
+  # will ban a user.
+  allowNoPrefix: false
+
+  # In addition to the bot's display name, !mjolnir, and optionally no prefix
+  # above, the bot will respond to these names. The items here can be used either
+  # as display names or prefixed with exclamation points.
+  additionalPrefixes:
+    - "mjolnir_bot"
+
+  # If true, ban commands that use wildcard characters require confirmation with
+  # an extra `--force` argument
+  confirmWildcardBan: true
+
+# Configuration specific to certain toggleable protections
+#protections:
+#  # Configuration for the wordlist plugin, which can ban users based if they say certain
+#  # blocked words shortly after joining.
+#  wordlist:
+#    # A list of words which should be monitored by the bot.  These will match if any part
+#    # of the word is present in the message in any case.  e.g. "hello" also matches
+#    # "HEllO".  Additionally, regular expressions can be used.
+#    words:
+#      - "CaSe"
+#      - "InSeNsAtIve"
+#      - "WoRd"
+#      - "LiSt"
+#
+#    # How long after a user joins the server should the bot monitor their messages.  After
+#    # this time, users can say words from the wordlist without being banned automatically.
+#    # Set to zero to disable (users will always be banned if they say a bad word)
+#    minutesBeforeTrusting: 20
+
+# Options for monitoring the health of the bot
+health:
+  # healthz options. These options are best for use in container environments
+  # like Kubernetes to detect how healthy the service is. The bot will report
+  # that it is unhealthy until it is able to process user requests. Typically
+  # this means that it'll flag itself as unhealthy for a number of minutes
+  # before saying "Now monitoring rooms" and flagging itself healthy.
+  #
+  # Health is flagged through HTTP status codes, defined below.
+  healthz:
+    # Whether the healthz integration should be enabled (default false)
+    enabled: false
+
+    # The port to expose the webserver on. Defaults to 8080.
+    port: 8080
+
+    # The address to listen for requests on. Defaults to all addresses.
+    address: "0.0.0.0"
+
+    # The path to expose the monitoring endpoint at. Defaults to `/healthz`
+    endpoint: "/healthz"
+
+    # The HTTP status code which reports that the bot is healthy/ready to
+    # process requests. Typically this should not be changed. Defaults to
+    # 200.
+    healthyStatus: 200
+
+    # The HTTP status code which reports that the bot is not healthy/ready.
+    # Defaults to 418.
+    unhealthyStatus: 418
--- a/roles/matrix-bot-mjolnir/templates/systemd/matrix-bot-mjolnir.service.j2
+++ b/roles/matrix-bot-mjolnir/templates/systemd/matrix-bot-mjolnir.service.j2
@ -0,0 +1,42 @@
+#jinja2: lstrip_blocks: "True"
+[Unit]
+Description=Matrix Mjolnir bot
+{% for service in matrix_bot_mjolnir_systemd_required_services_list %}
+Requires={{ service }}
+After={{ service }}
+{% endfor %}
+{% for service in matrix_bot_mjolnir_systemd_required_services_list %}
+Wants={{ service }}
+{% endfor %}
+DefaultDependencies=no
+
+[Service]
+Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
+ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill matrix-bot-mjolnir 2>/dev/null'
+ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-bot-mjolnir 2>/dev/null'
+
+# Intentional delay, so that the homeserver (we likely depend on) can manage to start.
+ExecStartPre={{ matrix_host_command_sleep }} 5
+
+ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-bot-mjolnir \
+			--log-driver=none \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--read-only \
+			--network={{ matrix_docker_network }} \
+			--mount type=bind,src={{ matrix_bot_mjolnir_config_path }},dst=/data/config,ro \
+			--mount type=bind,src={{ matrix_bot_mjolnir_data_path }},dst=/data \
+			{% for arg in matrix_bot_mjolnir_container_extra_arguments %}
+			{{ arg }} \
+			{% endfor %}
+			{{ matrix_bot_mjolnir_docker_image }}
+
+ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill matrix-bot-mjolnir 2>/dev/null'
+ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-bot-mjolnir 2>/dev/null'
+Restart=always
+RestartSec=30
+SyslogIdentifier=matrix-bot-mjolnir
+
+[Install]
+WantedBy=multi-user.target
--- a/roles/matrix-client-element/defaults/main.yml
+++ b/roles/matrix-client-element/defaults/main.yml
@ -3,7 +3,7 @@ matrix_client_element_enabled: true
 matrix_client_element_container_image_self_build: false
 matrix_client_element_container_image_self_build_repo: "https://github.com/vector-im/riot-web.git"

-matrix_client_element_version: v1.7.24
+matrix_client_element_version: v1.7.24.1
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else 'docker.io/' }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"
--- a/roles/matrix-common-after/tasks/awx_post.yml
+++ b/roles/matrix-common-after/tasks/awx_post.yml
@ -1,6 +1,6 @@
 ---

- name: Create user account
+- name: Create user account @janitor
  command: |
    /usr/local/bin/matrix-synapse-register-user janitor {{ matrix_awx_janitor_user_password | quote }} 1
  register: cmd
@ -18,3 +18,20 @@
    'matrix_awx_janitor_user_created': 'true'
  when: not matrix_awx_janitor_user_created|bool

+- name: Create user account @dimension
+  command: |
+    /usr/local/bin/matrix-synapse-register-user dimension {{ matrix_awx_dimension_user_password | quote }} 0
+  register: cmd
+  when: not matrix_awx_dimension_user_created|bool
+  no_log: True
+  
+- name: Update AWX dimension user created variable
+  delegate_to: 127.0.0.1
+  lineinfile:
+    path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+    regexp: "^#? *{{ item.key | regex_escape() }}:"
+    line: "{{ item.key }}: {{ item.value }}"
+    insertafter: 'AWX Settings'
+  with_dict:
+    'matrix_awx_dimension_user_created': 'true'
+  when: not matrix_awx_dimension_user_created|bool
--- a/roles/matrix-grafana/defaults/main.yml
+++ b/roles/matrix-grafana/defaults/main.yml
@ -3,7 +3,7 @@

 matrix_grafana_enabled: false

-matrix_grafana_version: 7.4.0
+matrix_grafana_version: 7.5.2
 matrix_grafana_docker_image: "docker.io/grafana/grafana:{{ matrix_grafana_version }}"
 matrix_grafana_docker_image_force_pull: "{{ matrix_grafana_docker_image.endswith(':latest') }}"

--- a/roles/matrix-grafana/templates/grafana.ini.j2
+++ b/roles/matrix-grafana/templates/grafana.ini.j2
@ -1,16 +1,16 @@
 [security]
 # default admin user, created on startup
-admin_user = {{ matrix_grafana_default_admin_user }}
+admin_user = "{{ matrix_grafana_default_admin_user }}"

 # default admin password, can be changed before first start of grafana, or in profile settings
-admin_password = {{ matrix_grafana_default_admin_password }}
+admin_password = """{{ matrix_grafana_default_admin_password }}"""

 [auth.anonymous]
 # enable anonymous access
 enabled = {{ matrix_grafana_anonymous_access }}

 # specify organization name that should be used for unauthenticated users
-org_name = {{ matrix_grafana_anonymous_access_org_name }}
+org_name = "{{ matrix_grafana_anonymous_access_org_name }}"

 [dashboards]
 {% if matrix_synapse_metrics_enabled %}
--- a/roles/matrix-nginx-proxy/defaults/main.yml
+++ b/roles/matrix-nginx-proxy/defaults/main.yml
@ -227,6 +227,15 @@ matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key: "{{ matrix_s
 # The tmpfs at /tmp needs to be large enough to handle multiple concurrent file uploads.
 matrix_nginx_proxy_tmp_directory_size_mb: "{{ (matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb | int) * 50 }}"

+# A list of strings containing additional configuration blocks to add to the nginx server configuration (nginx.conf).
+# for big matrixservers to enlarge the number of open files to prevent timeouts
+# matrix_nginx_proxy_proxy_additional_configuration_blocks:
+#  - 'worker_rlimit_nofile 30000;'
+matrix_nginx_proxy_proxy_additional_configuration_blocks: []
+
+# A list of strings containing additional configuration blocks to add to the nginx event server configuration (nginx.conf).
+matrix_nginx_proxy_proxy_event_additional_configuration_blocks: []
+
 # A list of strings containing additional configuration blocks to add to the nginx http's server configuration (nginx-http.conf).
 matrix_nginx_proxy_proxy_http_additional_server_configuration_blocks: []

--- a/roles/matrix-nginx-proxy/templates/nginx/nginx.conf.j2
+++ b/roles/matrix-nginx-proxy/templates/nginx/nginx.conf.j2
@ -11,10 +11,15 @@
 worker_processes {{ matrix_nginx_proxy_worker_processes }};
 error_log /var/log/nginx/error.log warn;
 pid /tmp/nginx.pid;
-
+{% for configuration_block in matrix_nginx_proxy_proxy_additional_configuration_blocks %}
+	{{- configuration_block }}
+{% endfor %}

 events {
 	worker_connections {{ matrix_nginx_proxy_worker_connections }};
+{% for configuration_block in matrix_nginx_proxy_proxy_event_additional_configuration_blocks %}
+	{{- configuration_block }}
+{% endfor %}
 }


--- a/roles/matrix-postgres-backup/defaults/main.yml
+++ b/roles/matrix-postgres-backup/defaults/main.yml
@ -0,0 +1,40 @@
+matrix_postgres_backup_enabled: false
+
+matrix_postgres_backup_connection_hostname: "matrix-postgres"
+matrix_postgres_backup_connection_port: 5432
+matrix_postgres_backup_connection_username: "matrix"
+matrix_postgres_backup_connection_password: ""
+
+matrix_postgres_backup_extra_opts: "-Z9 --schema=public --blobs"
+matrix_postgres_backup_schedule: "@daily"
+matrix_postgres_backup_keep_days: 7
+matrix_postgres_backup_keep_weeks: 4
+matrix_postgres_backup_keep_months: 12
+matrix_postgres_backup_healthcheck_port: "8080"
+matrix_postgres_backup_databases: []
+matrix_postgres_backup_path: "{{ matrix_base_data_path }}/postgres-backup"
+
+# Specifies where the Postgres data is.
+# We use this to autodetect the Postgres version during playbook runtime (by parsing the `PG_VERSION` file contained there).
+# You can leave this empty to prevent auto-detection.
+matrix_postgres_backup_postgres_data_path: ""
+
+matrix_postgres_backup_architecture: amd64
+
+# matrix_postgres_docker_image_suffix controls whether we use Alpine-based images (`-alpine`) or the normal Debian-based images.
+# Alpine-based Postgres images are smaller and we usually prefer them, but they don't work on ARM32 (tested on a Raspberry Pi 3 running Raspbian 10.7).
+# On ARM32, `-alpine` images fail with the following error:
+# > LOG:  startup process (PID 37) was terminated by signal 11: Segmentation fault
+matrix_postgres_backup_docker_image_suffix: "{{ '-alpine' if matrix_postgres_backup_architecture in ['amd64', 'arm64'] else '' }}"
+
+matrix_postgres_backup_docker_image_v9: "docker.io/prodrigestivill/postgres-backup-local:9.6{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v10: "docker.io/prodrigestivill/postgres-backup-local:10{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v11: "docker.io/prodrigestivill/postgres-backup-local:11{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v12: "docker.io/prodrigestivill/postgres-backup-local:12{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v13: "docker.io/prodrigestivill/postgres-backup-local:13{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_latest: "{{ matrix_postgres_backup_docker_image_v13 }}"
+
+# This variable is assigned at runtime. Overriding its value has no effect.
+matrix_postgres_backup_docker_image_to_use: '{{ matrix_postgres_backup_docker_image_latest }}'
+
+matrix_postgres_backup_docker_image_force_pull: "{{ matrix_postgres_backup_docker_image_to_use.endswith(':latest') }}"
--- a/roles/matrix-postgres-backup/tasks/init.yml
+++ b/roles/matrix-postgres-backup/tasks/init.yml
@ -0,0 +1,3 @@
+- set_fact:
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-postgres-backup.service'] }}"
+  when: matrix_postgres_backup_enabled|bool
--- a/roles/matrix-postgres-backup/tasks/main.yml
+++ b/roles/matrix-postgres-backup/tasks/main.yml
@ -0,0 +1,17 @@
+---
+
+- import_tasks: "{{ role_path }}/tasks/init.yml"
+  tags:
+    - always
+
+- import_tasks: "{{ role_path }}/tasks/validate_config.yml"
+  when: "run_setup|bool and matrix_postgres_backup_enabled|bool"
+  tags:
+    - setup-all
+    - setup-postgres-backup
+
+- import_tasks: "{{ role_path }}/tasks/setup_postgres_backup.yml"
+  when: run_setup|bool
+  tags:
+    - setup-all
+    - setup-postgres-backup
--- a/roles/matrix-postgres-backup/tasks/setup_postgres_backup.yml
+++ b/roles/matrix-postgres-backup/tasks/setup_postgres_backup.yml
@ -0,0 +1,103 @@
+---
+
+#
+# Tasks related to setting up an internal postgres server
+#
+- import_tasks: "{{ role_path }}/tasks/util/detect_existing_postgres_version.yml"
+  when: 'matrix_postgres_backup_enabled|bool and matrix_postgres_backup_postgres_data_path != ""'
+
+# If we have found an existing version (installed from before), we use its corresponding Docker image.
+# If not, we install using the latest Postgres.
+#
+# Upgrading is supposed to be performed separately and explicitly (see `upgrade_postgres.yml`).
+- set_fact:
+    matrix_postgres_backup_docker_image_to_use: "{{ matrix_postgres_backup_docker_image_latest if matrix_postgres_backup_detected_version_corresponding_docker_image|default('') == '' else matrix_postgres_backup_detected_version_corresponding_docker_image }}"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure postgres backup Docker image is pulled
+  docker_image:
+    name: "{{ matrix_postgres_backup_docker_image_to_use }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_postgres_backup_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_postgres_backup_docker_image_force_pull }}"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure Postgres backup paths exist
+  file:
+    path: "{{ item }}"
+    state: directory
+    mode: 0700
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - "{{ matrix_postgres_backup_path }}"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure Postgres environment variables file created
+  template:
+    src: "{{ role_path }}/templates/{{ item }}.j2"
+    dest: "{{ matrix_postgres_backup_path }}/{{ item }}"
+    mode: 0640
+  with_items:
+    - "env-postgres-backup"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure matrix-postgres-backup.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-postgres-backup.service.j2"
+    dest: "{{ matrix_systemd_path }}/matrix-postgres-backup.service"
+    mode: 0644
+  register: matrix_postgres_backup_systemd_service_result
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure systemd reloaded after matrix-postgres-backup.service installation
+  service:
+    daemon_reload: yes
+  when: "matrix_postgres_backup_enabled|bool and matrix_postgres_backup_systemd_service_result.changed"
+
+#
+# Tasks related to getting rid of the internal postgres backup server (if it was previously enabled)
+#
+
+- name: Check existence of matrix-postgres-backup service
+  stat:
+    path: "{{ matrix_systemd_path }}/matrix-postgres-backup.service"
+  register: matrix_postgres_backup_service_stat
+  when: "not matrix_postgres_backup_enabled|bool"
+
+- name: Ensure matrix-postgres-backup is stopped
+  service:
+    name: matrix-postgres-backup
+    state: stopped
+    daemon_reload: yes
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_service_stat.stat.exists"
+
+- name: Ensure matrix-postgres-backup.service doesn't exist
+  file:
+    path: "{{ matrix_systemd_path }}/matrix-postgres-backup.service"
+    state: absent
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_service_stat.stat.exists"
+
+- name: Ensure systemd reloaded after matrix-postgres-backup.service removal
+  service:
+    daemon_reload: yes
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_service_stat.stat.exists"
+
+- name: Check existence of matrix-postgres-backup backup path
+  stat:
+    path: "{{ matrix_postgres_backup_path }}"
+  register: matrix_postgres_backup_path_stat
+  when: "not matrix_postgres_backup_enabled|bool"
+
+# We just want to notify the user. Deleting data is too destructive.
+- name: Inject warning if matrix-postgres backup data remains
+  set_fact:
+    matrix_playbook_runtime_results: |
+      {{
+        matrix_playbook_runtime_results|default([])
+        +
+        [
+          "NOTE: You are not using the local backup service to backup the PostgreSQL database, but some old data remains from before in `{{ matrix_postgres_backup_path }}`. Feel free to delete it."
+        ]
+      }}
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_path_stat.stat.exists"
--- a/roles/matrix-postgres-backup/tasks/util/detect_existing_postgres_version.yml
+++ b/roles/matrix-postgres-backup/tasks/util/detect_existing_postgres_version.yml
@ -0,0 +1,56 @@
+---
+
+# This utility aims to determine if there is some existing Postgres version in use or not.
+# If there is, it also tries to detect the Docker image that corresponds to that version.
+
+- name: Initialize Postgres version determination variables (default to empty)
+  set_fact:
+    matrix_postgres_backup_detection_pg_version_path: "{{ matrix_postgres_data_path }}/PG_VERSION"
+    matrix_postgres_backup_detected_existing: false
+    matrix_postgres_backup_detected_version: ""
+    matrix_postgres_backup_detected_version_corresponding_docker_image: ""
+
+- name: Determine existing Postgres version (check PG_VERSION file)
+  stat:
+    path: "{{ matrix_postgres_backup_detection_pg_version_path }}"
+  register: result_pg_version_stat
+
+- set_fact:
+    matrix_postgres_backup_detected_existing: true
+  when: "result_pg_version_stat.stat.exists"
+
+- name: Determine existing Postgres version (read PG_VERSION file)
+  slurp:
+    src: "{{ matrix_postgres_backup_detection_pg_version_path }}"
+  register: result_pg_version
+  when: matrix_postgres_backup_detected_existing|bool
+
+- name: Determine existing Postgres version (make sense of PG_VERSION file)
+  set_fact:
+    matrix_postgres_backup_detected_version: "{{ result_pg_version['content']|b64decode|replace('\n', '') }}"
+  when: matrix_postgres_backup_detected_existing|bool
+
+- name: Determine corresponding Docker image to detected version (assume default of latest)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_latest }}"
+  when: "matrix_postgres_backup_detected_version != ''"
+
+- name: Determine corresponding Docker image to detected version (use 9.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v9 }}"
+  when: "matrix_postgres_backup_detected_version.startswith('9.')"
+
+- name: Determine corresponding Docker image to detected version (use 10.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v10 }}"
+  when: "matrix_postgres_backup_detected_version == '10' or matrix_postgres_backup_detected_version.startswith('10.')"
+
+- name: Determine corresponding Docker image to detected version (use 11.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v11 }}"
+  when: "matrix_postgres_backup_detected_version == '11' or matrix_postgres_backup_detected_version.startswith('11.')"
+
+- name: Determine corresponding Docker image to detected version (use 12.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v12 }}"
+  when: "matrix_postgres_backup_detected_version == '12' or matrix_postgres_backup_detected_version.startswith('12.')"
--- a/roles/matrix-postgres-backup/tasks/validate_config.yml
+++ b/roles/matrix-postgres-backup/tasks/validate_config.yml
@ -0,0 +1,18 @@
+---
+
+- name: Fail if required Postgres settings not defined
+  fail:
+    msg: >-
+      You need to define a required configuration setting (`{{ item }}`).
+  when: "vars[item] == ''"
+  with_items:
+    - "matrix_postgres_backup_connection_hostname"
+    - "matrix_postgres_backup_connection_username"
+    - "matrix_postgres_backup_connection_password"
+    - "matrix_postgres_backup_connection_port"
+    - "matrix_postgres_backup_schedule"
+    - "matrix_postgres_backup_keep_days"
+    - "matrix_postgres_backup_keep_weeks"
+    - "matrix_postgres_backup_keep_months"
+    - "matrix_postgres_backup_path"
+    - "matrix_postgres_backup_databases"
--- a/roles/matrix-postgres-backup/templates/env-postgres-backup.j2
+++ b/roles/matrix-postgres-backup/templates/env-postgres-backup.j2
@ -0,0 +1,12 @@
+#jinja2: lstrip_blocks: "True"
+POSTGRES_USER={{ matrix_postgres_backup_connection_username }}
+POSTGRES_PASSWORD={{ matrix_postgres_backup_connection_password }}
+POSTGRES_HOST={{ matrix_postgres_backup_connection_hostname }}
+POSTGRES_DB={{ matrix_postgres_backup_databases|join(', ') }}
+POSTGRES_EXTRA_OPTS={{ matrix_postgres_backup_extra_opts }}
+SCHEDULE={{ matrix_postgres_backup_schedule }}
+BACKUP_KEEP_DAYS={{ matrix_postgres_backup_keep_days }}
+BACKUP_KEEP_WEEKS={{ matrix_postgres_backup_keep_weeks }}
+BACKUP_KEEP_MONTHS={{ matrix_postgres_backup_keep_months }}
+HEALTHCHECK_PORT={{ matrix_postgres_backup_healthcheck_port }}
+POSTGRES_PORT={{ matrix_postgres_backup_connection_port }}
--- a/roles/matrix-postgres-backup/templates/systemd/matrix-postgres-backup.service.j2
+++ b/roles/matrix-postgres-backup/templates/systemd/matrix-postgres-backup.service.j2
@ -0,0 +1,31 @@
+#jinja2: lstrip_blocks: "True"
+[Unit]
+Description=Automatic Backup of  Matrix Postgres server
+After=docker.service
+Requires=docker.service
+DefaultDependencies=no
+
+[Service]
+Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
+ExecStartPre=-{{ matrix_host_command_docker }} stop matrix-postgres-backup
+ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-postgres-backup 2>/dev/null'
+
+ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-postgres-backup \
+			--log-driver=none \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--read-only \
+			--network={{ matrix_docker_network }} \
+			--env-file={{ matrix_postgres_backup_path }}/env-postgres-backup \
+			--mount type=bind,src={{ matrix_postgres_backup_path }},dst=/backups \
+			{{ matrix_postgres_backup_docker_image_to_use }}
+
+ExecStop=-{{ matrix_host_command_docker }} stop matrix-postgres-backup
+ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-postgres-backup 2>/dev/null'
+Restart=always
+RestartSec=30
+SyslogIdentifier=matrix-postgres-backup
+
+[Install]
+WantedBy=multi-user.target
--- a/roles/matrix-synapse/defaults/main.yml
+++ b/roles/matrix-synapse/defaults/main.yml
@ -15,8 +15,8 @@ matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_cont
 # amd64 gets released first.
 # arm32 relies on self-building, so the same version can be built immediately.
 # arm64 users need to wait for a prebuilt image to become available.
-matrix_synapse_version: v1.30.1
-matrix_synapse_version_arm64: v1.30.1
+matrix_synapse_version: v1.31.0
+matrix_synapse_version_arm64: v1.31.0
 matrix_synapse_docker_image_tag: "{{ matrix_synapse_version if matrix_architecture in ['arm32', 'amd64'] else matrix_synapse_version_arm64 }}"
 matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}"

@ -501,7 +501,7 @@ matrix_synapse_ext_password_provider_ldap_filter: ""
 # See: https://github.com/t2bot/synapse-simple-antispam
 matrix_synapse_ext_spam_checker_synapse_simple_antispam_enabled: false
 matrix_synapse_ext_spam_checker_synapse_simple_antispam_git_repository_url: "https://github.com/t2bot/synapse-simple-antispam"
-matrix_synapse_ext_spam_checker_synapse_simple_antispam_git_version: "f058d9ce2c7d4195ae461dcdd02df11a2d06a36b"
+matrix_synapse_ext_spam_checker_synapse_simple_antispam_git_version: "923ca5c85b08f157181721abbae50dd89c31e4b5"
 matrix_synapse_ext_spam_checker_synapse_simple_antispam_config_blocked_homeservers: []

 matrix_s3_media_store_enabled: false
--- a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2
+++ b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2
@ -869,10 +869,10 @@ rc_admin_redaction: {{ matrix_synapse_rc_admin_redaction|to_json }}
 #rc_joins:
 #  local:
 #    per_second: 0.1
-#    burst_count: 3
+#    burst_count: 10
 #  remote:
 #    per_second: 0.01
-#    burst_count: 3
+#    burst_count: 10
 rc_joins: {{ matrix_synapse_rc_joins|to_json }}
 #
 #rc_3pid_validation:
@ -1772,6 +1772,9 @@ saml2_config:
 #       Note that, if this is changed, users authenticating via that provider
 #       will no longer be recognised as the same user!
 #
+#       (Use "oidc" here if you are migrating from an old "oidc_config"
+#       configuration.)
+#
 #   idp_name: A user-facing name for this identity provider, which is used to
 #       offer the user a choice of login mechanisms.
 #
@ -1887,6 +1890,24 @@ saml2_config:
 #           which is set to the claims returned by the UserInfo Endpoint and/or
 #           in the ID Token.
 #
+#   It is possible to configure Synapse to only allow logins if certain attributes
+#   match particular values in the OIDC userinfo. The requirements can be listed under
+#   `attribute_requirements` as shown below. All of the listed attributes must
+#   match for the login to be permitted. Additional attributes can be added to
+#   userinfo by expanding the `scopes` section of the OIDC config to retrieve
+#   additional information from the OIDC provider.
+#
+#   If the OIDC claim is a list, then the attribute must match any value in the list.
+#   Otherwise, it must exactly match the value of the claim. Using the example
+#   below, the `family_name` claim MUST be "Stephensson", but the `groups`
+#   claim MUST contain "admin".
+#
+#   attribute_requirements:
+#     - attribute: family_name
+#       value: "Stephensson"
+#     - attribute: groups
+#       value: "admin"
+#
 # See https://github.com/matrix-org/synapse/blob/master/docs/openid.md
 # for information on how to configure these options.
 #
@ -1918,34 +1939,9 @@ oidc_providers:
  #      localpart_template: "{% raw %}{{ user.login }}{% endraw %}"
  #      display_name_template: "{% raw %}{{ user.name }}{% endraw %}"
  #      email_template: "{% raw %}{{ user.email }}{% endraw %}"
-
-  # For use with Keycloak
-  #
-  #- idp_id: keycloak
-  #  idp_name: Keycloak
-  #  issuer: "https://127.0.0.1:8443/auth/realms/my_realm_name"
-  #  client_id: "synapse"
-  #  client_secret: "copy secret generated in Keycloak UI"
-  #  scopes: ["openid", "profile"]
-
-  # For use with Github
-  #
-  #- idp_id: github
-  #  idp_name: Github
-  #  idp_brand: github
-  #  discover: false
-  #  issuer: "https://github.com/"
-  #  client_id: "your-client-id" # TO BE FILLED
-  #  client_secret: "your-client-secret" # TO BE FILLED
-  #  authorization_endpoint: "https://github.com/login/oauth/authorize"
-  #  token_endpoint: "https://github.com/login/oauth/access_token"
-  #  userinfo_endpoint: "https://api.github.com/user"
-  #  scopes: ["read:user"]
-  #  user_mapping_provider:
-  #    config:
-  #      subject_claim: "id"
-  #      localpart_template: "{% raw %}{{ user.login }}{% endraw %}"
-  #      display_name_template: "{% raw %}{{ user.name }}{% endraw %}"
+  #  attribute_requirements:
+  #    - attribute: userGroup
+  #      value: "synapseUsers"


 # Enable Central Authentication Service (CAS) for registration and login.
--- a/roles/matrix-synapse/vars/workers.yml
+++ b/roles/matrix-synapse/vars/workers.yml
@ -55,9 +55,6 @@ matrix_synapse_workers_generic_worker_endpoints:
  # Registration/login requests
  - ^/_matrix/client/(api/v1|r0|unstable)/login$
  - ^/_matrix/client/(r0|unstable)/register$
-  # FIXME: possible bug with SSO and multiple generic workers
-  # see https://github.com/matrix-org/synapse/issues/7530
-  # ^/_matrix/client/(r0|unstable)/auth/.*/fallback/web$

  # Event sending requests
  - ^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/redact
--- a/setup.yml
+++ b/setup.yml
@ -34,6 +34,7 @@
    - matrix-bridge-sms
    - matrix-bot-matrix-reminder-bot
    - matrix-bot-go-neb
+    - matrix-bot-mjolnir
    - matrix-synapse
    - matrix-synapse-admin
    - matrix-prometheus-node-exporter
@ -50,4 +51,6 @@
    - matrix-nginx-proxy
    - matrix-coturn
    - matrix-aux
+    - matrix-postgres-backup
    - matrix-common-after
+