Merge branch 'master' into pub.solar

This commit is contained in:
teutat3s 2022-09-18 13:54:11 +02:00
commit 71d239a28e
Signed by: teutat3s
GPG key ID: 18DAE600A6BBE705
79 changed files with 1355 additions and 331 deletions

View file

@ -13,7 +13,7 @@ jobs:
- name: Check out
uses: actions/checkout@v3
- name: Run yamllint
uses: frenck/action-yamllint@v1.2.0
uses: frenck/action-yamllint@v1.3.0
ansible-lint:
name: ansible-lint
runs-on: ubuntu-latest

View file

@ -1,3 +1,103 @@
# 2022-09-15
## (Potential Backward Compatibility Break) Major improvements to Synapse workers
People who are interested in running a Synapse worker setup should know that **our Synapse worker implementation is much more powerful now**:
- we've added support for [Stream writers](#stream-writers-support)
- we've added support for [multiple federation sender workers](#multiple-federation-sender-workers-support)
- we've added support for [multiple pusher workers](#multiple-pusher-workers-support)
- we've added support for [running background tasks on a worker](#background-tasks-can-run-on-a-worker)
- we've restored support for [`appservice` workers](#appservice-worker-support-is-back)
- we've restored support for [`user_dir` workers](#user-directory-worker-support-is-back)
- we've made it possible to [reliably use more than 1 `media_repository` worker](#using-more-than-1-media-repository-worker-is-now-more-reliable)
- see the [Potential Backward Incompatibilities after these Synapse worker changes](#potential-backward-incompatibilities-after-these-synapse-worker-changes)
### Stream writers support
From now on, the playbook lets you easily set up various [stream writer workers](https://matrix-org.github.io/synapse/latest/workers.html#stream-writers) which can handle different streams (`events` stream; `typing` URL endpoints, `to_device` URL endpoints, `account_data` URL endpoints, `receipts` URL endpoints, `presence` URL endpoints). All of this work was previously handled by the main Synapse process, but can now be offloaded to stream writer worker processes.
If you're using `matrix_synapse_workers_preset: one-of-each`, you'll automatically get 6 additional workers (one for each of the above stream types). Our `little-federation-helper` preset (meant to be quite minimal and focusing in improved federation performance) does not include stream writer workers.
If you'd like to customize the number of workers we also make that possible using these variables:
```yaml
# Synapse only supports more than 1 worker for the `events` stream.
# All other streams can utilize either 0 or 1 workers, not more than that.
matrix_synapse_workers_stream_writer_events_stream_workers_count: 5
matrix_synapse_workers_stream_writer_typing_stream_workers_count: 1
matrix_synapse_workers_stream_writer_to_device_stream_workers_count: 1
matrix_synapse_workers_stream_writer_account_data_stream_workers_count: 1
matrix_synapse_workers_stream_writer_receipts_stream_workers_count: 1
matrix_synapse_workers_stream_writer_presence_stream_workers_count: 1
```
### Multiple federation sender workers support
Until now, we only supported a single `federation_sender` worker (`matrix_synapse_workers_federation_sender_workers_count` could either be `0` or `1`).
From now on, you can have as many as you want to help with your federation traffic.
### Multiple pusher workers support
Until now, we only supported a single `pusher` worker (`matrix_synapse_workers_pusher_workers_count` could either be `0` or `1`).
From now on, you can have as many as you want to help with pushing notifications out.
### Background tasks can run on a worker
From now on, you can put [background task processing on a worker](https://matrix-org.github.io/synapse/latest/workers.html#background-tasks).
With `matrix_synapse_workers_preset: one-of-each`, you'll get one `background` worker automatically.
You can also control the `background` workers count with `matrix_synapse_workers_background_workers_count`. Only `0` or `1` workers of this type are supported by Synapse.
### Appservice worker support is back
We previously had an `appservice` worker type, which [Synapse deprecated in v1.59.0](https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types). So did we, at the time.
The new way to implement such workers is by using a `generic_worker` and dedicating it to the task of talking to Application Services.
From now on, we have support for this.
With `matrix_synapse_workers_preset: one-of-each`, you'll get one `appservice` worker automatically.
You can also control the `appservice` workers count with `matrix_synapse_workers_appservice_workers_count`. Only `0` or `1` workers of this type are supported by Synapse.
### User Directory worker support is back
We previously had a `user_dir` worker type, which [Synapse deprecated in v1.59.0](https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types). So did we, at the time.
The new way to implement such workers is by using a `generic_worker` and dedicating it to the task of serving the user directory.
From now on, we have support for this.
With `matrix_synapse_workers_preset: one-of-each`, you'll get one `user_dir` worker automatically.
You can also control the `user_dir` workers count with `matrix_synapse_workers_user_dir_workers_count`. Only `0` or `1` workers of this type are supported by Synapse.
### Using more than 1 media repository worker is now more reliable
With `matrix_synapse_workers_preset: one-of-each`, we only launch one `media_repository` worker.
If you've been configuring `matrix_synapse_workers_media_repository_workers_count` manually, you may have increased that to more workers.
When multiple media repository workers are in use, background tasks related to the media repository must always be configured to run on a single `media_repository` worker via `media_instance_running_background_jobs`. Until now, we weren't doing this correctly, but we now are.
### Potential Backward Incompatibilities after these Synapse worker changes
Below we'll discuss **potential backward incompatibilities**.
- **Worker names** (container names, systemd services, worker configuration files) **have changed**. Workers are now labeled sequentially (e.g. `matrix-synapse-worker_generic_worker-18111` -> `matrix-synapse-worker-generic-0`). The playbook will handle these changes automatically.
- Due to increased worker types support above, people who use `matrix_synapse_workers_preset: one-of-each` should be aware that with these changes, **the playbook will deploy 9 additional workers** (6 stream writers, 1 `appservice` worker, 1 `user_dir` worker, 1 background task worker). This **may increase RAM/CPU usage**, etc. If you find your server struggling, consider disabling some workers with the appropriate `matrix_synapse_workers_*_workers_count` variables.
- **Metric endpoints have also changed** (`/metrics/synapse/worker/generic_worker-18111` -> `/metrics/synapse/worker/generic-worker-0`). If you're [collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server), consider revisiting our [Collecting Synapse worker metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-synapse-worker-metrics-to-an-external-prometheus-server) docs and updating your Prometheus configuration. **If you're collecting metrics to the integrated Prometheus server** (not enabled by default), **your Prometheus configuration will be updated automatically**. Old data (from before this change) may stick around though.
- **the format of `matrix_synapse_workers_enabled_list` has changed**. You were never advised to use this variable for directly creating workers (we advise people to control workers using `matrix_synapse_workers_preset` or by tweaking `matrix_synapse_workers_*_workers_count` variables only), but some people may have started using the `matrix_synapse_workers_enabled_list` variable to gain more control over workers. If you're one of them, you'll need to adjust its value. See `roles/matrix-synapse/defaults/main.yml` for more information on the new format. The playbook will also do basic validation and complain if you got something wrong.
# 2022-09-09
## Cactus Comments support
Thanks to [Julian-Samuel Gebühr (@moan0s)](https://github.com/moan0s), the playbook can now set up [Cactus Comments](https://cactus.chat) - federated comment system for the web based on Matrix.
See our [Setting up a Cactus Comments server](docs/configuring-playbook-cactus-comments.md) documentation to get started.
# 2022-08-23
## Postmoogle email bridge support

View file

@ -137,6 +137,8 @@ Using this playbook, you can get the following services configured on your serve
- (optional) the [Buscarron](https://gitlab.com/etke.cc/buscarron) bot - see [docs/configuring-playbook-bot-buscarron.md](docs/configuring-playbook-bot-buscarron.md) for setup documentation
- (optional) [Cactus Comments](https://cactus.chat), a federated comment system built on matrix - see [docs/configuring-playbook-cactus-comments.md](docs/configuring-playbook-cactus-comments.md) for setup documentation
Basically, this playbook aims to get you up-and-running with all the necessities around Matrix, without you having to do anything else.
**Note**: the list above is exhaustive. It includes optional or even some advanced components that you will most likely not need.

View file

@ -28,18 +28,22 @@ If you are using Cloudflare DNS, make sure to disable the proxy and set all reco
## DNS settings for optional services/features
| Type | Host | Priority | Weight | Port | Target |
| ----- | ---------------------------- | -------- | ------ | ---- | ---------------------- |
| SRV | `_matrix-identity._tcp` | 10 | 0 | 443 | `matrix.<your-domain>` |
| CNAME | `dimension` | - | - | - | `matrix.<your-domain>` |
| CNAME | `jitsi` | - | - | - | `matrix.<your-domain>` |
| CNAME | `stats` | - | - | - | `matrix.<your-domain>` |
| CNAME | `goneb` | - | - | - | `matrix.<your-domain>` |
| CNAME | `sygnal` | - | - | - | `matrix.<your-domain>` |
| CNAME | `ntfy` | - | - | - | `matrix.<your-domain>` |
| CNAME | `hydrogen` | - | - | - | `matrix.<your-domain>` |
| CNAME | `cinny` | - | - | - | `matrix.<your-domain>` |
| CNAME | `buscarron` | - | - | - | `matrix.<your-domain>` |
| Used by component | Type | Host | Priority | Weight | Port | Target |
| ----------------------------------------------------------------------------------------------------------------------- | ----- | ------------------------------ | -------- | ------ | ---- | --------------------------- |
| [ma1sd](configuring-playbook-ma1sd.md) identity server | SRV | `_matrix-identity._tcp` | 10 | 0 | 443 | `matrix.<your-domain>` |
| [Dimension](configuring-playbook-dimension.md) integration server | CNAME | `dimension` | - | - | - | `matrix.<your-domain>` |
| [Jitsi](configuring-playbook-jitsi.md) video-conferencing platform | CNAME | `jitsi` | - | - | - | `matrix.<your-domain>` |
| [Prometheus/Grafana](configuring-playbook-prometheus-grafana.md) monitoring system | CNAME | `stats` | - | - | - | `matrix.<your-domain>` |
| [Go-NEB](configuring-playbook-bot-go-neb.md) bot | CNAME | `goneb` | - | - | - | `matrix.<your-domain>` |
| [Sygnal](configuring-playbook-sygnal.md) push notification gateway | CNAME | `sygnal` | - | - | - | `matrix.<your-domain>` |
| [ntfy](configuring-playbook-ntfy.md) push notifications server | CNAME | `ntfy` | - | - | - | `matrix.<your-domain>` |
| [Hydrogen](configuring-playbook-client-hydrogen.md) web client | CNAME | `hydrogen` | - | - | - | `matrix.<your-domain>` |
| [Cinny](configuring-playbook-client-cinny.md) web client | CNAME | `cinny` | - | - | - | `matrix.<your-domain>` |
| [Buscarron](configuring-playbook-bot-buscarron.md) helpdesk bot | CNAME | `buscarron` | - | - | - | `matrix.<your-domain>` |
| [Postmoogle](configuring-playbook-bot-postmoogle.md)/[Email2Matrix](configuring-playbook-email2matrix.md) email bridges | MX | `matrix` | 10 | 0 | - | `matrix.<your-domain>` |
| [Postmoogle](configuring-playbook-bot-postmoogle.md) email bridge | TXT | `matrix` | - | - | - | `v=spf1 ip4:<your-ip> -all` |
| [Postmoogle](configuring-playbook-bot-postmoogle.md) email bridge | TXT | `_dmarc.matrix` | - | - | - | `v=DMARC1; p=quarantine;` |
| [Postmoogle](configuring-playbook-bot-postmoogle.md) email bridge | TXT | `postmoogle._domainkey.matrix` | - | - | - | get it from `!pm dkim` |
## Subdomains setup
@ -77,3 +81,8 @@ This is an optional feature for the optionally-installed [ma1sd service](configu
Note: This `_matrix-identity._tcp` SRV record for the identity server is different from the `_matrix._tcp` that can be used for Synapse delegation. See [howto-server-delegation.md](howto-server-delegation.md) for more information about delegation.
When you're done with the DNS configuration and ready to proceed, continue with [Getting the playbook](getting-the-playbook.md).
## `_dmarc`, `postmoogle._domainkey` TXT and `matrix` MX records setup
To make the [postmoogle](configuring-playbook-bot-postmoogle.md) email bridge enable its email sending features, you need to configure
SPF (TXT), DMARC (TXT), DKIM (TXT) and MX records

View file

@ -56,7 +56,7 @@ ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
## Usage
To use the bot, create a **non-encrypted** room and invite `@bot.matrix-registration-bot:DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
To use the bot, create a **non-encrypted** room and invite `@bot.matrix-registration-bot:DOMAIN` (where `DOMAIN` is your base domain, not the `matrix.` domain).
In this room send `help` and the bot will reply with all options.

View file

@ -35,6 +35,9 @@ matrix_bot_postmoogle_enabled: true
matrix_bot_postmoogle_password: PASSWORD_FOR_THE_BOT
```
You will also need to add several DNS records so that postmoogle can send emails.
See [Configuring DNS](configuring-dns.md).
## Installing

View file

@ -2,6 +2,8 @@
The playbook can install and configure [matrix-appservice-kakaotalk](https://src.miscworks.net/fair/matrix-appservice-kakaotalk) for you. `matrix-appservice-kakaotalk` is a bridge to [Kakaotalk](https://www.kakaocorp.com/page/service/service/KakaoTalk?lang=ENG) based on [node-kakao](https://github.com/storycraft/node-kakao) (now unmaintained) and some [mautrix-facebook](https://github.com/mautrix/facebook) code.
**NOTE**: there have been recent reports (~2022-09-16) that **using this bridge may get your account banned**.
See the project's [documentation](https://src.miscworks.net/fair/matrix-appservice-kakaotalk) to learn what it does and why it might be useful to you.

View file

@ -0,0 +1,65 @@
# Setting up Cactus Comments (optional)
The playbook can install and configure [Cactus Comments](https://cactus.chat) for you.
Cactus Comments is a **federated comment system** built on Matrix. The role allows you to self-host the system.
It respects your privacy, and puts you in control.
See the project's [documentation](https://cactus.chat/docs/getting-started/introduction/) to learn what it
does and why it might be useful to you.
## Configuration
Add the following block to your `vars.yaml` and make sure to exchange the tokens to randomly generated values.
```yaml
#################
## Cactus Chat ##
#################
matrix_cactus_comments_enabled: true
# To allow guest comments without users needing to log in, you need to have guest registration enabled.
# To do this you need to uncomment one of the following lines (depending if you are using synapse or dentrite as a homeserver)
# If you don't know which one you use: The default is synapse ;)
# matrix_synapse_allow_guest_access: true
# matrix_dentrite_allow_guest_access
```
## Installing
After configuring the playbook, run the [installation](installing.md) command again:
```
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
```
## Usage
Upon starting Cactus Comments, a `bot.cactusbot` user account is created automatically.
To get started, send a `help` message to the `@bot.cactusbot:your-homeserver.com` bot to confirm it's working.
Then, register a site by typing: `register <sitename>`. You will then be invited into a moderation room.
Now you are good to go and can include the comment section on your website!
**Careful:** To really make use of self-hosting you need change a few things in comparison to the official docs!
Insert the following snippet into you page and make sure to replace `example.com` with your base domain!
```html
<script type="text/javascript" src="https://matrix.example.com/cactus-comments/cactus.js"></script>
<link rel="stylesheet" href="https://matrix.example.com/cactus-comments/style.css" type="text/css">
<div id="comment-section"></div>
<script>
initComments({
node: document.getElementById("comment-section"),
defaultHomeserverUrl: "https://matrix.example.com:8448",
serverName: "example.com",
siteName: "YourSiteName",
commentSectionId: "1"
})
</script>
```

View file

@ -1,6 +1,7 @@
# Setting up Email2Matrix (optional)
**Note**: email bridging can also happen via the [Postmoogle](configuring-playbook-bot-postmoogle.md) bot supported by the playbook.
Postmoogle is much more powerful and easier to use, so we recommend that you use it, instead of Email2Matrix.
The playbook can install and configure [email2matrix](https://github.com/devture/email2matrix) for you.
@ -9,6 +10,10 @@ See the project's [documentation](https://github.com/devture/email2matrix/blob/m
## Preparation
### DNS configuration
It's not strictly necessary, but you may increase the chances that incoming emails reach your server by adding an `MX` record for `matrix.DOMAIN`, as described in the [Configuring DNS](configuring-dns.md) documentation page.
### Port availability
Ensure that port 25 is available on your Matrix server and open in your firewall.

View file

@ -1,11 +1,14 @@
# Using your own webserver, instead of this playbook's nginx proxy (optional, advanced)
By default, this playbook installs its own nginx webserver (in a Docker container) which listens on ports 80 and 443.
By default, this playbook installs its own nginx webserver (called `matrix-nginx-proxy`, in a Docker container) which listens on ports 80 and 443.
If that's alright, you can skip this.
If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that,
and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can.
You should note, however, that the playbook's services work best when you keep using the integrated `matrix-nginx-proxy` webserver.
For example, disabling `matrix-nginx-proxy` when running a [Synapse worker setup for load-balancing](configuring-playbook-synapse.md#load-balancing-with-workers) (a more advanced, non-default configuration) is likely to cause various troubles (see [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090)). If you need a such more scalable setup, disabling `matrix-nginx-proxy` will be a bad idea. If yours will be a simple (default, non-worker-load-balancing) deployment, disabling `matrix-nginx-proxy` may be fine.
There are **2 ways you can go about it**, if you'd like to use your own webserver:
- [Method 1: Disabling the integrated nginx reverse-proxy webserver](#method-1-disabling-the-integrated-nginx-reverse-proxy-webserver)

View file

@ -94,7 +94,7 @@ Note : The playbook will hash the basic_auth password for you on setup. Thus, yo
### Collecting Synapse worker metrics to an external Prometheus server
If you are using workers (`matrix_synapse_workers_enabled: true`) and have enabled `matrix_synapse_metrics_proxying_enabled` as described above, the playbook will also automatically expose all Synapse worker threads' metrics to `https://matrix.DOMAIN/metrics/synapse/worker/TYPE-ID`, where `TYPE` corresponds to the type and `ID` to the instanceId of a worker as exemplified in `matrix_synapse_workers_enabled_list`.
If you are using workers (`matrix_synapse_workers_enabled: true`) and have enabled `matrix_synapse_metrics_proxying_enabled` as described above, the playbook will also automatically expose all Synapse worker threads' metrics to `https://matrix.DOMAIN/metrics/synapse/worker/ID`, where `ID` corresponds to the worker `id` as exemplified in `matrix_synapse_workers_enabled_list`.
The playbook also generates an exemplary config file (`/matrix/synapse/external_prometheus.yml.template`) with all the correct paths which you can copy to your Prometheus server and adapt to your needs. Make sure to edit the specified `password_file` path and contents and path to your `synapse-v2.rules`.
It will look a bit like this:
@ -111,8 +111,8 @@ scrape_configs:
labels:
job: "master"
index: 1
- job_name: 'synapse-generic_worker-1'
metrics_path: /metrics/synapse/worker/generic_worker-18111
- job_name: 'matrix-synapse-synapse-worker-generic-worker-0'
metrics_path: /metrics/synapse/worker/generic-worker-0
scheme: https
basic_auth:
username: prometheus

View file

@ -42,7 +42,7 @@ matrix_postgres_process_extra_arguments: [
]
```
If you're using the default setup (the `matrix-nginx-proxy` webserver being enabled) or you're using your own `nginx` server (which imports the configuration files generated by the playbook), you're good to go. If you use some other webserver, you may need to tweak your reverse-proxy setup manually to forward traffic to the various workers.
**NOTE**: Disabling `matrix-nginx-proxy` (`matrix_nginx_proxy_enabled: false`) (that is, [using your own other webserver](configuring-playbook-own-webserver.md) when running a Synapse worker setup is likely to cause various troubles (see [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090)).
In case any problems occur, make sure to have a look at the [list of synapse issues about workers](https://github.com/matrix-org/synapse/issues?q=workers+in%3Atitle) and your `journalctl --unit 'matrix-*'`.

View file

@ -179,3 +179,5 @@ When you're done with all the configuration you'd like to do, continue with [Ins
- [Setting up the Sygnal push gateway](configuring-playbook-sygnal.md) (optional)
- [Setting up the ntfy push notifications server](configuring-playbook-ntfy.md) (optional)
- [Setting up a Cactus Comments server](configuring-playbook-cactus-comments.md) - a federated comment system built on Matrix (optional)

View file

@ -117,3 +117,5 @@ These services are not part of our default installation, but can be enabled by [
- [matrixdotorg/sygnal](https://hub.docker.com/r/matrixdotorg/sygnal/) - [Sygnal](https://github.com/matrix-org/sygnal) is a reference Push Gateway for Matrix
- [binwiederhier/ntfy](https://hub.docker.com/r/binwiederhier/ntfy/) - [ntfy](https://ntfy.sh/) is a self-hosted, UnifiedPush-compatible push notifications server
- [cactuscomments/cactus-appservice](https://hub.docker.com/r/cactuscomments/cactus-appservice/) - [Cactus Comments](https://cactus.chat) a federated comment system built on Matrix

View file

@ -1210,6 +1210,9 @@ matrix_bot_buscarron_container_image_self_build: "{{ matrix_architecture not in
# We don't enable bots by default.
matrix_bot_postmoogle_enabled: false
matrix_bot_postmoogle_ssl_path: "{{ matrix_ssl_config_dir_path }}"
matrix_bot_postmoogle_tls_cert: "/ssl/live/{{ matrix_bot_postmoogle_domain }}/fullchain.pem"
matrix_bot_postmoogle_tls_key: "/ssl/live/{{ matrix_bot_postmoogle_domain }}/privkey.pem"
matrix_bot_postmoogle_systemd_required_services_list: |
{{
@ -1329,6 +1332,35 @@ matrix_backup_borg_systemd_required_services_list: |
# /matrix-backup-borg
#
######################################################################
######################################################################
#
# matrix-cactus-comments
#
######################################################################
matrix_cactus_comments_enabled: false
# Derive secret values from homeserver secret
matrix_cactus_comments_as_token: "{{ '%s' | format(matrix_homeserver_generic_secret_key) | password_hash('sha512', 'cactus.as.token') | to_uuid }}"
matrix_cactus_comments_hs_token: "{{ '%s' | format(matrix_homeserver_generic_secret_key) | password_hash('sha512', 'cactus.hs.token') | to_uuid }}"
matrix_cactus_comments_container_image_self_build: "{{ matrix_architecture not in ['amd64', 'arm32', 'arm64'] }}"
matrix_cactus_comments_systemd_required_services_list: |
{{
(['docker.service'])
+
(['matrix-nginx-proxy.service'] if matrix_nginx_proxy_enabled else [])
+
(['matrix-' + matrix_homeserver_implementation + '.service'])
}}
matrix_cactus_comments_client_nginx_path: "{{ '/cactus-comments/' if matrix_nginx_proxy_enabled else matrix_cactus_comments_client_path + '/' }}"
######################################################################
#
# /matrix-cactus-comments
#
######################################################################
######################################################################
#
@ -1728,14 +1760,20 @@ matrix_nginx_proxy_synapse_workers_enabled: "{{ matrix_synapse_workers_enabled }
matrix_nginx_proxy_synapse_workers_list: "{{ matrix_synapse_workers_enabled_list }}"
matrix_nginx_proxy_synapse_generic_worker_client_server_locations: "{{ matrix_synapse_workers_generic_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_generic_worker_federation_locations: "{{ matrix_synapse_workers_generic_worker_federation_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_media_repository_locations: "{{matrix_synapse_workers_media_repository_endpoints|default([]) }}"
matrix_nginx_proxy_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_endpoints|default([]) }}"
matrix_nginx_proxy_synapse_frontend_proxy_locations: "{{ matrix_synapse_workers_frontend_proxy_endpoints|default([]) }}"
matrix_nginx_proxy_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_worker_client_server_endpoints|default([]) }}"
matrix_nginx_proxy_systemd_wanted_services_list: |
{{
['matrix-' + matrix_homeserver_implementation + '.service']
+
(matrix_synapse_webserving_workers_systemd_services_list if matrix_homeserver_implementation == 'synapse' and matrix_synapse_workers_enabled else [])
+
(['matrix-corporal.service'] if matrix_corporal_enabled else [])
+
(['matrix-ma1sd.service'] if matrix_ma1sd_enabled else [])
@ -1791,6 +1829,8 @@ matrix_ssl_domains_to_obtain_certificates_for: |
+
([matrix_server_fqn_ntfy] if matrix_ntfy_enabled else [])
+
([matrix_bot_postmoogle_domain] if matrix_bot_postmoogle_enabled else [])
+
([matrix_domain] if matrix_nginx_proxy_base_domain_serving_enabled else [])
+
matrix_ssl_additional_domains_to_obtain_certificates_for

View file

@ -9,7 +9,7 @@ matrix_bot_honoroit_docker_repo: "https://gitlab.com/etke.cc/honoroit.git"
matrix_bot_honoroit_docker_repo_version: "{{ matrix_bot_honoroit_version }}"
matrix_bot_honoroit_docker_src_files_path: "{{ matrix_base_data_path }}/honoroit/docker-src"
matrix_bot_honoroit_version: v0.9.13
matrix_bot_honoroit_version: v0.9.14
matrix_bot_honoroit_docker_image: "{{ matrix_bot_honoroit_docker_image_name_prefix }}honoroit:{{ matrix_bot_honoroit_version }}"
matrix_bot_honoroit_docker_image_name_prefix: "{{ 'localhost/' if matrix_bot_honoroit_container_image_self_build else 'registry.gitlab.com/etke.cc/' }}"
matrix_bot_honoroit_docker_image_force_pull: "{{ matrix_bot_honoroit_docker_image.endswith(':latest') }}"
@ -88,6 +88,17 @@ matrix_bot_honoroit_loglevel: ''
# Disable encryption
matrix_bot_honoroit_noencryption: false
# A list of whitelisted users allowed to use/invite honoroit
# If not defined, everyone is allowed.
# Example set of rules:
# matrix_bot_honoroit_allowedusers:
# - @someone:example.com
# - @another:example.com
# - @bot.*:example.com
# - @*:another.com
matrix_bot_honoroit_allowedusers:
- "@*:*"
# Max items in cache
matrix_bot_honoroit_cachesize: ''

View file

@ -11,6 +11,7 @@ HONOROIT_CACHESIZE={{ matrix_bot_honoroit_cachesize }}
HONOROIT_NOENCRYPTION={{ matrix_bot_honoroit_noencryption }}
HONOROIT_IGNORENOTHREAD={{ matrix_bot_honoroit_ignorenothread }}
HONOROIT_IGNOREDROOMS={{ matrix_bot_honoroit_ignoredrooms | join(' ') }}
HONOROIT_ALLOWEDUSERS={{ matrix_bot_honoroit_allowedusers | join(' ') }}
HONOROIT_TEXT_PREFIX_OPEN={{ matrix_bot_honoroit_text_prefix_open }}
HONOROIT_TEXT_PREFIX_DONE={{ matrix_bot_honoroit_text_prefix_done }}
HONOROIT_TEXT_NOENCRYPTION={{ matrix_bot_honoroit_text_noencryption }}

View file

@ -9,7 +9,7 @@ matrix_bot_postmoogle_docker_repo: "https://gitlab.com/etke.cc/postmoogle.git"
matrix_bot_postmoogle_docker_repo_version: "{{ 'main' if matrix_bot_postmoogle_version == 'latest' else matrix_bot_postmoogle_version }}"
matrix_bot_postmoogle_docker_src_files_path: "{{ matrix_base_data_path }}/postmoogle/docker-src"
matrix_bot_postmoogle_version: v0.9.0
matrix_bot_postmoogle_version: v0.9.2
matrix_bot_postmoogle_docker_image: "{{ matrix_bot_postmoogle_docker_image_name_prefix }}postmoogle:{{ matrix_bot_postmoogle_version }}"
matrix_bot_postmoogle_docker_image_name_prefix: "{{ 'localhost/' if matrix_bot_postmoogle_container_image_self_build else 'registry.gitlab.com/etke.cc/' }}"
matrix_bot_postmoogle_docker_image_force_pull: "{{ matrix_bot_postmoogle_docker_image.endswith(':latest') }}"
@ -110,11 +110,35 @@ matrix_bot_postmoogle_noencryption: false
matrix_bot_postmoogle_domain: "{{ matrix_server_fqn_matrix }}"
# in-container port
# in-container ports
matrix_bot_postmoogle_port: '2525'
matrix_bot_postmoogle_tls_port: '25587'
# on-host port
# on-host ports
matrix_bot_postmoogle_smtp_host_bind_port: '25'
matrix_bot_postmoogle_submission_host_bind_port: '587'
### SSL
## on-host SSL dir
matrix_bot_postmoogle_ssl_path: ""
## in-container SSL paths
# matrix_bot_postmoogle_tls_cert is the SSL certificate's certificate.
# This is likely set via group_vars/matrix_servers, so you don't need to set it.
# If you do need to set it manually, note that this is an in-container path.
# To mount a certificates volumes into the container, use matrix_bot_postmoogle_ssl_path
# Example value: /ssl/live/{{ matrix_bot_postmoogle_domain }}/fullchain.pem
matrix_bot_postmoogle_tls_cert: ""
# matrix_bot_postmoogle_tls_key is the SSL certificate's key.
# This is likely set via group_vars/matrix_servers, so you don't need to set it.
# If you do need to set it manually, note that this is an in-container path.
# To mount a certificates volumes into the container, use matrix_bot_postmoogle_ssl_path
# Example value: /ssl/live/{{ matrix_bot_postmoogle_domain }}/privkey.pem
matrix_bot_postmoogle_tls_key: ""
# Mandatory TLS, even on plain SMTP port
matrix_bot_postmoogle_tls_required: false
# Additional environment variables to pass to the postmoogle container
#

View file

@ -10,7 +10,10 @@ POSTMOOGLE_MAXSIZE={{ matrix_bot_postmoogle_maxsize }}
POSTMOOGLE_SENTRY={{ matrix_bot_postmoogle_sentry }}
POSTMOOGLE_LOGLEVEL={{ matrix_bot_postmoogle_loglevel }}
POSTMOOGLE_NOENCRYPTION={{ matrix_bot_postmoogle_noencryption }}
POSTMOOGLE_USERS={{ matrix_bot_postmoogle_users | join(' ') }}
POSTMOOGLE_ADMINS={{ matrix_bot_postmoogle_admins | join(' ') }}
POSTMOOGLE_TLS_PORT={{ matrix_bot_postmoogle_tls_port }}
POSTMOOGLE_TLS_CERT={{ matrix_bot_postmoogle_tls_cert }}
POSTMOOGLE_TLS_KEY={{ matrix_bot_postmoogle_tls_key }}
POSTMOOGLE_TLS_REQUIRED={{ matrix_bot_postmoogle_tls_required }}
{{ matrix_bot_postmoogle_environment_variables_extension }}

View file

@ -24,7 +24,13 @@ ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-bot-postmoogle
--network={{ matrix_docker_network }} \
--env-file={{ matrix_bot_postmoogle_config_path }}/env \
-p {{ matrix_bot_postmoogle_smtp_host_bind_port }}:{{ matrix_bot_postmoogle_port }} \
{% if matrix_bot_postmoogle_ssl_path %}
-p {{ matrix_bot_postmoogle_submission_host_bind_port }}:{{ matrix_bot_postmoogle_tls_port }} \
{% endif %}
--mount type=bind,src={{ matrix_bot_postmoogle_data_path }},dst=/data \
{% if matrix_bot_postmoogle_ssl_path %}
--mount type=bind,src={{ matrix_bot_postmoogle_ssl_path }},dst=/ssl \
{% endif %}
{% for arg in matrix_bot_postmoogle_container_extra_arguments %}
{{ arg }} \
{% endfor %}

View file

@ -11,10 +11,11 @@ matrix_appservice_irc_docker_src_files_path: "{{ matrix_base_data_path }}/appser
# matrix_appservice_irc_version used to contain the full Docker image tag (e.g. `release-X.X.X`).
# It's a bare version number now. We try to somewhat retain compatibility below.
matrix_appservice_irc_version: 0.34.0
matrix_appservice_irc_version: 0.35.0
matrix_appservice_irc_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/matrix-appservice-irc:{{ matrix_appservice_irc_docker_image_tag }}"
matrix_appservice_irc_docker_image_tag: "{{ 'latest' if matrix_appservice_irc_version == 'latest' else ('release-' + matrix_appservice_irc_version) }}"
matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_image.endswith(':latest') }}"
matrix_appservice_irc_docker_image_name_prefix: "{{ 'localhost/' if matrix_appservice_irc_container_image_self_build else matrix_container_global_registry_prefix }}"
matrix_appservice_irc_base_path: "{{ matrix_base_data_path }}/appservice-irc"
matrix_appservice_irc_config_path: "{{ matrix_appservice_irc_base_path }}/config"

View file

@ -10,7 +10,7 @@ matrix_hookshot_container_image_self_build: false
matrix_hookshot_container_image_self_build_repo: "https://github.com/matrix-org/matrix-hookshot.git"
matrix_hookshot_container_image_self_build_branch: "{{ 'main' if matrix_hookshot_version == 'latest' else matrix_hookshot_version }}"
matrix_hookshot_version: 2.1.2
matrix_hookshot_version: 2.2.0
matrix_hookshot_docker_image: "{{ matrix_hookshot_docker_image_name_prefix }}halfshot/matrix-hookshot:{{ matrix_hookshot_version }}"
matrix_hookshot_docker_image_name_prefix: "{{ 'localhost/' if matrix_hookshot_container_image_self_build else matrix_container_global_registry_prefix }}"

View file

@ -9,7 +9,7 @@ matrix_mautrix_signal_docker_repo: "https://mau.dev/mautrix/signal.git"
matrix_mautrix_signal_docker_repo_version: "{{ 'master' if matrix_mautrix_signal_version == 'latest' else matrix_mautrix_signal_version }}"
matrix_mautrix_signal_docker_src_files_path: "{{ matrix_base_data_path }}/mautrix-signal/docker-src"
matrix_mautrix_signal_version: v0.3.0
matrix_mautrix_signal_version: v0.4.0
matrix_mautrix_signal_daemon_version: 0.21.1
# See: https://mau.dev/mautrix/signal/container_registry
matrix_mautrix_signal_docker_image: "dock.mau.dev/mautrix/signal:{{ matrix_mautrix_signal_version }}"

View file

@ -8,7 +8,7 @@ matrix_mautrix_whatsapp_container_image_self_build: false
matrix_mautrix_whatsapp_container_image_self_build_repo: "https://mau.dev/mautrix/whatsapp.git"
matrix_mautrix_whatsapp_container_image_self_build_branch: "{{ 'master' if matrix_mautrix_whatsapp_version == 'latest' else matrix_mautrix_whatsapp_version }}"
matrix_mautrix_whatsapp_version: v0.6.1
matrix_mautrix_whatsapp_version: v0.7.0
# See: https://mau.dev/mautrix/whatsapp/container_registry
matrix_mautrix_whatsapp_docker_image: "{{ matrix_mautrix_whatsapp_docker_image_name_prefix }}mautrix/whatsapp:{{ matrix_mautrix_whatsapp_version }}"
matrix_mautrix_whatsapp_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_whatsapp_container_image_self_build else 'dock.mau.dev/' }}"
@ -86,10 +86,6 @@ matrix_mautrix_whatsapp_login_shared_secret: ''
matrix_mautrix_whatsapp_bridge_login_shared_secret_map:
"{{ {matrix_mautrix_whatsapp_homeserver_domain: matrix_mautrix_whatsapp_login_shared_secret} if matrix_mautrix_whatsapp_login_shared_secret else {} }}"
# Servers to always allow double puppeting from
matrix_mautrix_whatsapp_bridge_double_puppet_server_map:
"{{ matrix_mautrix_whatsapp_homeserver_domain : matrix_mautrix_whatsapp_homeserver_address }}"
# Enable End-to-bridge encryption
matrix_mautrix_whatsapp_bridge_encryption_allow: false
matrix_mautrix_whatsapp_bridge_encryption_default: "{{ matrix_mautrix_whatsapp_bridge_encryption_allow }}"

View file

@ -5,6 +5,9 @@ homeserver:
address: {{ matrix_mautrix_whatsapp_homeserver_address }}
# The domain of the homeserver (for MXIDs, etc).
domain: {{ matrix_mautrix_whatsapp_homeserver_domain }}
# What software is the homeserver running?
# Standard Matrix homeservers like Synapse, Dendrite and Conduit should just use "standard" here.
software: standard
# The URL to push real-time bridge status to.
# If set, the bridge will make POST requests to this URL whenever a user's whatsapp connection state changes.
# The bridge will use the appservice as_token to authorize requests.
@ -52,7 +55,7 @@ appservice:
# Whether or not to receive ephemeral events via appservice transactions.
# Requires MSC2409 support (i.e. Synapse 1.22+).
# You should disable bridge -> sync_with_custom_puppets when this is enabled.
ephemeral_events: false
ephemeral_events: true
# Authentication tokens for AS <-> HS communication. Autogenerated; do not modify.
as_token: "{{ matrix_mautrix_whatsapp_appservice_token }}"
@ -188,7 +191,7 @@ bridge:
# Should Matrix users leaving groups be bridged to WhatsApp?
bridge_matrix_leave: true
# Should the bridge sync with double puppeting to receive EDUs that aren't normally sent to appservices.
sync_with_custom_puppets: true
sync_with_custom_puppets: false
# Should the bridge update the m.direct account data event when double puppeting is enabled.
# Note that updating the m.direct event is not atomic (except with mautrix-asmux)
# and is therefore prone to race conditions.
@ -268,6 +271,9 @@ bridge:
# Should the bridge never send alerts to the bridge management room?
# These are mostly things like the user being logged out.
disable_bridge_alerts: false
# Should the bridge stop if the WhatsApp server says another user connected with the same session?
# This is only safe on single-user bridges.
crash_on_stream_replaced: false
# Should the bridge detect URLs in outgoing messages, ask the homeserver to generate a preview,
# and send it to WhatsApp? URL previews can always be sent using the `com.beeper.linkpreviews`
# key in the event content even if this is disabled.
@ -311,6 +317,8 @@ bridge:
# This will cause the bridge bot to be in private chats for the encryption to work properly.
# It is recommended to also set private_chat_portal_meta to true when using this.
default: {{ matrix_mautrix_whatsapp_bridge_encryption_default|to_json }}
# Whether to use MSC2409/MSC3202 instead of /sync long polling for receiving encryption-related data.
appservice: false
# Require encryption, drop any unencrypted messages.
require: false
# Enable key sharing? If enabled, key requests for rooms where users are in will be fulfilled.

View file

@ -0,0 +1,60 @@
---
# Cactus Comments is a federated comment system built on Matrix
# Project source code URL: https://gitlab.com/cactus-comments/cactus-appservice
# Project source code URL: https://gitlab.com/cactus-comments/cactus-client
matrix_cactus_comments_enabled: true
matrix_cactus_comments_serve_client_enabled: true
matrix_cactus_comments_container_image_self_build: false
matrix_cactus_comments_docker_repo: "https://gitlab.com/cactus-comments/cactus-appservice.git"
matrix_cactus_comments_docker_repo_version: "{{ matrix_cactus_comments_version if matrix_cactus_comments_version != 'latest' else 'main' }}"
matrix_cactus_comments_docker_src_files_path: "{{ matrix_cactus_comments_base_path }}/docker-src"
matrix_cactus_comments_base_path: "{{ matrix_base_data_path }}/cactus-comments"
matrix_cactus_comments_container_tmp_path: "{{ matrix_cactus_comments_base_path }}/tmp"
matrix_cactus_comments_client_path: "{{ matrix_cactus_comments_base_path }}/client"
matrix_cactus_comments_client_file_permissions: "0644"
matrix_cactus_comments_app_service_config_file: "{{ matrix_cactus_comments_base_path }}/cactus_appservice.yaml"
matrix_cactus_comments_app_service_env_file: "{{ matrix_cactus_comments_base_path }}/cactus.env"
matrix_cactus_comments_as_token: ''
matrix_cactus_comments_hs_token: ''
matrix_cactus_comments_homeserver_url: "{{ matrix_homeserver_container_url }}"
matrix_cactus_comments_user_id: "bot.cactusbot"
matrix_cactus_comments_tmp_directory_size_mb: 1
matrix_cactus_comments_container_port: 5000
matrix_cactus_comments_version: 0.9.0
matrix_cactus_comments_docker_image: "{{ matrix_container_global_registry_prefix }}cactuscomments/cactus-appservice:{{ matrix_cactus_comments_version }}"
matrix_cactus_comments_docker_image_force_pull: "{{ matrix_cactus_comments_docker_image.endswith(':latest') }}"
# matrix_cactus_comments_client_version specifies the version of the cactus-client release to use.
# For available versions, see: https://gitlab.com/cactus-comments/cactus-client/-/releases
# Also see: `matrix_cactus_comments_client_local_dir`
matrix_cactus_comments_client_version: "0.13.0"
# matrix_cactus_comments_client_local_dir specifies a local directory (on the Ansible controller, not on the remote server) with cactus-client files to use.
# This is an alternative to `matrix_cactus_comments_client_version`, to be used when you'd like to
# provide the files locally / manually.
matrix_cactus_comments_client_local_dir: ''
# matrix_cactus_comments_client_nginx_path specifies the path where nginx can access the client files.
# The default value assumes a container setup. If you're running nginx without a container, consider adjusting this path
matrix_cactus_comments_client_nginx_path: "/cactus-comments/"
# matrix_cactus_comments_client_endpoint specifies where nginx will serve the files in nginx is enabled
matrix_cactus_comments_client_endpoint: "/cactus-comments/"
# List of systemd services that matrix-cactus-comments.service depends on
matrix_bot_cactus_comments_systemd_required_services_list: ['docker.service']
# List of systemd services that matrix-cactus-comments.service wants
matrix_bot_cactus_comments_systemd_wanted_services_list: []
# A list of extra arguments to pass to the container
matrix_cactus_comments_container_extra_arguments: []
matrix_cactus_comments_environment_variables_extension: ''

View file

@ -0,0 +1,69 @@
---
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-cactus-comments.service'] }}"
when: matrix_cactus_comments_enabled | bool
# If the matrix-synapse role is not used, these variables may not exist.
- ansible.builtin.set_fact:
matrix_homeserver_container_runtime_injected_arguments: >
{{
matrix_homeserver_container_runtime_injected_arguments | default([])
+
["--mount type=bind,src={{ matrix_cactus_comments_app_service_config_file }},dst=/matrix-cactus-comments.yaml,ro"]
}}
matrix_homeserver_app_service_runtime_injected_config_files: >
{{
matrix_homeserver_app_service_runtime_injected_config_files | default([])
+
["/matrix-cactus-comments.yaml"]
}}
when: matrix_cactus_comments_enabled | bool
- block:
- name: Fail if matrix-nginx-proxy role already executed
ansible.builtin.fail:
msg: >-
Trying to append Cactus Comment's reverse-proxying configuration to matrix-nginx-proxy,
but it's pointless since the matrix-nginx-proxy role had already executed.
To fix this, please change the order of roles in your playbook,
so that the matrix-nginx-proxy role would run after the matrix-cactus-comments role.
when: matrix_nginx_proxy_role_executed | default(False) | bool
- name: Mount volume
ansible.builtin.set_fact:
matrix_nginx_proxy_container_additional_volumes: >
{{
matrix_nginx_proxy_container_additional_volumes | default([])
+
[{"src": "{{ matrix_cactus_comments_client_path }}", "dst": "/cactus-comments/cactus-comments", "options": "ro"}]
}}
- name: Generate Cactus Comment proxying configuration for matrix-nginx-proxy
ansible.builtin.set_fact:
matrix_cactus_comments_nginx_proxy_configuration: |
location {{ matrix_cactus_comments_client_endpoint }} {
root {{ matrix_cactus_comments_client_nginx_path }};
}
- name: Register Cactus Comment proxying configuration with matrix-nginx-proxy
ansible.builtin.set_fact:
matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: |
{{
matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks | default([])
+
[matrix_cactus_comments_nginx_proxy_configuration]
}}
- name: Warn about reverse-proxying if matrix-nginx-proxy not used
ansible.builtin.debug:
msg: >-
NOTE: You've enabled Cactus Comments but are not using the matrix-nginx-proxy
reverse proxy.
Please make sure that you're proxying client files in {{ matrix_cactus_comments_client_path }} correctly
when: "not matrix_nginx_proxy_enabled | default(False) | bool"
tags:
- always
when: matrix_cactus_comments_enabled | bool and matrix_cactus_comments_serve_client_enabled | bool

View file

@ -0,0 +1,23 @@
---
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/init.yml"
tags:
- always
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/validate_config.yml"
when: "run_setup | bool and matrix_cactus_comments_enabled | bool"
tags:
- setup-all
- setup-cactus-comments
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/setup_install.yml"
when: "run_setup | bool and matrix_cactus_comments_enabled | bool"
tags:
- setup-all
- setup-cactus-comments
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
when: "run_setup | bool and not matrix_cactus_comments_enabled | bool"
tags:
- setup-all
- setup-cactus-comments

View file

@ -0,0 +1,138 @@
---
- name: Ensure cactus comments paths exist
ansible.builtin.file:
path: "{{ item.path }}"
state: directory
mode: 0750
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
with_items:
- {path: "{{ matrix_cactus_comments_base_path }}", when: true}
- {path: "{{ matrix_cactus_comments_client_path }}", when: true}
- {path: "{{ matrix_cactus_comments_container_tmp_path }}", when: true}
- {path: "{{ matrix_cactus_comments_docker_src_files_path }}", when: matrix_cactus_comments_container_image_self_build}
when: "item.when | bool"
- name: Ensure cactus comments environment file created
ansible.builtin.template:
src: "{{ role_path }}/templates/env.j2"
dest: "{{ matrix_cactus_comments_app_service_env_file }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
mode: 0640
- name: Ensure cactus comments appservice file created
ansible.builtin.template:
src: "{{ role_path }}/templates/cactus_appservice.yaml.j2"
dest: "{{ matrix_cactus_comments_app_service_config_file }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
mode: 0640
- name: Ensure cactus comments image is pulled
docker_image:
name: "{{ matrix_cactus_comments_docker_image }}"
source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
force_source: "{{ matrix_cactus_comments_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_cactus_comments_docker_image_force_pull }}"
when: "not matrix_cactus_comments_container_image_self_build | bool"
register: result
retries: "{{ matrix_container_retries_count }}"
delay: "{{ matrix_container_retries_delay }}"
until: result is not failed
- name: Ensure cactus comments repository is present on self-build
ansible.builtin.git:
repo: "{{ matrix_cactus_comments_docker_repo }}"
version: "{{ matrix_cactus_comments_docker_repo_version }}"
dest: "{{ matrix_cactus_comments_docker_src_files_path }}"
force: "yes"
become: true
become_user: "{{ matrix_user_username }}"
register: matrix_cactus_comments_git_pull_results
when: "matrix_cactus_comments_container_image_self_build | bool"
- name: Ensure cactus comments image is built
docker_image:
name: "{{ matrix_cactus_comments_docker_image }}"
source: build
force_source: "{{ matrix_cactus_comments_git_pull_results.changed if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mailer_git_pull_results.changed }}"
build:
dockerfile: Dockerfile
path: "{{ matrix_cactus_comments_docker_src_files_path }}"
pull: true
when: "matrix_cactus_comments_container_image_self_build | bool"
- block:
- name: Download client binary to local folder
ansible.builtin.get_url:
url: "https://gitlab.com/cactus-comments/cactus-client/-/archive/v{{ matrix_cactus_comments_client_version }}/cactus-client-v{{ matrix_cactus_comments_client_version }}.tar.gz"
dest: "/tmp/cactus-comments-{{ matrix_cactus_comments_client_version }}.tar.gz"
mode: '0644'
register: _download_client
until: _download_client is succeeded
retries: 5
delay: 2
check_mode: false
- name: Unpack client
ansible.builtin.unarchive:
src: "/tmp/cactus-comments-{{ matrix_cactus_comments_client_version }}.tar.gz"
dest: "/tmp/"
remote_src: true
mode: 0600
check_mode: false
- name: Propagate client javascript file
ansible.builtin.copy:
src: "/tmp/cactus-client-v{{ matrix_cactus_comments_client_version }}/src/cactus.js"
remote_src: true
dest: "{{ matrix_cactus_comments_client_path }}/cactus.js"
mode: "{{ matrix_cactus_comments_client_file_permissions }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
- name: Propagate client style file
ansible.builtin.copy:
src: "/tmp/cactus-client-v{{ matrix_cactus_comments_client_version }}/src/style.css"
remote_src: true
dest: "{{ matrix_cactus_comments_client_path }}/style.css"
mode: "{{ matrix_cactus_comments_client_file_permissions }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
when: matrix_cactus_comments_client_local_dir | length == 0
- block:
- name: Propagate locally distributed client javascreipt
ansible.builtin.copy:
src: "{{ matrix_cactus_comments_client_local_dir }}/src/cactus.js"
dest: "{{ matrix_cactus_comments_client_path }}/cactus.js"
mode: "{{ matrix_cactus_comments_client_file_permissions }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
- name: Propagate locally distributed client style.css
ansible.builtin.copy:
src: "{{ matrix_cactus_comments_client_local_dir }}/src/style.css"
dest: "{{ matrix_cactus_comments_client_path }}/style.css"
mode: "{{ matrix_cactus_comments_client_file_permissions }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
when: matrix_cactus_comments_client_local_dir | length > 0
- name: Ensure matrix-cactus-comments.service installed
ansible.builtin.template:
src: "{{ role_path }}/templates/systemd/matrix-cactus-comments.service.j2"
dest: "{{ matrix_systemd_path }}/matrix-cactus-comments.service"
mode: 0644
register: matrix_cactus_comments_systemd_service_result
- name: Ensure systemd reloaded after matrix-cactus-comments.service installation
ansible.builtin.service:
daemon_reload: true
when: "matrix_cactus_comments_systemd_service_result.changed | bool"
- name: Ensure matrix-cactus-comments.service restarted, if necessary
ansible.builtin.service:
name: "matrix-cactus-comments.service"
state: restarted

View file

@ -0,0 +1,36 @@
---
- name: Check existence of matrix-cactus-comments service
ansible.builtin.stat:
path: "{{ matrix_systemd_path }}/matrix-cactus-comments.service"
register: matrix_cactus_comments_service_stat
- name: Ensure cactus comments is stopped
ansible.builtin.service:
name: matrix-cactus-comments
state: stopped
enabled: false
daemon_reload: true
register: stopping_result
when: "matrix_cactus_comments_service_stat.stat.exists | bool"
- name: Ensure matrix-cactus-comments.service doesn't exist
ansible.builtin.file:
path: "{{ matrix_systemd_path }}/matrix-cactus-comments.service"
state: absent
when: "matrix_cactus_comments_service_stat.stat.exists | bool"
- name: Ensure systemd reloaded after matrix-cactus-comments.service removal
ansible.builtin.service:
daemon_reload: true
when: "matrix_cactus_comments_service_stat.stat.exists | bool"
- name: Ensure Matrix cactus comments paths don't exist
ansible.builtin.file:
path: "{{ matrix_cactus_comments_base_path }}"
state: absent
- name: Ensure cactus comments Docker image doesn't exist
docker_image:
name: "{{ matrix_cactus_comments_docker_image }}"
state: absent

View file

@ -0,0 +1,10 @@
---
- name: Fail if required settings not defined
ansible.builtin.fail:
msg: >-
You need to define a required configuration setting (`{{ item }}`).
when: "vars[item] == ''"
with_items:
- "matrix_cactus_comments_as_token"
- "matrix_cactus_comments_hs_token"

View file

@ -0,0 +1,19 @@
# A unique, user-defined ID of the application service which will never change.
id: "Cactus Comments"
# Where the cactus-appservice is hosted:
url: "http://matrix-cactus-comments:{{ matrix_cactus_comments_container_port }}"
# Unique tokens used to authenticate requests between our service and the
# homeserver (and the other way). Use the sha256 hashes of something random.
# CHANGE THESE VALUES.
as_token: {{ matrix_cactus_comments_as_token | to_json }}
hs_token: {{ matrix_cactus_comments_hs_token | to_json }}
# The user id of the cactusbot which can be used to register and moderate sites
sender_localpart: "{{ matrix_cactus_comments_user_id }}"
namespaces:
aliases:
- exclusive: true
regex: "#comments_.*"

View file

@ -0,0 +1,6 @@
CACTUS_HS_TOKEN={{ matrix_cactus_comments_hs_token }}
CACTUS_AS_TOKEN={{ matrix_cactus_comments_as_token }}
CACTUS_HOMESERVER_URL={{ matrix_cactus_comments_homeserver_url }}
CACTUS_USER_ID=@{{ matrix_cactus_comments_user_id }}:{{ matrix_domain }}
{{ matrix_cactus_comments_environment_variables_extension }}

View file

@ -0,0 +1,36 @@
#jinja2: lstrip_blocks: "True"
[Unit]
Description=Cactus Comments
{% for service in matrix_bot_cactus_comments_systemd_required_services_list %}
Requires={{ service }}
After={{ service }}
{% endfor %}
{% for service in matrix_bot_cactus_comments_systemd_wanted_services_list %}
Wants={{ service }}
{% endfor %}
DefaultDependencies=no
[Service]
Type=simple
Environment="HOME={{ matrix_systemd_unit_home_path }}"
ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill matrix-cactus-comments 2>/dev/null || true'
ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-cactus-comments 2>/dev/null || true'
ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-cactus-comments \
--log-driver=none \
--cap-drop=ALL \
--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
--read-only \
--env-file {{ matrix_cactus_comments_app_service_env_file }} \
--tmpfs=/tmp:rw,noexec,nosuid,size={{ matrix_cactus_comments_tmp_directory_size_mb }}m \
--network={{ matrix_docker_network }} \
{{ matrix_cactus_comments_docker_image }}
ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill matrix-cactus-comments 2>/dev/null || true'
ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-cactus-comments 2>/dev/null || true'
Restart=always
RestartSec=30
SyslogIdentifier=matrix-cactus-comments
[Install]
WantedBy=multi-user.target

View file

@ -6,7 +6,7 @@ matrix_client_cinny_enabled: true
matrix_client_cinny_container_image_self_build: false
matrix_client_cinny_container_image_self_build_repo: "https://github.com/ajbura/cinny.git"
matrix_client_cinny_version: v2.1.3
matrix_client_cinny_version: v2.2.0
matrix_client_cinny_docker_image: "{{ matrix_client_cinny_docker_image_name_prefix }}ajbura/cinny:{{ matrix_client_cinny_version }}"
matrix_client_cinny_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_cinny_container_image_self_build else matrix_container_global_registry_prefix }}"
matrix_client_cinny_docker_image_force_pull: "{{ matrix_client_cinny_docker_image.endswith(':latest') }}"

View file

@ -10,7 +10,7 @@ matrix_client_element_container_image_self_build_repo: "https://github.com/vecto
# - https://github.com/vector-im/element-web/issues/19544
matrix_client_element_container_image_self_build_low_memory_system_patch_enabled: "{{ ansible_memtotal_mb < 4096 }}"
matrix_client_element_version: v1.11.4
matrix_client_element_version: v1.11.5
matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"

View file

@ -38,7 +38,7 @@ matrix_conduit_max_request_size: 20_000_000
# Maximum number of open files for Conduit's embedded RocksDB database
# See https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide#tuning-other-options
# If not specified, Conduit defaults to a relatively low value of 20
# By default, Conduit uses a relatively low value of 20.
matrix_conduit_rocksdb_max_open_files: 64
# Enables registration. If set to false, no users can register on this server.

View file

@ -8,7 +8,7 @@ matrix_coturn_container_image_self_build_repo: "https://github.com/coturn/coturn
matrix_coturn_container_image_self_build_repo_version: "docker/{{ matrix_coturn_version }}"
matrix_coturn_container_image_self_build_repo_dockerfile_path: "docker/coturn/alpine/Dockerfile"
matrix_coturn_version: 4.5.2-r14
matrix_coturn_version: 4.6.0-r0
matrix_coturn_docker_image: "{{ matrix_coturn_docker_image_name_prefix }}coturn/coturn:{{ matrix_coturn_version }}-alpine"
matrix_coturn_docker_image_name_prefix: "{{ 'localhost/' if matrix_coturn_container_image_self_build else matrix_container_global_registry_prefix }}"
matrix_coturn_docker_image_force_pull: "{{ matrix_coturn_docker_image.endswith(':latest') }}"

View file

@ -6,7 +6,7 @@ matrix_dendrite_enabled: true
matrix_dendrite_docker_image: "{{ matrix_dendrite_docker_image_name_prefix }}matrixdotorg/dendrite-monolith:{{ matrix_dendrite_docker_image_tag }}"
matrix_dendrite_docker_image_name_prefix: "docker.io/"
matrix_dendrite_docker_image_tag: "v0.9.6"
matrix_dendrite_docker_image_tag: "v0.9.8"
matrix_dendrite_docker_image_force_pull: "{{ matrix_dendrite_docker_image.endswith(':latest') }}"
matrix_dendrite_base_path: "{{ matrix_base_data_path }}/dendrite"
@ -90,7 +90,7 @@ matrix_dendrite_tmp_directory_size_mb: 500
# Rate limits
matrix_dendrite_rate_limiting_enabled: true
matrix_dendrite_rate_limiting_threshold: 5
matrix_dendrite_rate_limiting_threshold: 20
matrix_dendrite_rate_limiting_cooloff_ms: 500
# Controls whether people with access to the homeserver can register by themselves.

View file

@ -58,6 +58,10 @@ global:
# e.g. localhost:443
well_known_server_name: ""
# The server name to delegate client-server communications to, with optional port
# e.g. localhost:443
well_known_client_name: ""
# Lists of domains that the server will trust as identity servers to verify third
# party identifiers such as phone numbers and email addresses.
trusted_third_party_id_servers: {{ matrix_dendrite_trusted_id_servers|to_json }}
@ -73,6 +77,25 @@ global:
# Whether outbound presence events are allowed, e.g. sending presence events to other servers
enable_outbound: false
# Configuration for in-memory caches. Caches can often improve performance by
# keeping frequently accessed items (like events, identifiers etc.) in memory
# rather than having to read them from the database.
cache:
# The estimated maximum size for the global cache in bytes, or in terabytes,
# gigabytes, megabytes or kilobytes when the appropriate 'tb', 'gb', 'mb' or
# 'kb' suffix is specified. Note that this is not a hard limit, nor is it a
# memory limit for the entire process. A cache that is too small may ultimately
# provide little or no benefit.
max_size_estimated: 1gb
# The maximum amount of time that a cache entry can live for in memory before
# it will be evicted and/or refreshed from the database. Lower values result in
# easier admission of new cache entries but may also increase database load in
# comparison to higher values, so adjust conservatively. Higher values may make
# it harder for new items to make it into the cache, e.g. if new rooms suddenly
# become popular.
max_age: 1h
# Server notices allows server admins to send messages to all users.
server_notices:
enabled: false
@ -186,6 +209,8 @@ client_api:
enabled: {{ matrix_dendrite_rate_limiting_enabled|to_json }}
threshold: {{ matrix_dendrite_rate_limiting_threshold|to_json }}
cooloff_ms: {{ matrix_dendrite_rate_limiting_cooloff_ms|to_json }}
exempt_user_ids:
# - "@user:domain.com"
# Configuration for the Federation API.
federation_api:
@ -324,6 +349,10 @@ sync_api:
# a reverse proxy server.
# real_ip_header: X-Real-IP
real_ip_header: {{ matrix_dendrite_sync_api_real_ip_header|to_json }}
fulltext:
enabled: false
index_path: "./fulltextindex"
language: "en" # more possible languages can be found at https://github.com/blevesearch/bleve/tree/master/analysis/lang
# Configuration for the User API.
user_api:

View file

@ -7,7 +7,7 @@ matrix_dynamic_dns_enabled: true
# The dynamic dns daemon interval
matrix_dynamic_dns_daemon_interval: '300'
matrix_dynamic_dns_version: v3.9.1-ls97
matrix_dynamic_dns_version: v3.9.1-ls98
# The docker container to use when in mode
matrix_dynamic_dns_docker_image: "{{ matrix_dynamic_dns_docker_image_name_prefix }}linuxserver/ddclient:{{ matrix_dynamic_dns_version }}"

View file

@ -5,7 +5,7 @@
matrix_grafana_enabled: false
matrix_grafana_version: 9.1.2
matrix_grafana_version: 9.1.5
matrix_grafana_docker_image: "{{ matrix_container_global_registry_prefix }}grafana/grafana:{{ matrix_grafana_version }}"
matrix_grafana_docker_image_force_pull: "{{ matrix_grafana_docker_image.endswith(':latest') }}"

View file

@ -547,7 +547,7 @@ matrix_ssl_lets_encrypt_staging: false
# Learn more here: https://eff-certbot.readthedocs.io/en/stable/using.html#changing-the-acme-server
matrix_ssl_lets_encrypt_server: ''
matrix_ssl_lets_encrypt_certbot_docker_image: "{{ matrix_container_global_registry_prefix }}certbot/certbot:{{ matrix_ssl_architecture }}-v1.28.0"
matrix_ssl_lets_encrypt_certbot_docker_image: "{{ matrix_container_global_registry_prefix }}certbot/certbot:{{ matrix_ssl_architecture }}-v1.30.0"
matrix_ssl_lets_encrypt_certbot_docker_image_force_pull: "{{ matrix_ssl_lets_encrypt_certbot_docker_image.endswith(':latest') }}"
matrix_ssl_lets_encrypt_certbot_standalone_http_port: 2402
matrix_ssl_lets_encrypt_support_email: ~
@ -623,9 +623,13 @@ matrix_nginx_proxy_synapse_workers_enabled: false
matrix_nginx_proxy_synapse_workers_list: []
matrix_nginx_proxy_synapse_generic_worker_client_server_locations: []
matrix_nginx_proxy_synapse_generic_worker_federation_locations: []
matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_media_repository_locations: []
matrix_nginx_proxy_synapse_user_dir_locations: []
matrix_nginx_proxy_synapse_frontend_proxy_locations: []
# synapse content caching
matrix_nginx_proxy_synapse_cache_enabled: false

View file

@ -26,7 +26,7 @@
- name: Obtain Let's Encrypt certificates
ansible.builtin.include_tasks: "{{ role_path }}/tasks/ssl/setup_ssl_lets_encrypt_obtain_for_domain.yml"
with_items: "{{ matrix_ssl_domains_to_obtain_certificates_for }}"
with_items: "{{ matrix_ssl_domains_to_obtain_certificates_for | unique }}"
loop_control:
loop_var: domain_name

View file

@ -2,7 +2,7 @@
- name: Verify certificates
ansible.builtin.include_tasks: "{{ role_path }}/tasks/ssl/setup_ssl_manually_managed_verify_for_domain.yml"
with_items: "{{ matrix_ssl_domains_to_obtain_certificates_for }}"
with_items: "{{ matrix_ssl_domains_to_obtain_certificates_for | unique }}"
loop_control:
loop_var: domain_name
when: "matrix_ssl_retrieval_method == 'manually-managed'"

View file

@ -5,7 +5,7 @@
- name: Generate self-signed certificates
ansible.builtin.include_tasks: "{{ role_path }}/tasks/ssl/setup_ssl_self_signed_obtain_for_domain.yml"
with_items: "{{ matrix_ssl_domains_to_obtain_certificates_for }}"
with_items: "{{ matrix_ssl_domains_to_obtain_certificates_for | unique }}"
loop_control:
loop_var: domain_name
when: "matrix_ssl_retrieval_method == 'self-signed'"

View file

@ -1,24 +1,52 @@
#jinja2: lstrip_blocks: "True"
{% set generic_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'generic_worker') | list %}
{% set stream_writer_typing_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'typing') | list %}
{% set stream_writer_to_device_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'to_device') | list %}
{% set stream_writer_account_data_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'account_data') | list %}
{% set stream_writer_receipts_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'receipts') | list %}
{% set stream_writer_presence_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'presence') | list %}
{% set media_repository_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'media_repository') | list %}
{% set user_dir_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'user_dir') | list %}
{% set frontend_proxy_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'frontend_proxy') | list %}
{% macro render_worker_upstream(name, workers, matrix_nginx_proxy_enabled) %}
{% if workers | length > 0 %}
upstream {{ name }} {
{% for worker in workers %}
{% if matrix_nginx_proxy_enabled %}
server "{{ worker.name }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{% endmacro %}
{% macro render_locations_to_upstream(locations, upstream_name) %}
{% for location in locations %}
location ~ {{ location }} {
proxy_pass http://{{ upstream_name }}$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% endmacro %}
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{% if matrix_nginx_proxy_synapse_cache_enabled %}
proxy_cache_path {{ matrix_nginx_proxy_synapse_cache_path }} levels=1:2 keys_zone={{ matrix_nginx_proxy_synapse_cache_keys_zone_name }}:{{ matrix_nginx_proxy_synapse_cache_keys_zone_size }} inactive={{ matrix_nginx_proxy_synapse_cache_inactive_time }} max_size={{ matrix_nginx_proxy_synapse_cache_max_size_mb }}m;
{% endif %}
# Round Robin "upstream" pools for workers
{% if generic_workers %}
upstream generic_worker_upstream {
{% if generic_workers |length > 0 %}
upstream generic_workers_upstream {
# ensures that requests from the same client will always be passed
# to the same server (except when this server is unavailable)
hash $http_x_forwarded_for;
{% for worker in generic_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
server "{{ worker.name }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
@ -26,41 +54,15 @@
}
{% endif %}
{% if frontend_proxy_workers %}
upstream frontend_proxy_upstream {
{% for worker in frontend_proxy_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('stream_writer_typing_stream_workers_upstream', stream_writer_typing_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_to_device_stream_workers_upstream', stream_writer_to_device_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_account_data_stream_workers_upstream', stream_writer_account_data_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_receipts_stream_workers_upstream', stream_writer_receipts_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_presence_stream_workers_upstream', stream_writer_presence_stream_workers, matrix_nginx_proxy_enabled) }}
{% if media_repository_workers %}
upstream media_repository_upstream {
{% for worker in media_repository_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('media_repository_workers_upstream', media_repository_workers, matrix_nginx_proxy_enabled) }}
{% if user_dir_workers %}
upstream user_dir_upstream {
{% for worker in user_dir_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('user_dir_workers_upstream', user_dir_workers, matrix_nginx_proxy_enabled) }}
{% endif %}
server {
@ -78,21 +80,41 @@ server {
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{# Workers redirects BEGIN #}
{% if generic_workers %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappgeneric_worker
{% for location in matrix_nginx_proxy_synapse_generic_worker_client_server_locations %}
location ~ {{ location }} {
proxy_pass http://generic_worker_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% if generic_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappgeneric_worker
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_generic_worker_client_server_locations, 'generic_workers_upstream') }}
{% endif %}
{% if media_repository_workers %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappmedia_repository
{% if stream_writer_typing_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-typing-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations, 'stream_writer_typing_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_to_device_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-to_device-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations, 'stream_writer_to_device_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_account_data_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-account_data-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations, 'stream_writer_account_data_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_receipts_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-receipts-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations, 'stream_writer_receipts_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_presence_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-presence-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations, 'stream_writer_presence_stream_workers_upstream') }}
{% endif %}
{% if media_repository_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository
{% for location in matrix_nginx_proxy_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_upstream$request_uri;
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;
@ -110,32 +132,11 @@ server {
{% endfor %}
{% endif %}
{% if user_dir_workers %}
# FIXME: obsolete if matrix_nginx_proxy_proxy_matrix_user_directory_search_enabled is set
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappuser_dir
{% for location in matrix_nginx_proxy_synapse_user_dir_locations %}
location ~ {{ location }} {
proxy_pass http://user_dir_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% endif %}
{% if frontend_proxy_workers %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappfrontend_proxy
{% for location in matrix_nginx_proxy_synapse_frontend_proxy_locations %}
location ~ {{ location }} {
proxy_pass http://frontend_proxy_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% if matrix_nginx_proxy_synapse_presence_disabled %}
# FIXME: keep in sync with synapse workers documentation manually
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/presence/[^/]+/status {
proxy_pass http://frontend_proxy_upstream$request_uri;
proxy_set_header Host $host;
}
{% endif %}
{% if user_dir_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#updating-the-user-directory
# If matrix_nginx_proxy_proxy_matrix_user_directory_search_enabled is set, requests may not reach here,
# but could be captured early on (see `matrix-domain.conf.j2`) and forwarded elsewhere (to an identity server, etc.).
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_user_dir_locations, 'user_dir_workers_upstream') }}
{% endif %}
{# Workers redirects END #}
{% endif %}
@ -180,20 +181,15 @@ server {
gzip_types text/plain application/json;
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{% if generic_workers %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappgeneric_worker
{% for location in matrix_nginx_proxy_synapse_generic_worker_federation_locations %}
location ~ {{ location }} {
proxy_pass http://generic_worker_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% if generic_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappgeneric_worker
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_generic_worker_federation_locations, 'generic_workers_upstream') }}
{% endif %}
{% if media_repository_workers %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappmedia_repository
{% if media_repository_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository
{% for location in matrix_nginx_proxy_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_upstream$request_uri;
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;

View file

@ -32,16 +32,17 @@ scrape_configs:
static_configs:
- targets: {{ matrix_prometheus_scraper_synapse_targets|to_json }}
labels:
instance: {{ matrix_domain }}
instance: {{ matrix_domain | to_json }}
job: master
index: 0
{% for worker in matrix_prometheus_scraper_synapse_workers_enabled_list %}
{% if worker.metrics_port != 0 %}
- targets: ['matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.metrics_port }}']
- targets: ['{{ worker.name }}:{{ worker.metrics_port }}']
labels:
instance: {{ matrix_domain }}
job: {{ worker.type }}
index: {{ worker.instanceId }}
instance: {{ matrix_domain | to_json }}
worker_id: {{ worker.id | to_json }}
job: {{ worker.type | to_json }}
app: {{ worker.app | to_json }}
{% endif %}
{% endfor %}
{% endif %}

View file

@ -9,7 +9,7 @@ matrix_synapse_container_image_self_build_repo: "https://github.com/matrix-org/s
matrix_synapse_docker_image: "{{ matrix_synapse_docker_image_name_prefix }}matrixdotorg/synapse:{{ matrix_synapse_docker_image_tag }}"
matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_container_image_self_build else matrix_container_global_registry_prefix }}"
matrix_synapse_version: v1.66.0
matrix_synapse_version: v1.67.0
matrix_synapse_docker_image_tag: "{{ matrix_synapse_version }}"
matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}"
@ -394,23 +394,31 @@ matrix_synapse_workers_presets:
little-federation-helper:
generic_workers_count: 0
pusher_workers_count: 0
appservice_workers_count: 0
federation_sender_workers_count: 1
media_repository_workers_count: 0
appservice_workers_count: 0
user_dir_workers_count: 0
frontend_proxy_workers_count: 0
background_workers_count: 0
stream_writer_events_stream_workers_count: 0
stream_writer_typing_stream_workers_count: 0
stream_writer_to_device_stream_workers_count: 0
stream_writer_account_data_stream_workers_count: 0
stream_writer_receipts_stream_workers_count: 0
stream_writer_presence_stream_workers_count: 0
one-of-each:
generic_workers_count: 1
pusher_workers_count: 1
# appservice workers are deprecated since Synapse v1.59. This will be removed.
appservice_workers_count: 0
federation_sender_workers_count: 1
media_repository_workers_count: 1
# Disabled until https://github.com/matrix-org/synapse/issues/8787 is resolved.
# user_dir workers are deprecated since Synapse v1.59. This will be removed.
# See: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
user_dir_workers_count: 0
frontend_proxy_workers_count: 1
appservice_workers_count: 1
user_dir_workers_count: 1
background_workers_count: 1
stream_writer_events_stream_workers_count: 1
stream_writer_typing_stream_workers_count: 1
stream_writer_to_device_stream_workers_count: 1
stream_writer_account_data_stream_workers_count: 1
stream_writer_receipts_stream_workers_count: 1
stream_writer_presence_stream_workers_count: 1
# Controls whether the matrix-synapse container exposes the various worker ports
# (see `port` and `metrics_port` in `matrix_synapse_workers_enabled_list`) outside of the container.
@ -423,38 +431,144 @@ matrix_synapse_workers_generic_workers_count: "{{ matrix_synapse_workers_presets
matrix_synapse_workers_generic_workers_port_range_start: 18111
matrix_synapse_workers_generic_workers_metrics_range_start: 19111
# matrix_synapse_workers_pusher_workers_count can only be 0 or 1 for now.
# More instances are not supported due to a playbook limitation having to do with keeping `pusher_instances` in `homeserver.yaml` updated.
# See https://github.com/matrix-org/synapse/commit/ddfdf945064925eba761ae3748e38f3a1c73c328
# matrix_synapse_workers_stream_writer_events_stream_workers_count controls how many stream writers that handle the `events` stream to spawn.
# More than 1 worker is also supported of this type.
matrix_synapse_workers_stream_writer_events_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_events_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_typing_stream_workers_count controls how many stream writers that handle the `typing` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_typing_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_typing_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_to_device_stream_workers_count controls how many stream writers that handle the `to_device` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_to_device_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_to_device_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_account_data_stream_workers_count controls how many stream writers that handle the `account_data` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_account_data_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_account_data_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_receipts_stream_workers_count controls how many stream writers that handle the `receipts` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_receipts_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_receipts_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_presence_stream_workers_count controls how many stream writers that handle the `presence` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_presence_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_presence_stream_workers_count'] }}"
# A list of stream writer workers to enable. This list is built automatically based on other variables.
# You're encouraged to enable/disable stream writer workers by setting `matrix_synapse_workers_stream_writer_*_stream_workers_count` variables, instead of adjusting this list manually.
matrix_synapse_workers_stream_writers: |
{{
[]
+
([{'stream': 'events'}] * matrix_synapse_workers_stream_writer_events_stream_workers_count | int)
+
([{'stream': 'typing'}] * matrix_synapse_workers_stream_writer_typing_stream_workers_count | int)
+
([{'stream': 'to_device'}] * matrix_synapse_workers_stream_writer_to_device_stream_workers_count | int)
+
([{'stream': 'account_data'}] * matrix_synapse_workers_stream_writer_account_data_stream_workers_count | int)
+
([{'stream': 'receipts'}] * matrix_synapse_workers_stream_writer_receipts_stream_workers_count | int)
+
([{'stream': 'presence'}] * matrix_synapse_workers_stream_writer_presence_stream_workers_count | int)
}}
# matrix_synapse_stream_writers populates the `stream_writers` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# What you see below is an initial default value which will be adjusted at runtime based on the value of `matrix_synapse_workers_stream_writers`.
# Adjusting this value manually is generally not necessary.
#
# It's tempting to initialize this like this:
# matrix_synapse_stream_writers:
# - typing: []
# - events: []
# - to_device: []
# - account_data: []
# - receipts: []
# - presence: []
# .. but Synapse does not like empty lists (see https://github.com/matrix-org/synapse/issues/13804)
matrix_synapse_stream_writers: {}
# `matrix_synapse_workers_stream_writer_workers_` variables control the port numbers of various stream writer workers
# defined in `matrix_synapse_workers_stream_writers`.
# It should be noted that not all of the background worker types will need to expose HTTP services, etc.
matrix_synapse_workers_stream_writer_workers_http_port_range_start: 20011
matrix_synapse_workers_stream_writer_workers_replication_port_range_start: 25011
matrix_synapse_workers_stream_writer_workers_metrics_range_start: 19211
# matrix_synapse_workers_pusher_workers_count controls the number of pusher workers (workers who push out notifications) to spawn.
# See https://matrix-org.github.io/synapse/latest/workers.html#synapseapppusher
matrix_synapse_workers_pusher_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['pusher_workers_count'] }}"
matrix_synapse_workers_pusher_workers_metrics_range_start: 19200
# matrix_synapse_workers_appservice_workers_count can only be 0 or 1. More instances are not supported.
# appservice workers are deprecated since Synapse v1.59. This will be removed.
# See: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
matrix_synapse_workers_appservice_workers_count: 0
matrix_synapse_workers_appservice_workers_metrics_range_start: 19300
# matrix_synapse_federation_pusher_instances populates the `pusher_instances` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# What you see below is an initial default value which will be adjusted at runtime based on the value of `matrix_synapse_workers_pusher_workers_count` or `matrix_synapse_workers_enabled_list`.
# Adjusting this value manually is generally not necessary.
matrix_synapse_federation_pusher_instances: []
# matrix_synapse_workers_federation_sender_workers_count can only be 0 or 1 for now.
# More instances are not supported due to a playbook limitation having to do with keeping `federation_sender_instances` in `homeserver.yaml` updated.
# See https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappfederation_sender
# matrix_synapse_start_pushers controls if the main Synapse process should push out notifications or if it should be left to pusher workers (see `matrix_synapse_federation_pusher_instances`).
# This is enabled if workers are disabled, or if they are enabled, but there are no pusher workers.
# Adjusting this value manually is generally not necessary.
matrix_synapse_start_pushers: "{{ not matrix_synapse_workers_enabled or (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list | length == 0) }}"
# matrix_synapse_workers_federation_sender_workers_count controls the number of federation sender workers to spawn.
# See https://matrix-org.github.io/synapse/latest/workers.html#synapseappfederation_sender
matrix_synapse_workers_federation_sender_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['federation_sender_workers_count'] }}"
matrix_synapse_workers_federation_sender_workers_metrics_range_start: 19400
# matrix_synapse_federation_sender_instances populates the `federation_sender_instances` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# What you see below is an initial default value which will be adjusted at runtime based on the value of `matrix_synapse_workers_federation_sender_workers_count` or `matrix_synapse_workers_enabled_list`.
# Adjusting this value manually is generally not necessary.
matrix_synapse_federation_sender_instances: []
# matrix_synapse_send_federation controls if the main Synapse process should send federation traffic or if it should be left to federation_sender workers (see `matrix_synapse_federation_sender_instances`).
# This is allowed if workers are disabled, or they are enabled, but there are no federation sender workers.
# Adjusting this value manually is generally not necessary.
matrix_synapse_send_federation: "{{ not matrix_synapse_workers_enabled or (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list | length == 0) }}"
matrix_synapse_workers_media_repository_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['media_repository_workers_count'] }}"
matrix_synapse_workers_media_repository_workers_port_range_start: 18551
matrix_synapse_workers_media_repository_workers_metrics_range_start: 19551
# Disabled until https://github.com/matrix-org/synapse/issues/8787 is resolved.
# user_dir workers are deprecated since Synapse v1.59. This will be removed.
# See: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
matrix_synapse_workers_user_dir_workers_count: 0
# matrix_synapse_enable_media_repo controls if the main Synapse process should serve media repository endpoints or if it should be left to media_repository workers (see `matrix_synapse_workers_media_repository_workers_count`).
# This is enabled if workers are disabled, or if they are enabled, but there are no media repository workers.
# Adjusting this value manually is generally not necessary.
matrix_synapse_enable_media_repo: "{{ not matrix_synapse_workers_enabled or (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list | length == 0) }}"
# matrix_synapse_media_instance_running_background_jobs populates the `media_instance_running_background_jobs` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# `media_instance_running_background_jobs` is meant to point to a single media-repository worker, which is dedicated to running background tasks that maintain the media repository.
# Multiple `media_repository` workers may be enabled. We always pick the first one as the background tasks worker.
matrix_synapse_media_instance_running_background_jobs: "{{ (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list)[0].name if (matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list | length > 0) else '' }}"
# matrix_synapse_workers_appservice_workers_count can only be 0 or 1. More instances are not supported.
# appservice workers were deprecated since Synapse v1.59 (see: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types).
# Our implementation uses generic worker services and assigns them to perform appservice work using the `notify_appservices_from_worker` Synapse option.
matrix_synapse_workers_appservice_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['appservice_workers_count'] }}"
matrix_synapse_workers_appservice_workers_metrics_range_start: 19300
# matrix_synapse_notify_appservices_from_worker populates the `notify_appservices_from_worker` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# `notify_appservices_from_worker` is meant to point to a worker, which is dedicated to sending output traffic to Application Services.
matrix_synapse_notify_appservices_from_worker: "{{ (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'appservice') | list)[0].name if (matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'appservice') | list | length > 0) else '' }}"
# matrix_synapse_workers_user_dir_workers_count can only be 0 or 1. More instances are not supported.
# user_dir workers were deprecated since Synapse v1.59 (see: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types).
# Our implementation uses generic worker services and assigns them to perform appservice work using the `update_user_directory_from_worker` Synapse option.
matrix_synapse_workers_user_dir_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['user_dir_workers_count'] }}"
matrix_synapse_workers_user_dir_workers_port_range_start: 18661
matrix_synapse_workers_user_dir_workers_metrics_range_start: 19661
matrix_synapse_workers_frontend_proxy_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['frontend_proxy_workers_count'] }}"
matrix_synapse_workers_frontend_proxy_workers_port_range_start: 18771
matrix_synapse_workers_frontend_proxy_workers_metrics_range_start: 19771
# matrix_synapse_update_user_directory_from_worker populates the `update_user_directory_from_worker` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# `update_user_directory_from_worker` is meant to point to a worker, which is dedicated to updating the user directory and servicing some user directory URL endpoints (`matrix_synapse_workers_user_dir_worker_client_server_endpoints`).
matrix_synapse_update_user_directory_from_worker: "{{ (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'user_dir') | list)[0].name if (matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'user_dir') | list | length > 0) else '' }}"
# matrix_synapse_workers_background_workers_count can only be 0 or 1. More instances are not supported.
# Our implementation uses a generic worker and assigns Synapse to perform background work on this worker using the `run_background_tasks_on` Synapse option.
matrix_synapse_workers_background_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['background_workers_count'] }}"
matrix_synapse_workers_background_workers_metrics_range_start: 19700
# matrix_synapse_run_background_tasks_on populates the `run_background_tasks_on` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# `run_background_tasks_on` is meant to point to a worker, which is dedicated to processing background tasks.
matrix_synapse_run_background_tasks_on: "{{ (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'background') | list)[0].name if (matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'background') | list | length > 0) else '' }}"
# Default list of workers to spawn.
#
@ -469,25 +583,36 @@ matrix_synapse_workers_frontend_proxy_workers_metrics_range_start: 19771
# as certain workers can only be spawned just once.
#
# Each worker instance in the list defines the following fields:
# - `type` - the type of worker (`generic_worker`, etc.)
# - `instanceId` - a string that identifies the worker. The combination of (`type` + `instanceId`) represents the name of the worker and must be unique.
# - `id` - a string that uniquely identifies the worker
# - `name` - a string that will be used as the container and systemd service name
# - `type` - the type of worker (`generic_worker`, `stream_writer`, `pusher`, etc.)
# - `app` - the Synapse app (https://matrix-org.github.io/synapse/latest/workers.html#available-worker-applications) that powers this worker (`generic_worker`, `federation_sender`, etc.).
# The `app` usually matches the `type`, but not always. For example, `type = stream_writer` workers are served by the `generic_worker` type.
# - `port` - an HTTP port where the worker listens for requests (can be `0` for workers that don't do HTTP request processing)
# - `metrics_port` - an HTTP port where the worker exports Prometheus metrics
# - `replication_port` - an HTTP port where the worker serves `replication` endpoints (used by stream writers, etc.)
# - `webserving` - tells whether this type of worker serves web (client or federation) requests, so that it can be injected as a dependency to the reverse-proxy
#
# Example of what this needs to look like, if you're defining it manually:
# matrix_synapse_workers_enabled_list:
# - { type: generic_worker, instanceId: '18111', port: 18111, metrics_port: 19111 }
# - { type: generic_worker, instanceId: '18112', port: 18112, metrics_port: 19112 }
# - { type: generic_worker, instanceId: '18113', port: 18113, metrics_port: 19113 }
# - { type: generic_worker, instanceId: '18114', port: 18114, metrics_port: 19114 }
# - { type: generic_worker, instanceId: '18115', port: 18115, metrics_port: 19115 }
# - { type: generic_worker, instanceId: '18116', port: 18116, metrics_port: 19116 }
# - { type: pusher, instanceId: '0', port: 0, metrics_port: 19200 }
# - { type: appservice, instanceId: '0', port: 0, metrics_port: 19300 }
# - { type: federation_sender, instanceId: '0', port: 0, metrics_port: 19400 }
# - { type: media_repository, instanceId: '18551', port: 18551, metrics_port: 19551 }
# - { 'id': 'generic-worker-0', 'name': 'matrix-synapse-worker-generic-0', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18111, 'metrics_port': 19111, 'webserving': true }
# - { 'id': 'generic-worker-1', 'name': 'matrix-synapse-worker-generic-1', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18112, 'metrics_port': 19112, 'webserving': true }
# - { 'id': 'generic-worker-2', 'name': 'matrix-synapse-worker-generic-2', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18113, 'metrics_port': 19113, 'webserving': true }
# - { 'id': 'generic-worker-3', 'name': 'matrix-synapse-worker-generic-3', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18114, 'metrics_port': 19114, 'webserving': true }
# - { 'id': 'generic-worker-4', 'name': 'matrix-synapse-worker-generic-4', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18115, 'metrics_port': 19115, 'webserving': true }
# - { 'id': 'generic-worker-5', 'name': 'matrix-synapse-worker-generic-5', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18116, 'metrics_port': 19116, 'webserving': true }
# - { 'id': 'stream-writer-0-events', 'name': 'matrix-synapse-worker-stream-writer-0-events', 'type': 'stream_writer', 'app': 'generic_worker', 'stream_writer_stream': 'events', 'port': 0, 'replication_port': 25011, metrics_port: 19111, 'webserving': false }
# - { 'id': 'stream-writer-1-typing', 'name': 'matrix-synapse-worker-stream-writer-1-typing', 'type': 'stream_writer', 'app': 'generic_worker', 'stream_writer_stream': 'typing', 'port': 20012, 'replication_port': 25012, metrics_port: 19112, 'webserving': true }
# - { 'id': 'pusher-0', 'name': 'matrix-synapse-worker-pusher-0', 'type': 'pusher', 'app': 'pusher', 'port': 0, 'metrics_port': 19200, 'webserving': false }
# - { 'id': 'appservice-0', 'name': 'matrix-synapse-worker-appservice-0', 'type': 'appservice', 'port': 0, 'metrics_port': 19300, 'webserving': false }
# - { 'id': 'federation-sender-0', 'name': 'matrix-synapse-worker-federation-sender-0', 'type': 'federation_sender', 'port': 0, 'metrics_port': 19400, 'webserving': false }
# - { 'id': 'media-repository-0', 'name': 'matrix-synapse-worker-media-repository-0', 'type': 'media_repository', 'port': 18551, 'metrics_port': 19551, 'webserving': true }
matrix_synapse_workers_enabled_list: []
# matrix_synapse_instance_map holds the instance map used for mapping worker names (for certain generic workers only!) to where they live (host, port which handles replication traffic).
# This is populated automatically based on `matrix_synapse_workers_enabled_list` during runtime, so you're not required to tweak it manually.
matrix_synapse_instance_map: {}
# Redis information
matrix_synapse_redis_enabled: false
matrix_synapse_redis_host: ""

View file

@ -8,15 +8,15 @@
# Unless `matrix_synapse_workers_enabled_list` is explicitly defined,
# we'll generate it dynamically.
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/synapse/workers/init.yml"
- ansible.builtin.include_tasks: "{{ role_path }}/tasks/synapse/workers/init.yml"
when: "matrix_synapse_enabled and matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | length == 0"
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-synapse.service'] }}"
when: matrix_synapse_enabled | bool
- name: Ensure systemd services for workers are injected
ansible.builtin.include_tasks: "{{ role_path }}/tasks/synapse/workers/util/inject_systemd_services_for_worker.yml"
- name: Ensure workers are injected into various places
ansible.builtin.include_tasks: "{{ role_path }}/tasks/synapse/workers/util/inject_worker.yml"
with_items: "{{ matrix_synapse_workers_enabled_list }}"
loop_control:
loop_var: matrix_synapse_worker_details
@ -65,9 +65,9 @@
matrix_synapse_worker_nginx_metrics_configuration_block: |
{% for worker in matrix_synapse_workers_enabled_list %}
{% if worker.metrics_port != 0 %}
location /metrics/synapse/worker/{{ worker.type }}-{{ worker.instanceId }} {
location /metrics/synapse/worker/{{ worker.id }} {
resolver 127.0.0.11 valid=5s;
set $backend "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.metrics_port }}";
set $backend "{{ worker.name }}:{{ worker.metrics_port }}";
proxy_pass http://$backend/_synapse/metrics;
proxy_set_header Host $host;
}

View file

@ -7,29 +7,66 @@
- name: Build generic workers
ansible.builtin.set_fact:
worker:
id: "generic-worker-{{ item }}"
name: "matrix-synapse-worker-generic-{{ item }}"
type: 'generic_worker'
instanceId: "{{ matrix_synapse_workers_generic_workers_port_range_start + item }}"
app: 'generic_worker'
webserving: true
port: "{{ matrix_synapse_workers_generic_workers_port_range_start + item }}"
metrics_port: "{{ matrix_synapse_workers_generic_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_generic_workers"
loop: "{{ range(0, matrix_synapse_workers_generic_workers_count | int) | list }}"
- name: Build stream writer workers
ansible.builtin.set_fact:
worker:
id: "stream-writer-{{ idx }}-{{ item.stream }}"
# Names must not include understores. Certain stream writer streams (to_device, account_data, ..) do, so we fix them up.
name: "matrix-synapse-worker-stream-writer-{{ idx }}-{{ item.stream | replace('_', '-') }}"
type: 'stream_writer'
app: "generic_worker"
webserving: "{{ item.stream in matrix_synapse_workers_webserving_stream_writer_types }}"
stream_writer_stream: "{{ item.stream }}"
port: "{{ matrix_synapse_workers_stream_writer_workers_http_port_range_start + idx }}"
replication_port: "{{ matrix_synapse_workers_stream_writer_workers_replication_port_range_start + idx }}"
metrics_port: "{{ matrix_synapse_workers_stream_writer_workers_metrics_range_start + idx }}"
register: "matrix_synapse_workers_list_results_stream_writer_workers"
loop: "{{ matrix_synapse_workers_stream_writers }}"
loop_control:
index_var: idx
- name: Populate matrix_synapse_stream_writers from enabled stream writer workers list
ansible.builtin.set_fact:
matrix_synapse_stream_writers: "{{ matrix_synapse_stream_writers | combine({item.ansible_facts.worker.stream_writer_stream: [item.ansible_facts.worker.name]}) }}"
with_items: "{{ matrix_synapse_workers_list_results_stream_writer_workers.results }}"
- name: Build federation sender workers
ansible.builtin.set_fact:
worker:
id: "federation-sender-{{ item }}"
name: "matrix-synapse-worker-federation-sender-{{ item }}"
type: 'federation_sender'
instanceId: "{{ item }}"
app: 'federation_sender'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_federation_sender_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_federation_sender_workers"
loop: "{{ range(0, matrix_synapse_workers_federation_sender_workers_count | int) | list }}"
- name: Populate matrix_synapse_federation_sender_instances from enabled federation sender workers list
ansible.builtin.set_fact:
matrix_synapse_federation_sender_instances: "{{ matrix_synapse_federation_sender_instances + [item.ansible_facts.worker.name] }}"
with_items: "{{ matrix_synapse_workers_list_results_federation_sender_workers.results }}"
# This type of worker can only have a count of 1, at most
- name: Build pusher workers
ansible.builtin.set_fact:
worker:
id: "pusher-{{ item }}"
name: "matrix-synapse-worker-pusher-{{ item }}"
type: 'pusher'
instanceId: "{{ item }}"
app: 'pusher'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_pusher_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_pusher_workers"
@ -39,48 +76,76 @@
- name: Build appservice workers
ansible.builtin.set_fact:
worker:
id: "appservice-{{ item }}"
name: "matrix-synapse-worker-appservice-{{ item }}"
type: 'appservice'
instanceId: "{{ item }}"
app: 'generic_worker'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_appservice_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_appservice_workers"
loop: "{{ range(0, matrix_synapse_workers_appservice_workers_count | int) | list }}"
# This type of worker can only have a count of 1, at most
- name: Build user_dir workers
ansible.builtin.set_fact:
worker:
id: "user-dir-{{ item }}"
name: "matrix-synapse-worker-user-dir-{{ item }}"
type: 'user_dir'
app: 'generic_worker'
webserving: true
port: "{{ matrix_synapse_workers_user_dir_workers_port_range_start + item }}"
metrics_port: "{{ matrix_synapse_workers_user_dir_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_user_dir_workers"
loop: "{{ range(0, matrix_synapse_workers_user_dir_workers_count | int) | list }}"
# This type of worker can only have a count of 1, at most
- name: Build background workers
ansible.builtin.set_fact:
worker:
id: "background-{{ item }}"
name: "matrix-synapse-worker-background-{{ item }}"
type: 'background'
app: 'generic_worker'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_background_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_background_workers"
loop: "{{ range(0, matrix_synapse_workers_background_workers_count | int) | list }}"
- name: Build media_repository workers
ansible.builtin.set_fact:
worker:
id: "media-repository-{{ item }}"
name: "matrix-synapse-worker-media-repository-{{ item }}"
type: 'media_repository'
instanceId: "{{ matrix_synapse_workers_media_repository_workers_port_range_start + item }}"
app: 'media_repository'
webserving: true
port: "{{ matrix_synapse_workers_media_repository_workers_port_range_start + item }}"
metrics_port: "{{ matrix_synapse_workers_media_repository_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_media_repository_workers"
loop: "{{ range(0, matrix_synapse_workers_media_repository_workers_count | int) | list }}"
- name: Build frontend_proxy workers
ansible.builtin.set_fact:
worker:
type: 'frontend_proxy'
instanceId: "{{ matrix_synapse_workers_frontend_proxy_workers_port_range_start + item }}"
port: "{{ matrix_synapse_workers_frontend_proxy_workers_port_range_start + item }}"
metrics_port: "{{ matrix_synapse_workers_frontend_proxy_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_frontend_proxy_workers"
loop: "{{ range(0, matrix_synapse_workers_frontend_proxy_workers_count | int) | list }}"
- ansible.builtin.set_fact:
matrix_synapse_dynamic_workers_list: "{{ matrix_synapse_dynamic_workers_list | default([]) + [item.ansible_facts.worker] }}"
with_items: |
{{
matrix_synapse_workers_list_results_generic_workers.results
+
matrix_synapse_workers_list_results_stream_writer_workers.results
+
matrix_synapse_workers_list_results_federation_sender_workers.results
+
matrix_synapse_workers_list_results_pusher_workers.results
+
matrix_synapse_workers_list_results_appservice_workers.results
+
matrix_synapse_workers_list_results_user_dir_workers.results
+
matrix_synapse_workers_list_results_media_repository_workers.results
+
matrix_synapse_workers_list_results_frontend_proxy_workers.results
matrix_synapse_workers_list_results_background_workers.results
}}
- ansible.builtin.set_fact:

View file

@ -1,19 +0,0 @@
---
# The tasks below run before `validate_config.yml`.
# To avoid failing with a cryptic error message, we'll do validation here.
#
# This check is mostly relevant to people who explicitly define `matrix_synapse_workers_enabled_list`
# (Synapse Workers users from the earlier days of this PR - https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/456).
#
# In the future, it should be possible to remove this check.
# Our own code which dynamically builds `matrix_synapse_workers_enabled_list` does things right.
- name: Fail if instanceId not defined for worker
ansible.builtin.fail:
msg: "Synapse workers (like {{ matrix_synapse_worker_details | to_json }}) need to define an instanceId property (type + instanceId must be unique)"
when: "'instanceId' not in matrix_synapse_worker_details"
- ansible.builtin.set_fact:
matrix_synapse_worker_systemd_service_name: "matrix-synapse-worker-{{ matrix_synapse_worker_details.type }}-{{ matrix_synapse_worker_details.instanceId }}.service"
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + [matrix_synapse_worker_systemd_service_name] }}"

View file

@ -0,0 +1,65 @@
---
# The tasks below run before `validate_config.yml`.
# To avoid failing with a cryptic error message, we'll do validation here.
#
# This check is mostly relevant to people who explicitly define `matrix_synapse_workers_enabled_list`
# (Synapse Workers users from the earlier days of this PR - https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/456).
#
# In the future, it should be possible to remove this check.
# Our own code which dynamically builds `matrix_synapse_workers_enabled_list` does things right.
- name: Fail if required property not defined for worker
ansible.builtin.fail:
msg: "Synapse workers (like {{ matrix_synapse_worker_details | to_json }}) need to define a `{{ item }}` property"
with_items:
- id
- name
- type
- app
- port
- webserving
when: "item not in matrix_synapse_worker_details"
# Names are used for container names and systemd services.
# Routing happens based on container names, so Synapse processes that try to route to workers with underscores in the name will complain. Example:
# > InvalidCodepoint Codepoint U+005F at position 46 of 'matrix-synapse-worker-stream-writer-3-account_data' not allowed
- name: Fail if worker name includes underscore
ansible.builtin.fail:
msg: "Unrecognized Synapse worker `name`: `{{ matrix_synapse_worker_details.name }}`. It must not include underscores"
when: "'_' in matrix_synapse_worker_details.name"
- name: Fail if worker type unknown
ansible.builtin.fail:
msg: "Unrecognized Synapse worker `type`: `{{ matrix_synapse_worker_details.type }}`. Supported types are: {{ matrix_synapse_known_worker_types | join(', ') }}"
when: "matrix_synapse_worker_details.type not in matrix_synapse_known_worker_types"
- name: Fail if worker app unknown
ansible.builtin.fail:
msg: "Unrecognized Synapse worker `app`: `{{ matrix_synapse_worker_details.app }}`. Supported types are: {{ matrix_synapse_workers_avail_list | join(', ') }}"
when: "matrix_synapse_worker_details.app not in matrix_synapse_workers_avail_list"
- block:
- name: Fail if stream_writer_stream not defined for stream_writer worker
ansible.builtin.fail:
msg: >-
Synapse stream_writer workers (such as {{ item }}) need to define a valid `stream_writer_stream` property
(not `{{ matrix_synapse_worker_details.stream_writer_stream | default('undefined') }}`).
Supported types are: {{ matrix_synapse_workers_known_stream_writer_stream_types | join(', ') }}
when: "'stream_writer_stream' not in matrix_synapse_worker_details or matrix_synapse_worker_details.stream_writer_stream not in matrix_synapse_workers_known_stream_writer_stream_types"
- name: Fail if replication_port not defined for stream_writer worker
ansible.builtin.fail:
msg: "Synapse background workers of type stream_writer (such as {{ item }}) need to define a valid `replication_port` property"
when: "'replication_port' not in matrix_synapse_worker_details"
when: "matrix_synapse_worker_details.type == 'stream_writer'"
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + [matrix_synapse_worker_details.name + '.service'] }}"
- ansible.builtin.set_fact:
matrix_synapse_webserving_workers_systemd_services_list: "{{ matrix_synapse_webserving_workers_systemd_services_list + [matrix_synapse_worker_details.name + '.service'] }}"
when: matrix_synapse_worker_details.webserving | bool
# Inject stream writers into the instance map.
- ansible.builtin.set_fact:
matrix_synapse_instance_map: "{{ matrix_synapse_instance_map | combine({matrix_synapse_worker_details.name: {'host': matrix_synapse_worker_details.name, 'port': matrix_synapse_worker_details.replication_port}}) }}"
when: matrix_synapse_worker_details.type in matrix_synapse_known_instance_map_eligible_worker_types

View file

@ -1,13 +1,9 @@
---
- ansible.builtin.set_fact:
matrix_synapse_worker_systemd_service_name: "matrix-synapse-worker-{{ matrix_synapse_worker_details.type }}-{{ matrix_synapse_worker_details.instanceId }}"
- ansible.builtin.set_fact:
matrix_synapse_worker_container_name: "{{ matrix_synapse_worker_systemd_service_name }}"
- ansible.builtin.set_fact:
matrix_synapse_worker_config_file_name: "worker.{{ matrix_synapse_worker_details.type }}_{{ matrix_synapse_worker_details.instanceId }}.yaml"
matrix_synapse_worker_systemd_service_name: "{{ matrix_synapse_worker_details.name }}"
matrix_synapse_worker_container_name: "{{ matrix_synapse_worker_details.name }}"
matrix_synapse_worker_config_file_name: "worker.{{ matrix_synapse_worker_details.name }}.yaml"
- name: Ensure configuration exists for {{ matrix_synapse_worker_systemd_service_name }}
ansible.builtin.template:

View file

@ -12,26 +12,20 @@
- "matrix_synapse_database_password"
- "matrix_synapse_database_database"
- name: Fail if asking to configure deprecaed workers (appservice, userdir)
ansible.builtin.fail:
msg: >-
`{{ item }}` cannot be more than 0.
This type of worker has been deprecated since Synapse v1.59.
Please remove your `{{ item }}` configuration to solve this problem.
See: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
when: "vars[item]|int != 0"
with_items:
- "matrix_synapse_workers_appservice_workers_count"
- "matrix_synapse_workers_user_dir_workers_count"
- name: Fail if asking for more than 1 instance of single-instance workers
ansible.builtin.fail:
msg: >-
`{{ item }}` cannot be more than 1. This is a single-instance worker.
when: "vars[item] | int > 1"
with_items:
- "matrix_synapse_workers_pusher_workers_count"
- "matrix_synapse_workers_federation_sender_workers_count"
- "matrix_synapse_workers_appservice_workers_count"
- "matrix_synapse_workers_user_dir_workers_count"
- "matrix_synapse_workers_background_workers_count"
- "matrix_synapse_workers_stream_writer_typing_stream_workers_count"
- "matrix_synapse_workers_stream_writer_to_device_stream_workers_count"
- "matrix_synapse_workers_stream_writer_account_data_stream_workers_count"
- "matrix_synapse_workers_stream_writer_receipts_stream_workers_count"
- "matrix_synapse_workers_stream_writer_presence_stream_workers_count"
- name: (Deprecation) Catch and report renamed settings
ansible.builtin.fail:
@ -62,6 +56,9 @@
- {'old': 'matrix_synapse_version_arm64', 'new': '<superseded by matrix_synapse_version - see https://github.com/matrix-org/synapse/pull/11810>'}
- {'old': 'matrix_synapse_enable_group_creation', 'new': '<removed in Synapse v1.61.0 - use the new Spaces feature instead>'}
- {'old': 'matrix_synapse_account_threepid_delegates_email', 'new': '<removed in Synapse v1.66.0 - make sure to configure email settings for Synapse - see https://matrix-org.github.io/synapse/v1.66/upgrade.html#delegation-of-email-validation-no-longer-supported>'}
- {'old': 'matrix_synapse_workers_frontend_proxy_workers_count', 'new': '<removed in favor of generic workers - see https://github.com/matrix-org/synapse/pull/13645>'}
- {'old': 'matrix_synapse_workers_frontend_proxy_workers_port_range_start', 'new': '<removed in favor of generic workers - see https://github.com/matrix-org/synapse/pull/13645>'}
- {'old': 'matrix_synapse_workers_frontend_proxy_workers_metrics_range_start', 'new': '<removed in favor of generic workers - see https://github.com/matrix-org/synapse/pull/13645>'}
- name: (Deprecation) Catch and report renamed settings in matrix_synapse_configuration_extension_yaml
ansible.builtin.fail:

View file

@ -349,19 +349,6 @@ listeners:
# c.f. https://github.com/matrix-org/synapse/tree/master/contrib/systemd-with-workers/README.md
worker_app: synapse.app.homeserver
# thx https://oznetnerd.com/2017/04/18/jinja2-selectattr-filter/
# reduce the main worker's offerings to core homeserver business
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list %}
send_federation: false
{% endif %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list %}
enable_media_repo: false
{% endif %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list %}
start_pushers: false
{% endif %}
daemonize: false
{% endif %}
@ -1024,6 +1011,7 @@ federation_rr_transactions_per_room_per_second: {{ matrix_synapse_federation_rr_
# following if you are using a separate media store worker.
#
#enable_media_repo: false
enable_media_repo: {{ matrix_synapse_enable_media_repo | to_json }}
# Directory where uploaded images and attachments are stored.
#
@ -2852,6 +2840,7 @@ opentracing:
# Uncomment if using a federation sender worker.
#
#send_federation: false
send_federation: {{ matrix_synapse_send_federation | to_json }}
# It is possible to run multiple federation sender workers, in which case the
# work is balanced across them.
@ -2863,6 +2852,14 @@ opentracing:
#
#federation_sender_instances:
# - federation_sender1
{% if matrix_synapse_federation_sender_instances | length > 0 %}
federation_sender_instances: {{ matrix_synapse_federation_sender_instances | to_json }}
{% endif %}
{% if matrix_synapse_federation_pusher_instances | length > 0 %}
pusher_instances: {{ matrix_synapse_federation_pusher_instances | to_json }}
{% endif %}
start_pushers: {{ matrix_synapse_start_pushers | to_json }}
# When using workers this should be a map from `worker_name` to the
# HTTP replication listener of the worker, if configured.
@ -2871,6 +2868,7 @@ opentracing:
# worker1:
# host: localhost
# port: 8034
instance_map: {{ matrix_synapse_instance_map | to_json }}
# Experimental: When using workers you can define which workers should
# handle event persistence and typing notifications. Any worker
@ -2879,11 +2877,27 @@ opentracing:
#stream_writers:
# events: worker1
# typing: worker1
stream_writers: {{ matrix_synapse_stream_writers | to_json }}
{% if matrix_synapse_notify_appservices_from_worker != '' %}
notify_appservices_from_worker: {{ matrix_synapse_notify_appservices_from_worker | to_json }}
{% endif %}
{% if matrix_synapse_update_user_directory_from_worker != '' %}
update_user_directory_from_worker: {{ matrix_synapse_update_user_directory_from_worker | to_json }}
{% endif %}
# The worker that is used to run background tasks (e.g. cleaning up expired
# data). If not provided this defaults to the main process.
#
#run_background_tasks_on: worker1
{% if matrix_synapse_run_background_tasks_on != '' %}
run_background_tasks_on: {{ matrix_synapse_run_background_tasks_on | to_json }}
{% endif %}
{% if matrix_synapse_media_instance_running_background_jobs != '' %}
media_instance_running_background_jobs: {{ matrix_synapse_media_instance_running_background_jobs | to_json }}
{% endif %}
# A shared secret used by the replication APIs to authenticate HTTP requests
# from workers.

View file

@ -24,8 +24,8 @@ scrape_configs:
job: "master"
index: "0"
{% for worker in matrix_synapse_workers_enabled_list %}
- job_name: 'synapse-{{ worker.type }}-{{ worker.instanceId }}'
metrics_path: /metrics/synapse/worker/{{ worker.type }}-{{ worker.instanceId }}
- job_name: '{{ worker.name }}'
metrics_path: /metrics/synapse/worker/{{ worker.id }}
scheme: {{ 'https' if matrix_nginx_proxy_https_enabled|default(true) else 'http' }}
{% if matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled|default(true) %}
basic_auth:
@ -35,6 +35,7 @@ scrape_configs:
static_configs:
- targets: ['{{ matrix_server_fqn_matrix }}:{{ matrix_nginx_proxy_container_https_host_bind_port|default(443) if matrix_nginx_proxy_https_enabled|default(true) else matrix_nginx_proxy_container_http_host_bind_port|default(80) }}']
labels:
worker_id: {{ worker.id }}
job: "{{ worker.type }}"
index: "{{ worker.instanceId }}"
app: {{ worker.app }}
{% endfor %}

View file

@ -8,8 +8,8 @@ After=matrix-synapse.service
Type=simple
Environment="HOME={{ matrix_systemd_unit_home_path }}"
ExecStartPre=-{{ matrix_host_command_docker }} kill {{ matrix_synapse_worker_container_name }}
ExecStartPre=-{{ matrix_host_command_docker }} rm {{ matrix_synapse_worker_container_name }}
ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill {{ matrix_synapse_worker_container_name }} 2>/dev/null || true'
ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm {{ matrix_synapse_worker_container_name }} 2>/dev/null || true'
# Intentional delay, so that the homeserver can manage to start.
ExecStartPre={{ matrix_host_command_sleep }} 5
@ -43,11 +43,11 @@ ExecStart={{ matrix_host_command_docker }} run --rm --name {{ matrix_synapse_wor
{{ arg }} \
{% endfor %}
{{ matrix_synapse_docker_image }} \
run -m synapse.app.{{ matrix_synapse_worker_details.type }} -c /data/homeserver.yaml -c /data/{{ matrix_synapse_worker_config_file_name }}
run -m synapse.app.{{ matrix_synapse_worker_details.app }} -c /data/homeserver.yaml -c /data/{{ matrix_synapse_worker_config_file_name }}
ExecStop=-{{ matrix_host_command_docker }} kill {{ matrix_synapse_worker_container_name }}
ExecStop=-{{ matrix_host_command_docker }} rm {{ matrix_synapse_worker_container_name }}
ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill {{ matrix_synapse_worker_container_name }} 2>/dev/null || true'
ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm {{ matrix_synapse_worker_container_name }} 2>/dev/null || true'
ExecReload={{ matrix_host_command_docker }} exec {{ matrix_synapse_worker_container_name }} /bin/sh -c 'kill -HUP 1'
Restart=always

View file

@ -12,7 +12,7 @@ Wants={{ service }}
{% if matrix_synapse_workers_enabled %}
{% for matrix_synapse_worker_details in matrix_synapse_workers_enabled_list %}
Wants=matrix-synapse-worker-{{ matrix_synapse_worker_details.type }}-{{ matrix_synapse_worker_details.port }}.service
Wants={{ matrix_synapse_worker_details.name }}.service
{% endfor %}
{% endif %}

View file

@ -1,27 +1,45 @@
#jinja2: lstrip_blocks: "True"
worker_app: synapse.app.{{ matrix_synapse_worker_details.type }}
worker_name: {{ matrix_synapse_worker_details.type ~ ':' ~ matrix_synapse_worker_details.port }}
worker_app: synapse.app.{{ matrix_synapse_worker_details.app }}
worker_name: {{ matrix_synapse_worker_details.name }}
worker_daemonize: false
worker_log_config: /data/{{ matrix_server_fqn_matrix }}.log.config
{% if matrix_synapse_replication_listener_enabled %}
worker_replication_host: matrix-synapse
worker_replication_http_port: {{ matrix_synapse_replication_http_port }}
{% endif %}
{% set has_listeners = (matrix_synapse_worker_details.type not in [ 'appservice', 'federation_sender', 'pusher' ] or matrix_synapse_metrics_enabled) %}
{% if matrix_synapse_worker_details.type == 'generic_worker' %}
worker_main_http_uri: http://matrix-synapse:{{ matrix_synapse_container_client_api_port }}
{% endif %}
{% set http_resources = [] %}
{% if matrix_synapse_worker_details.type in ['generic_worker', 'frontend_proxy', 'user_dir'] %}
{% if matrix_synapse_worker_details.type == 'user_dir' %}
{% set http_resources = http_resources + ['client'] %}
{% endif %}
{% if matrix_synapse_worker_details.type in ['generic_worker'] %}
{% set http_resources = http_resources+ ['federation'] %}
{% if matrix_synapse_worker_details.type == 'generic_worker' %}
{% set http_resources = http_resources + ['client', 'federation'] %}
{% endif %}
{% if matrix_synapse_worker_details.type in ['media_repository'] %}
{#
None of the background workers need to handle federation traffic.
Only some of the stream writers need to handle client traffic.
#}
{% if matrix_synapse_worker_details.type == 'stream_writer' and matrix_synapse_worker_details.webserving %}
{% set http_resources = http_resources + ['client'] %}
{% endif %}
{% if matrix_synapse_worker_details.type == 'media_repository' %}
{% set http_resources = http_resources + ['media'] %}
{% endif %}
{% if http_resources|length > 0 or matrix_synapse_metrics_enabled %}
{% set replication_http_resources = [] %}
{% if matrix_synapse_worker_details.type == 'stream_writer' %}
{# All background workers need to handle replication traffic. #}
{% set replication_http_resources = replication_http_resources + ['replication'] %}
{% endif %}
{% if http_resources|length > 0 or matrix_synapse_metrics_enabled or replication_http_resources|length > 0 %}
worker_listeners:
{% if http_resources|length > 0 %}
- type: http
@ -36,11 +54,11 @@ worker_listeners:
bind_addresses: ['0.0.0.0']
port: {{ matrix_synapse_worker_details.metrics_port }}
{% endif %}
{% if replication_http_resources|length > 0 %}
- type: http
bind_addresses: ['::']
port: {{ matrix_synapse_worker_details.replication_port }}
resources:
- names: {{ replication_http_resources|to_json }}
{% endif %}
{% if matrix_synapse_worker_details.type == 'frontend_proxy' %}
worker_main_http_uri: http://matrix-synapse:{{ matrix_synapse_container_client_api_port }}
{% endif %}
worker_daemonize: false
worker_log_config: /data/{{ matrix_server_fqn_matrix }}.log.config

View file

@ -36,3 +36,77 @@ matrix_synapse_workers_generic_worker_federation_endpoints: "{{ matrix_synapse_w
# matrix_synapse_workers_generic_worker_federation_endpoints_regex contains the regex used in matrix_synapse_workers_generic_worker_federation_endpoints.
# It's intentionally put in a separate variable, to avoid tripping ansible-lint's var-spacing rule.
matrix_synapse_workers_generic_worker_federation_endpoints_regex: '.*(/_matrix/federation|/_matrix/key).*'
# matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints contains the endpoints serviced by the `typing` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-typing-stream
matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints:
- ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/typing
# matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints contains the endpoints serviced by the `to_device` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-to_device-stream
matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/sendToDevice/
# matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints contains the endpoints serviced by the `account_data` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-account_data-stream
matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/.*/tags
- ^/_matrix/client/(r0|v3|unstable)/.*/account_data
# matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints contains the endpoints serviced by the `recepts` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-receipts-stream
matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/rooms/.*/receipt
- ^/_matrix/client/(r0|v3|unstable)/rooms/.*/read_markers
# matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints contains the endpoints serviced by the `presence` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-presence-stream
matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints:
- ^/_matrix/client/(api/v1|r0|v3|unstable)/presence/
# matrix_synapse_workers_user_dir_worker_client_server_endpoints contains the endpoints serviced by the `type = user_dir` (`app = generic_worker`) worker.
# See: https://matrix-org.github.io/synapse/latest/workers.html#updating-the-user-directory
matrix_synapse_workers_user_dir_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/user_directory/search$
# matrix_synapse_workers_known_stream_writer_stream_types contains the list of stream writer stream types that the playbook recognizes.
# This is used for validation purposes. If adding support for a new type, besides adding it to this list,
# don't forget to actually configure it where appropriate (see worker.yaml.j2`, the nginx proxy configuration, etc).
matrix_synapse_workers_known_stream_writer_stream_types: ['events', 'typing', 'to_device', 'account_data', 'receipts', 'presence']
# matrix_synapse_workers_webserving_stream_writer_types contains a list of stream writer types that serve web (client) requests.
# Not all stream writers serve web requests. Some just perform background tasks.
matrix_synapse_workers_webserving_stream_writer_types: ['typing', 'to_device', 'account_data', 'receipts', 'presence']
# matrix_synapse_workers_systemd_services_list contains a list of systemd services (one for each worker systemd service which serves web requests).
# This list is built during runtime.
# Not all workers serve web requests. Those that don't won't be injected here.
matrix_synapse_webserving_workers_systemd_services_list: []
# matrix_synapse_known_worker_types contains the list of known worker types.
#
# A worker type is different than a worker app (e.g. `generic_worker`).
# For example, the `stream_writer` worker type is served by the `generic_worker` app, but is a separate type that we recognize.
#
# Some other types (`appservice` and `user_dir`) used to be Synapse worker apps, which got subsequently deprecated.
# We still allow these types of workers and map them to the `generic_worker` app,
# which is why we make sure they're part of the list below.
# We use the `unique` filter because they're part of `matrix_synapse_workers_avail_list` too (for now; scheduled for removal).
matrix_synapse_known_worker_types: |
{{
(
matrix_synapse_workers_avail_list
+
['stream_writer']
+
['appservice']
+
['user_dir']
+
['background']
) | unique
}}
# matrix_synapse_known_instance_map_eligible_worker_types contains the list of worker types that are to be injected into `matrix_synapse_instance_map`.
matrix_synapse_known_instance_map_eligible_worker_types:
- stream_writer

View file

@ -55,10 +55,12 @@ matrix_synapse_workers_generic_worker_endpoints:
- ^/_matrix/client/(api/v1|r0|v3|unstable)/search$
# Encryption requests
# Note that ^/_matrix/client/(r0|v3|unstable)/keys/upload/ requires `worker_main_http_uri`
- ^/_matrix/client/(r0|v3|unstable)/keys/query$
- ^/_matrix/client/(r0|v3|unstable)/keys/changes$
- ^/_matrix/client/(r0|v3|unstable)/keys/claim$
- ^/_matrix/client/(r0|v3|unstable)/room_keys/
- ^/_matrix/client/(r0|v3|unstable)/keys/upload/
# Registration/login requests
- ^/_matrix/client/(api/v1|r0|v3|unstable)/login$
@ -88,7 +90,9 @@ matrix_synapse_workers_generic_worker_endpoints:
# - ^/_matrix/client/(api/v1|r0|v3|unstable)/presence/
# User directory search requests
- ^/_matrix/client/(r0|v3|unstable)/user_directory/search$
# Any worker can handle these, but we have a dedicated user_dir worker for this,
# so we'd like for other generic workers to not try and capture these requests.
# - ^/_matrix/client/(r0|v3|unstable)/user_directory/search$
# Additionally, the following REST endpoints can be handled for GET requests:
@ -172,7 +176,6 @@ matrix_synapse_workers_generic_worker_endpoints:
# Additionally, the writing of specific streams (such as events) can be moved off
# of the main process to a particular worker.
# (This is only supported with Redis-based replication.)
# To enable this, the worker must have a HTTP replication listener configured,
# have a `worker_name` and be listed in the `instance_map` config. The same worker
@ -432,35 +435,9 @@ matrix_synapse_workers_user_dir_endpoints:
# If `update_user_directory` is set to `false`, and this worker is not running,
# the above endpoint may give outdated results.
matrix_synapse_workers_frontend_proxy_endpoints:
# Proxies some frequently-requested client endpoints to add caching and remove
# load from the main synapse. It can handle REST endpoints matching the following
# regular expressions:
- ^/_matrix/client/(r0|v3|unstable)/keys/upload
# If `use_presence` is False in the homeserver config, it can also handle REST
# endpoints matching the following regular expressions:
# FIXME: ADDITIONAL CONDITIONS REQUIRED: to be enabled manually
# ^/_matrix/client/(api/v1|r0|v3|unstable)/presence/[^/]+/status
# This "stub" presence handler will pass through `GET` request but make the
# `PUT` effectively a no-op.
# It will proxy any requests it cannot handle to the main synapse instance. It
# must therefore be configured with the location of the main instance, via
# the `worker_main_http_uri` setting in the `frontend_proxy` worker configuration
# file. For example:
# ```yaml
# worker_main_http_uri: http://127.0.0.1:8008
# ```
matrix_synapse_workers_avail_list:
- appservice
- federation_sender
- frontend_proxy
- generic_worker
- media_repository
- pusher

View file

@ -46,6 +46,7 @@
- matrix-bot-postmoogle
- matrix-bot-go-neb
- matrix-bot-mjolnir
- matrix-cactus-comments
- matrix-synapse
- matrix-dendrite
- matrix-conduit