Add support for stream writer Synapse workers

As stream writer workers are also powered by the `generic_worker`
Synapse app, this necessitated that we provide means for distinguishing
between them and regular `generic_workers`.

I've also taken the time to optimize nginx configuration generation
(more Jinja2 macro usage, less duplication).

Worker names have also changed.
Workers are now named sequentially like this:
- `matrix-synapse-worker-0-generic`
- `matrix-synapse-worker-1-stream-writer-typing`
- `matrix-synapse-worker-2-pusher`

instead of `matrix-synapse-worker_generic_worker-18111` (indexed with a
port number).

People who modify `matrix_synapse_workers_enabled_list` directly will
need to adjust their configuration.
This commit is contained in:
Slavi Pantaleev 2022-09-15 07:05:25 +03:00
parent 99f4f5edc7
commit 226c550ffa
19 changed files with 449 additions and 121 deletions

View file

@ -1,3 +1,42 @@
# 2022-09-15
## (Potential Backward Compatibility Break) Major changes to Synapse workers
People who are interested in running a Synapse worker setup should know that **our Synapse worker implementation is much more powerful now**:
- we've added [Stream writers support](#stream-writers-support)
- see the [Potential Backward Incompatibilities after these Synapse worker changes](#potential-backward-incompatibilities-after-these-synapse-worker-changes)
### Stream writers support
From now on, the playbook lets you easily set up various [stream writer workers](https://matrix-org.github.io/synapse/latest/workers.html#stream-writers) which can handle different streams (`events` stream; `typing` URL endpoints, `to_device` URL endpoints, `account_data` URL endpoints, `receipts` URL endpoints, `presence` URL endpoints). All of this work was previously handled by the main Synapse process, but can now be offloaded to stream writer worker processes.
If you're using `matrix_synapse_workers_preset: one-of-each`, you'll automatically get 6 additional workers (one for each of the above stream types). Our `little-federation-helper` preset (meant to be quite minimal and focusing in improved federation performance) does not include stream writer workers.
If you'd like to customize the number of workers we also make that possible using these variables:
```yaml
# Synapse only supports more than 1 worker for the `events` stream.
# All other streams can utilize either 0 or 1 workers, not more than that.
matrix_synapse_workers_stream_writer_events_stream_workers_count: 5
matrix_synapse_workers_stream_writer_typing_stream_workers_count: 1
matrix_synapse_workers_stream_writer_to_device_stream_workers_count: 1
matrix_synapse_workers_stream_writer_account_data_stream_workers_count: 1
matrix_synapse_workers_stream_writer_receipts_stream_workers_count: 1
matrix_synapse_workers_stream_writer_presence_stream_workers_count: 1
```
### Potential Backward Incompatibilities after these Synapse worker changes
Below we'll discuss **potential backward incompatibilities**.
- **Worker names** (container names, systemd services, worker configuration files) **have changed**. Workers are now labeled sequentially (e.g. `matrix-synapse-worker_generic_worker-18111` -> `matrix-synapse-worker-generic-0`). The playbook will handle these changes automatically.
- **Metric endpoints have also changed** (`/metrics/synapse/worker/generic_worker-18111` -> `/metrics/synapse/worker/generic-worker-0`). If you're [collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server), consider revisiting our [Collecting Synapse worker metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-synapse-worker-metrics-to-an-external-prometheus-server) docs and updating your Prometheus configuration. **If you're collecting metrics to the integrated Prometheus server** (not enabled by default), **your Prometheus configuration will be updated automatically**. Old data (from before this change) may stick around though.
- **the format of `matrix_synapse_workers_enabled_list` has changed**. You were never advised to use this variable for directly creating workers (we advise people to control workers using `matrix_synapse_workers_preset` or by tweaking `matrix_synapse_workers_*_workers_count` variables only), but some people may have started using the `matrix_synapse_workers_enabled_list` variable to gain more control over workers. If you're one of them, you'll need to adjust its value. See `roles/matrix-synapse/defaults/main.yml` for more information on the new format. The playbook will also do basic validation and complain if you got something wrong.
# 2022-09-09
## Cactus Comments support

View file

@ -94,7 +94,7 @@ Note : The playbook will hash the basic_auth password for you on setup. Thus, yo
### Collecting Synapse worker metrics to an external Prometheus server
If you are using workers (`matrix_synapse_workers_enabled: true`) and have enabled `matrix_synapse_metrics_proxying_enabled` as described above, the playbook will also automatically expose all Synapse worker threads' metrics to `https://matrix.DOMAIN/metrics/synapse/worker/TYPE-ID`, where `TYPE` corresponds to the type and `ID` to the instanceId of a worker as exemplified in `matrix_synapse_workers_enabled_list`.
If you are using workers (`matrix_synapse_workers_enabled: true`) and have enabled `matrix_synapse_metrics_proxying_enabled` as described above, the playbook will also automatically expose all Synapse worker threads' metrics to `https://matrix.DOMAIN/metrics/synapse/worker/ID`, where `ID` corresponds to the worker `id` as exemplified in `matrix_synapse_workers_enabled_list`.
The playbook also generates an exemplary config file (`/matrix/synapse/external_prometheus.yml.template`) with all the correct paths which you can copy to your Prometheus server and adapt to your needs. Make sure to edit the specified `password_file` path and contents and path to your `synapse-v2.rules`.
It will look a bit like this:
@ -111,8 +111,8 @@ scrape_configs:
labels:
job: "master"
index: 1
- job_name: 'synapse-generic_worker-1'
metrics_path: /metrics/synapse/worker/generic_worker-18111
- job_name: 'matrix-synapse-synapse-worker-generic-worker-0'
metrics_path: /metrics/synapse/worker/generic-worker-0
scheme: https
basic_auth:
username: prometheus

View file

@ -1760,6 +1760,11 @@ matrix_nginx_proxy_synapse_workers_enabled: "{{ matrix_synapse_workers_enabled }
matrix_nginx_proxy_synapse_workers_list: "{{ matrix_synapse_workers_enabled_list }}"
matrix_nginx_proxy_synapse_generic_worker_client_server_locations: "{{ matrix_synapse_workers_generic_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_generic_worker_federation_locations: "{{ matrix_synapse_workers_generic_worker_federation_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_media_repository_locations: "{{matrix_synapse_workers_media_repository_endpoints|default([]) }}"
matrix_nginx_proxy_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_endpoints|default([]) }}"
@ -1767,6 +1772,8 @@ matrix_nginx_proxy_systemd_wanted_services_list: |
{{
['matrix-' + matrix_homeserver_implementation + '.service']
+
(matrix_synapse_webserving_workers_systemd_services_list if matrix_homeserver_implementation == 'synapse' and matrix_synapse_workers_enabled)
+
(['matrix-corporal.service'] if matrix_corporal_enabled else [])
+
(['matrix-ma1sd.service'] if matrix_ma1sd_enabled else [])

View file

@ -623,6 +623,11 @@ matrix_nginx_proxy_synapse_workers_enabled: false
matrix_nginx_proxy_synapse_workers_list: []
matrix_nginx_proxy_synapse_generic_worker_client_server_locations: []
matrix_nginx_proxy_synapse_generic_worker_federation_locations: []
matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_media_repository_locations: []
matrix_nginx_proxy_synapse_user_dir_locations: []

View file

@ -1,23 +1,52 @@
#jinja2: lstrip_blocks: "True"
{% set generic_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'generic_worker') | list %}
{% set stream_writer_typing_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'typing') | list %}
{% set stream_writer_to_device_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'to_device') | list %}
{% set stream_writer_account_data_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'account_data') | list %}
{% set stream_writer_receipts_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'receipts') | list %}
{% set stream_writer_presence_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'presence') | list %}
{% set media_repository_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'media_repository') | list %}
{% set user_dir_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'user_dir') | list %}
{% macro render_worker_upstream(name, workers, matrix_nginx_proxy_enabled) %}
{% if workers | length > 0 %}
upstream {{ name }} {
{% for worker in workers %}
{% if matrix_nginx_proxy_enabled %}
server "{{ worker.name }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{% endmacro %}
{% macro render_locations_to_upstream(locations, upstream_name) %}
{% for location in locations %}
location ~ {{ location }} {
proxy_pass http://{{ upstream_name }}$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% endmacro %}
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{% if matrix_nginx_proxy_synapse_cache_enabled %}
proxy_cache_path {{ matrix_nginx_proxy_synapse_cache_path }} levels=1:2 keys_zone={{ matrix_nginx_proxy_synapse_cache_keys_zone_name }}:{{ matrix_nginx_proxy_synapse_cache_keys_zone_size }} inactive={{ matrix_nginx_proxy_synapse_cache_inactive_time }} max_size={{ matrix_nginx_proxy_synapse_cache_max_size_mb }}m;
{% endif %}
# Round Robin "upstream" pools for workers
{% if generic_workers %}
upstream generic_worker_upstream {
{% if generic_workers |length > 0 %}
upstream generic_workers_upstream {
# ensures that requests from the same client will always be passed
# to the same server (except when this server is unavailable)
hash $http_x_forwarded_for;
{% for worker in generic_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
server "{{ worker.name }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
@ -25,29 +54,15 @@
}
{% endif %}
{% if media_repository_workers %}
upstream media_repository_upstream {
{% for worker in media_repository_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('stream_writer_typing_stream_workers_upstream', stream_writer_typing_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_to_device_stream_workers_upstream', stream_writer_to_device_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_account_data_stream_workers_upstream', stream_writer_account_data_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_receipts_stream_workers_upstream', stream_writer_receipts_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_presence_stream_workers_upstream', stream_writer_presence_stream_workers, matrix_nginx_proxy_enabled) }}
{% if user_dir_workers %}
upstream user_dir_upstream {
{% for worker in user_dir_workers %}
{% if matrix_nginx_proxy_enabled %}
server "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('media_repository_workers_upstream', media_repository_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('user_dir_workers_upstream', user_dir_workers, matrix_nginx_proxy_enabled) }}
{% endif %}
server {
@ -65,21 +80,41 @@ server {
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{# Workers redirects BEGIN #}
{% if generic_workers %}
{% if generic_workers | length > 0 %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappgeneric_worker
{% for location in matrix_nginx_proxy_synapse_generic_worker_client_server_locations %}
location ~ {{ location }} {
proxy_pass http://generic_worker_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_generic_worker_client_server_locations, 'generic_workers_upstream') }}
{% endif %}
{% if media_repository_workers %}
{% if stream_writer_typing_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-typing-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations, 'stream_writer_typing_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_to_device_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-to_device-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations, 'stream_writer_to_device_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_account_data_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-account_data-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations, 'stream_writer_account_data_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_receipts_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-receipts-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations, 'stream_writer_receipts_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_presence_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-presence-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations, 'stream_writer_presence_stream_workers_upstream') }}
{% endif %}
{% if media_repository_workers | length > 0 %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappmedia_repository
{% for location in matrix_nginx_proxy_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_upstream$request_uri;
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;
@ -97,15 +132,10 @@ server {
{% endfor %}
{% endif %}
{% if user_dir_workers %}
{% if user_dir_workers | length > 0 %}
# FIXME: obsolete if matrix_nginx_proxy_proxy_matrix_user_directory_search_enabled is set
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappuser_dir
{% for location in matrix_nginx_proxy_synapse_user_dir_locations %}
location ~ {{ location }} {
proxy_pass http://user_dir_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_user_dir_locations, 'user_dir_workers_upstream') }}
{% endif %}
{# Workers redirects END #}
{% endif %}
@ -150,20 +180,15 @@ server {
gzip_types text/plain application/json;
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{% if generic_workers %}
{% if generic_workers | length > 0 %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappgeneric_worker
{% for location in matrix_nginx_proxy_synapse_generic_worker_federation_locations %}
location ~ {{ location }} {
proxy_pass http://generic_worker_upstream$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_generic_worker_federation_locations, 'generic_workers_upstream') }}
{% endif %}
{% if media_repository_workers %}
{% if media_repository_workers | length > 0 %}
# https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappmedia_repository
{% for location in matrix_nginx_proxy_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_upstream$request_uri;
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;

View file

@ -37,11 +37,12 @@ scrape_configs:
index: 0
{% for worker in matrix_prometheus_scraper_synapse_workers_enabled_list %}
{% if worker.metrics_port != 0 %}
- targets: ['matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.metrics_port }}']
- targets: ['{{ worker.name }}:{{ worker.metrics_port }}']
labels:
instance: {{ matrix_domain }}
worker_id: {{ worker.id }}
job: {{ worker.type }}
index: {{ worker.instanceId }}
app: {{ worker.app }}
{% endif %}
{% endfor %}
{% endif %}

View file

@ -398,6 +398,12 @@ matrix_synapse_workers_presets:
federation_sender_workers_count: 1
media_repository_workers_count: 0
user_dir_workers_count: 0
stream_writer_events_stream_workers_count: 0
stream_writer_typing_stream_workers_count: 0
stream_writer_to_device_stream_workers_count: 0
stream_writer_account_data_stream_workers_count: 0
stream_writer_receipts_stream_workers_count: 0
stream_writer_presence_stream_workers_count: 0
one-of-each:
generic_workers_count: 1
pusher_workers_count: 1
@ -409,6 +415,12 @@ matrix_synapse_workers_presets:
# user_dir workers are deprecated since Synapse v1.59. This will be removed.
# See: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
user_dir_workers_count: 0
stream_writer_events_stream_workers_count: 1
stream_writer_typing_stream_workers_count: 1
stream_writer_to_device_stream_workers_count: 1
stream_writer_account_data_stream_workers_count: 1
stream_writer_receipts_stream_workers_count: 1
stream_writer_presence_stream_workers_count: 1
# Controls whether the matrix-synapse container exposes the various worker ports
# (see `port` and `metrics_port` in `matrix_synapse_workers_enabled_list`) outside of the container.
@ -421,6 +433,71 @@ matrix_synapse_workers_generic_workers_count: "{{ matrix_synapse_workers_presets
matrix_synapse_workers_generic_workers_port_range_start: 18111
matrix_synapse_workers_generic_workers_metrics_range_start: 19111
# matrix_synapse_workers_stream_writer_events_stream_workers_count controls how many stream writers that handle the `events` stream to spawn.
# More than 1 worker is also supported of this type.
matrix_synapse_workers_stream_writer_events_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_events_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_typing_stream_workers_count controls how many stream writers that handle the `typing` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_typing_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_typing_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_to_device_stream_workers_count controls how many stream writers that handle the `to_device` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_to_device_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_to_device_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_account_data_stream_workers_count controls how many stream writers that handle the `account_data` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_account_data_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_account_data_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_receipts_stream_workers_count controls how many stream writers that handle the `receipts` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_receipts_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_receipts_stream_workers_count'] }}"
# matrix_synapse_workers_stream_writer_presence_stream_workers_count controls how many stream writers that handle the `presence` stream to spawn.
# The count of these workers can only be 0 or 1.
matrix_synapse_workers_stream_writer_presence_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_presence_stream_workers_count'] }}"
# A list of stream writer workers to enable. This list is built automatically based on other variables.
# You're encouraged to enable/disable stream writer workers by setting `matrix_synapse_workers_stream_writer_*_stream_workers_count` variables, instead of adjusting this list manually.
matrix_synapse_workers_stream_writers: |
{{
[]
+
([{'stream': 'events'}] * matrix_synapse_workers_stream_writer_events_stream_workers_count | int)
+
([{'stream': 'typing'}] * matrix_synapse_workers_stream_writer_typing_stream_workers_count | int)
+
([{'stream': 'to_device'}] * matrix_synapse_workers_stream_writer_to_device_stream_workers_count | int)
+
([{'stream': 'account_data'}] * matrix_synapse_workers_stream_writer_account_data_stream_workers_count | int)
+
([{'stream': 'receipts'}] * matrix_synapse_workers_stream_writer_receipts_stream_workers_count | int)
+
([{'stream': 'presence'}] * matrix_synapse_workers_stream_writer_presence_stream_workers_count | int)
}}
# matrix_synapse_stream_writers populates the `stream_writers` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# What you see below is an initial default value which will be adjusted at runtime based on the value of `matrix_synapse_workers_stream_writers`.
# Adjusting this value manually is generally not necessary.
#
# It's tempting to initialize this like this:
# matrix_synapse_stream_writers:
# - typing: []
# - events: []
# - to_device: []
# - account_data: []
# - receipts: []
# - presence: []
# .. but Synapse does not like empty lists (see https://github.com/matrix-org/synapse/issues/13804)
matrix_synapse_stream_writers: {}
# `matrix_synapse_workers_stream_writer_workers_` variables control the port numbers of various stream writer workers
# defined in `matrix_synapse_workers_stream_writers`.
# It should be noted that not all of the background worker types will need to expose HTTP services, etc.
matrix_synapse_workers_stream_writer_workers_http_port_range_start: 20011
matrix_synapse_workers_stream_writer_workers_replication_port_range_start: 25011
matrix_synapse_workers_stream_writer_workers_metrics_range_start: 19111
# matrix_synapse_workers_pusher_workers_count can only be 0 or 1 for now.
# More instances are not supported due to a playbook limitation having to do with keeping `pusher_instances` in `homeserver.yaml` updated.
# See https://github.com/matrix-org/synapse/commit/ddfdf945064925eba761ae3748e38f3a1c73c328
@ -463,25 +540,36 @@ matrix_synapse_workers_user_dir_workers_metrics_range_start: 19661
# as certain workers can only be spawned just once.
#
# Each worker instance in the list defines the following fields:
# - `type` - the type of worker (`generic_worker`, etc.)
# - `instanceId` - a string that identifies the worker. The combination of (`type` + `instanceId`) represents the name of the worker and must be unique.
# - `id` - a string that uniquely identifies the worker
# - `name` - a string that will be used as the container and systemd service name
# - `type` - the type of worker (`generic_worker`, `stream_writer`, `pusher`, etc.)
# - `app` - the Synapse app (https://matrix-org.github.io/synapse/latest/workers.html#available-worker-applications) that powers this worker (`generic_worker`, `federation_sender`, etc.).
# The `app` usually matches the `type`, but not always. For example, `type = stream_writer` workers are served by the `generic_worker` type.
# - `port` - an HTTP port where the worker listens for requests (can be `0` for workers that don't do HTTP request processing)
# - `metrics_port` - an HTTP port where the worker exports Prometheus metrics
# - `replication_port` - an HTTP port where the worker serves `replication` endpoints (used by stream writers, etc.)
# - `webserving` - tells whether this type of worker serves web (client or federation) requests, so that it can be injected as a dependency to the reverse-proxy
#
# Example of what this needs to look like, if you're defining it manually:
# matrix_synapse_workers_enabled_list:
# - { type: generic_worker, instanceId: '18111', port: 18111, metrics_port: 19111 }
# - { type: generic_worker, instanceId: '18112', port: 18112, metrics_port: 19112 }
# - { type: generic_worker, instanceId: '18113', port: 18113, metrics_port: 19113 }
# - { type: generic_worker, instanceId: '18114', port: 18114, metrics_port: 19114 }
# - { type: generic_worker, instanceId: '18115', port: 18115, metrics_port: 19115 }
# - { type: generic_worker, instanceId: '18116', port: 18116, metrics_port: 19116 }
# - { type: pusher, instanceId: '0', port: 0, metrics_port: 19200 }
# - { type: appservice, instanceId: '0', port: 0, metrics_port: 19300 }
# - { type: federation_sender, instanceId: '0', port: 0, metrics_port: 19400 }
# - { type: media_repository, instanceId: '18551', port: 18551, metrics_port: 19551 }
# - { 'id': 'generic-worker-0', 'name': 'matrix-synapse-worker-generic-0', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18111, 'metrics_port': 19111, 'webserving': true }
# - { 'id': 'generic-worker-1', 'name': 'matrix-synapse-worker-generic-1', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18112, 'metrics_port': 19112, 'webserving': true }
# - { 'id': 'generic-worker-2', 'name': 'matrix-synapse-worker-generic-2', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18113, 'metrics_port': 19113, 'webserving': true }
# - { 'id': 'generic-worker-3', 'name': 'matrix-synapse-worker-generic-3', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18114, 'metrics_port': 19114, 'webserving': true }
# - { 'id': 'generic-worker-4', 'name': 'matrix-synapse-worker-generic-4', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18115, 'metrics_port': 19115, 'webserving': true }
# - { 'id': 'generic-worker-5', 'name': 'matrix-synapse-worker-generic-5', 'type': 'generic_worker', 'app': 'generic_worker', 'port': 18116, 'metrics_port': 19116, 'webserving': true }
# - { 'id': 'stream-writer-0-events', 'name': 'matrix-synapse-worker-stream-writer-0-events', 'type': 'stream_writer', 'app': 'generic_worker', 'stream_writer_stream': 'events', 'port': 0, 'replication_port': 25011, metrics_port: 19111, 'webserving': false }
# - { 'id': 'stream-writer-1-typing', 'name': 'matrix-synapse-worker-stream-writer-1-typing', 'type': 'stream_writer', 'app': 'generic_worker', 'stream_writer_stream': 'typing', 'port': 20012, 'replication_port': 25012, metrics_port: 19112, 'webserving': true }
# - { 'id': 'pusher-0', 'name': 'matrix-synapse-worker-pusher-0', 'type': 'pusher', 'app': 'pusher', 'port': 0, 'metrics_port': 19200, 'webserving': false }
# - { 'id': 'appservice-0', 'name': 'matrix-synapse-worker-appservice-0', 'type': 'appservice', 'port': 0, 'metrics_port': 19300, 'webserving': false }
# - { 'id': 'federation-sender-0', 'name': 'matrix-synapse-worker-federation-sender-0', 'type': 'federation_sender', 'port': 0, 'metrics_port': 19400, 'webserving': false }
# - { 'id': 'media-repository-0', 'name': 'matrix-synapse-worker-media-repository-0', 'type': 'media_repository', 'port': 18551, 'metrics_port': 19551, 'webserving': true }
matrix_synapse_workers_enabled_list: []
# matrix_synapse_instance_map holds the instance map used for mapping worker names (for certain generic workers only!) to where they live (host, port which handles replication traffic).
# This is populated automatically based on `matrix_synapse_workers_enabled_list` during runtime, so you're not required to tweak it manually.
matrix_synapse_instance_map: {}
# Redis information
matrix_synapse_redis_enabled: false
matrix_synapse_redis_host: ""

View file

@ -8,15 +8,15 @@
# Unless `matrix_synapse_workers_enabled_list` is explicitly defined,
# we'll generate it dynamically.
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/synapse/workers/init.yml"
- ansible.builtin.include_tasks: "{{ role_path }}/tasks/synapse/workers/init.yml"
when: "matrix_synapse_enabled and matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list|length == 0"
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-synapse.service'] }}"
when: matrix_synapse_enabled | bool
- name: Ensure systemd services for workers are injected
ansible.builtin.include_tasks: "{{ role_path }}/tasks/synapse/workers/util/inject_systemd_services_for_worker.yml"
- name: Ensure workers are injected into various places
ansible.builtin.include_tasks: "{{ role_path }}/tasks/synapse/workers/util/inject_worker.yml"
with_items: "{{ matrix_synapse_workers_enabled_list }}"
loop_control:
loop_var: matrix_synapse_worker_details
@ -65,9 +65,9 @@
matrix_synapse_worker_nginx_metrics_configuration_block: |
{% for worker in matrix_synapse_workers_enabled_list %}
{% if worker.metrics_port != 0 %}
location /metrics/synapse/worker/{{ worker.type }}-{{ worker.instanceId }} {
location /metrics/synapse/worker/{{ worker.id }} {
resolver 127.0.0.11 valid=5s;
set $backend "matrix-synapse-worker-{{ worker.type }}-{{ worker.instanceId }}:{{ worker.metrics_port }}";
set $backend "{{ worker.name }}:{{ worker.metrics_port }}";
proxy_pass http://$backend/_synapse/metrics;
proxy_set_header Host $host;
}

View file

@ -7,18 +7,47 @@
- name: Build generic workers
ansible.builtin.set_fact:
worker:
id: "generic-worker-{{ item }}"
name: "matrix-synapse-worker-generic-{{ item }}"
type: 'generic_worker'
instanceId: "{{ matrix_synapse_workers_generic_workers_port_range_start + item }}"
app: 'generic_worker'
webserving: true
port: "{{ matrix_synapse_workers_generic_workers_port_range_start + item }}"
metrics_port: "{{ matrix_synapse_workers_generic_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_generic_workers"
loop: "{{ range(0, matrix_synapse_workers_generic_workers_count | int) | list }}"
- name: Build stream writer workers
ansible.builtin.set_fact:
worker:
id: "stream-writer-{{ item }}-{{ item.stream }}"
# Names must not include understores. Certain stream writer streams (to_device, account_data, ..) do, so we fix them up.
name: "matrix-synapse-worker-stream-writer-{{ idx }}-{{ item.stream | replace('_', '-') }}"
type: 'stream_writer'
app: "generic_worker"
webserving: "{{ item.stream in matrix_synapse_workers_webserving_stream_writer_types }}"
stream_writer_stream: "{{ item.stream }}"
port: "{{ matrix_synapse_workers_stream_writer_workers_http_port_range_start + idx }}"
replication_port: "{{ matrix_synapse_workers_stream_writer_workers_replication_port_range_start + idx }}"
metrics_port: "{{ matrix_synapse_workers_stream_writer_workers_metrics_range_start + idx }}"
register: "matrix_synapse_workers_list_results_stream_writer_workers"
loop: "{{ matrix_synapse_workers_stream_writers }}"
loop_control:
index_var: idx
- name: Populate matrix_synapse_stream_writers from enabled stream writer workers list
ansible.builtin.set_fact:
matrix_synapse_stream_writers: "{{ matrix_synapse_stream_writers | combine ({item.ansible_facts.worker.stream_writer_stream: [item.ansible_facts.worker.name]}) }}"
with_items: "{{ matrix_synapse_workers_list_results_stream_writer_workers.results }}"
- name: Build federation sender workers
ansible.builtin.set_fact:
worker:
id: "federation-sender-{{ item }}"
name: "matrix-synapse-worker-federation-sender-{{ item }}"
type: 'federation_sender'
instanceId: "{{ item }}"
app: 'federation_sender'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_federation_sender_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_federation_sender_workers"
@ -28,8 +57,11 @@
- name: Build pusher workers
ansible.builtin.set_fact:
worker:
id: "pusher-{{ item }}"
name: "matrix-synapse-worker-pusher-{{ item }}"
type: 'pusher'
instanceId: "{{ item }}"
app: 'pusher'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_pusher_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_pusher_workers"
@ -39,8 +71,11 @@
- name: Build appservice workers
ansible.builtin.set_fact:
worker:
id: "appservice-{{ item }}"
name: "matrix-synapse-worker-appservice-{{ item }}"
type: 'appservice'
instanceId: "{{ item }}"
app: 'appservice'
webserving: false
port: 0
metrics_port: "{{ matrix_synapse_workers_appservice_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_appservice_workers"
@ -49,8 +84,11 @@
- name: Build media_repository workers
ansible.builtin.set_fact:
worker:
id: "media-repository-{{ item }}"
name: "matrix-synapse-worker-media-repository-{{ item }}"
type: 'media_repository'
instanceId: "{{ matrix_synapse_workers_media_repository_workers_port_range_start + item }}"
app: 'media_repository'
webserving: true
port: "{{ matrix_synapse_workers_media_repository_workers_port_range_start + item }}"
metrics_port: "{{ matrix_synapse_workers_media_repository_workers_metrics_range_start + item }}"
register: "matrix_synapse_workers_list_results_media_repository_workers"
@ -62,6 +100,8 @@
{{
matrix_synapse_workers_list_results_generic_workers.results
+
matrix_synapse_workers_list_results_stream_writer_workers.results
+
matrix_synapse_workers_list_results_federation_sender_workers.results
+
matrix_synapse_workers_list_results_pusher_workers.results

View file

@ -1,19 +0,0 @@
---
# The tasks below run before `validate_config.yml`.
# To avoid failing with a cryptic error message, we'll do validation here.
#
# This check is mostly relevant to people who explicitly define `matrix_synapse_workers_enabled_list`
# (Synapse Workers users from the earlier days of this PR - https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/456).
#
# In the future, it should be possible to remove this check.
# Our own code which dynamically builds `matrix_synapse_workers_enabled_list` does things right.
- name: Fail if instanceId not defined for worker
ansible.builtin.fail:
msg: "Synapse workers (like {{ matrix_synapse_worker_details | to_json }}) need to define an instanceId property (type + instanceId must be unique)"
when: "'instanceId' not in matrix_synapse_worker_details"
- ansible.builtin.set_fact:
matrix_synapse_worker_systemd_service_name: "matrix-synapse-worker-{{ matrix_synapse_worker_details.type }}-{{ matrix_synapse_worker_details.instanceId }}.service"
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + [matrix_synapse_worker_systemd_service_name] }}"

View file

@ -0,0 +1,65 @@
---
# The tasks below run before `validate_config.yml`.
# To avoid failing with a cryptic error message, we'll do validation here.
#
# This check is mostly relevant to people who explicitly define `matrix_synapse_workers_enabled_list`
# (Synapse Workers users from the earlier days of this PR - https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/456).
#
# In the future, it should be possible to remove this check.
# Our own code which dynamically builds `matrix_synapse_workers_enabled_list` does things right.
- name: Fail if required property not defined for worker
ansible.builtin.fail:
msg: "Synapse workers (like {{ matrix_synapse_worker_details | to_json }}) need to define a `{{ item }}` property"
with_items:
- id
- name
- type
- app
- port
- webserving
when: "item not in matrix_synapse_worker_details"
# Names are used for container names and systemd services.
# Routing happens based on container names, so Synapse processes that try to route to workers with underscores in the name will complain. Example:
# > InvalidCodepoint Codepoint U+005F at position 46 of 'matrix-synapse-worker-stream-writer-3-account_data' not allowed
- name: Fail if worker name includes underscore
ansible.builtin.fail:
msg: "Unrecognized Synapse worker `name`: `{{ matrix_synapse_worker_details.name }}`. It must not include underscores"
when: "'_' in matrix_synapse_worker_details.name"
- name: Fail if worker type unknown
ansible.builtin.fail:
msg: "Unrecognized Synapse worker `type`: `{{ matrix_synapse_worker_details.type }}`. Supported types are: {{ matrix_synapse_known_worker_types | join(', ') }}"
when: "matrix_synapse_worker_details.type not in matrix_synapse_known_worker_types"
- name: Fail if worker app unknown
ansible.builtin.fail:
msg: "Unrecognized Synapse worker `app`: `{{ matrix_synapse_worker_details.app }}`. Supported types are: {{ matrix_synapse_workers_avail_list | join(', ') }}"
when: "matrix_synapse_worker_details.app not in matrix_synapse_workers_avail_list"
- block:
- name: Fail if stream_writer_stream not defined for stream_writer worker
ansible.builtin.fail:
msg: >-
Synapse stream_writer workers (such as {{ item }}) need to define a valid `stream_writer_stream` property
(not `{{ matrix_synapse_worker_details.stream_writer_stream|default('undefined') }}`).
Supported types are: {{ matrix_synapse_workers_known_stream_writer_stream_types | join(', ') }}
when: "'stream_writer_stream' not in matrix_synapse_worker_details or matrix_synapse_worker_details.stream_writer_stream not in matrix_synapse_workers_known_stream_writer_stream_types"
- name: Fail if replication_port not defined for stream_writer worker
ansible.builtin.fail:
msg: "Synapse background workers of type stream_writer (such as {{ item }}) need to define a valid `replication_port` property"
when: "'replication_port' not in matrix_synapse_worker_details"
when: "matrix_synapse_worker_details.type == 'stream_writer'"
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + [matrix_synapse_worker_details.name + '.service'] }}"
- ansible.builtin.set_fact:
matrix_synapse_webserving_workers_systemd_services_list: "{{ matrix_synapse_webserving_workers_systemd_services_list + [matrix_synapse_worker_details.name + '.service'] }}"
when: matrix_synapse_worker_details.webserving | bool
# Inject stream writers and various other background workers into the instance map.
- ansible.builtin.set_fact:
matrix_synapse_instance_map: "{{ matrix_synapse_instance_map | combine({matrix_synapse_worker_details.name: {'host': matrix_synapse_worker_details.name, 'port': matrix_synapse_worker_details.replication_port}}) }}"
when: matrix_synapse_worker_details.type in matrix_synapse_known_instance_map_eligible_worker_types

View file

@ -1,13 +1,9 @@
---
- ansible.builtin.set_fact:
matrix_synapse_worker_systemd_service_name: "matrix-synapse-worker-{{ matrix_synapse_worker_details.type }}-{{ matrix_synapse_worker_details.instanceId }}"
- ansible.builtin.set_fact:
matrix_synapse_worker_container_name: "{{ matrix_synapse_worker_systemd_service_name }}"
- ansible.builtin.set_fact:
matrix_synapse_worker_config_file_name: "worker.{{ matrix_synapse_worker_details.type }}_{{ matrix_synapse_worker_details.instanceId }}.yaml"
matrix_synapse_worker_systemd_service_name: "{{ matrix_synapse_worker_details.name }}"
matrix_synapse_worker_container_name: "{{ matrix_synapse_worker_details.name }}"
matrix_synapse_worker_config_file_name: "worker.{{ matrix_synapse_worker_details.name }}.yaml"
- name: Ensure configuration exists for {{ matrix_synapse_worker_systemd_service_name }}
ansible.builtin.template:

View file

@ -32,6 +32,11 @@
with_items:
- "matrix_synapse_workers_pusher_workers_count"
- "matrix_synapse_workers_federation_sender_workers_count"
- "matrix_synapse_workers_stream_writer_typing_stream_workers_count"
- "matrix_synapse_workers_stream_writer_to_device_stream_workers_count"
- "matrix_synapse_workers_stream_writer_account_data_stream_workers_count"
- "matrix_synapse_workers_stream_writer_receipts_stream_workers_count"
- "matrix_synapse_workers_stream_writer_presence_stream_workers_count"
- name: (Deprecation) Catch and report renamed settings
ansible.builtin.fail:

View file

@ -352,13 +352,13 @@ worker_app: synapse.app.homeserver
# thx https://oznetnerd.com/2017/04/18/jinja2-selectattr-filter/
# reduce the main worker's offerings to core homeserver business
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list | length > 0 %}
send_federation: false
{% endif %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list | length > 0 %}
enable_media_repo: false
{% endif %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list | length > 0 %}
start_pushers: false
{% endif %}
@ -2870,6 +2870,7 @@ opentracing:
# worker1:
# host: localhost
# port: 8034
instance_map: {{ matrix_synapse_instance_map | to_json }}
# Experimental: When using workers you can define which workers should
# handle event persistence and typing notifications. Any worker
@ -2878,6 +2879,7 @@ opentracing:
#stream_writers:
# events: worker1
# typing: worker1
stream_writers: {{ matrix_synapse_stream_writers | to_json }}
# The worker that is used to run background tasks (e.g. cleaning up expired
# data). If not provided this defaults to the main process.

View file

@ -24,8 +24,8 @@ scrape_configs:
job: "master"
index: "0"
{% for worker in matrix_synapse_workers_enabled_list %}
- job_name: 'synapse-{{ worker.type }}-{{ worker.instanceId }}'
metrics_path: /metrics/synapse/worker/{{ worker.type }}-{{ worker.instanceId }}
- job_name: '{{ worker.name }}'
metrics_path: /metrics/synapse/worker/{{ worker.id }}
scheme: {{ 'https' if matrix_nginx_proxy_https_enabled|default(true) else 'http' }}
{% if matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled|default(true) %}
basic_auth:
@ -35,6 +35,7 @@ scrape_configs:
static_configs:
- targets: ['{{ matrix_server_fqn_matrix }}:{{ matrix_nginx_proxy_container_https_host_bind_port|default(443) if matrix_nginx_proxy_https_enabled|default(true) else matrix_nginx_proxy_container_http_host_bind_port|default(80) }}']
labels:
worker_id: {{ worker.id }}
job: "{{ worker.type }}"
index: "{{ worker.instanceId }}"
app: {{ worker.app }}
{% endfor %}

View file

@ -43,7 +43,7 @@ ExecStart={{ matrix_host_command_docker }} run --rm --name {{ matrix_synapse_wor
{{ arg }} \
{% endfor %}
{{ matrix_synapse_docker_image }} \
run -m synapse.app.{{ matrix_synapse_worker_details.type }} -c /data/homeserver.yaml -c /data/{{ matrix_synapse_worker_config_file_name }}
run -m synapse.app.{{ matrix_synapse_worker_details.app }} -c /data/homeserver.yaml -c /data/{{ matrix_synapse_worker_config_file_name }}
ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill {{ matrix_synapse_worker_container_name }} 2>/dev/null || true'

View file

@ -12,7 +12,7 @@ Wants={{ service }}
{% if matrix_synapse_workers_enabled %}
{% for matrix_synapse_worker_details in matrix_synapse_workers_enabled_list %}
Wants=matrix-synapse-worker-{{ matrix_synapse_worker_details.type }}-{{ matrix_synapse_worker_details.port }}.service
Wants={{ matrix_synapse_worker_details.name }}.service
{% endfor %}
{% endif %}

View file

@ -1,27 +1,38 @@
#jinja2: lstrip_blocks: "True"
worker_app: synapse.app.{{ matrix_synapse_worker_details.type }}
worker_name: {{ matrix_synapse_worker_details.type ~ ':' ~ matrix_synapse_worker_details.port }}
worker_app: synapse.app.{{ matrix_synapse_worker_details.app }}
worker_name: {{ matrix_synapse_worker_details.name }}
{% if matrix_synapse_replication_listener_enabled %}
worker_replication_host: matrix-synapse
worker_replication_http_port: {{ matrix_synapse_replication_http_port }}
{% endif %}
{% set has_listeners = (matrix_synapse_worker_details.type not in [ 'appservice', 'federation_sender', 'pusher' ] or matrix_synapse_metrics_enabled) %}
{% set http_resources = [] %}
{% if matrix_synapse_worker_details.type in ['generic_worker', 'user_dir'] %}
{% if matrix_synapse_worker_details.type == 'user_dir' %}
{% set http_resources = http_resources + ['client'] %}
{% endif %}
{% if matrix_synapse_worker_details.type in ['generic_worker'] %}
{% set http_resources = http_resources+ ['federation'] %}
{% if matrix_synapse_worker_details.type == 'generic_worker' %}
{% set http_resources = http_resources + ['client', 'federation'] %}
{% endif %}
{% if matrix_synapse_worker_details.type in ['media_repository'] %}
{#
None of the background workers need to handle federation traffic.
Only some of the stream writers need to handle client traffic.
#}
{% if matrix_synapse_worker_details.type == 'stream_writer' and matrix_synapse_worker_details.webserving %}
{% set http_resources = http_resources + ['client'] %}
{% endif %}
{% if matrix_synapse_worker_details.type == 'media_repository' %}
{% set http_resources = http_resources + ['media'] %}
{% endif %}
{% if http_resources|length > 0 or matrix_synapse_metrics_enabled %}
{% set replication_http_resources = [] %}
{% if matrix_synapse_worker_details.type == 'stream_writer' %}
{# All background workers need to handle replication traffic. #}
{% set replication_http_resources = replication_http_resources + ['replication'] %}
{% endif %}
{% if http_resources|length > 0 or matrix_synapse_metrics_enabled or replication_http_resources|length > 0 %}
worker_listeners:
{% if http_resources|length > 0 %}
- type: http
@ -36,6 +47,13 @@ worker_listeners:
bind_addresses: ['0.0.0.0']
port: {{ matrix_synapse_worker_details.metrics_port }}
{% endif %}
{% if replication_http_resources|length > 0 %}
- type: http
bind_addresses: ['::']
port: {{ matrix_synapse_worker_details.replication_port }}
resources:
- names: {{ replication_http_resources|to_json }}
{% endif %}
{% endif %}
{% if matrix_synapse_worker_details.type == 'generic_worker' %}

View file

@ -36,3 +36,58 @@ matrix_synapse_workers_generic_worker_federation_endpoints: "{{ matrix_synapse_w
# matrix_synapse_workers_generic_worker_federation_endpoints_regex contains the regex used in matrix_synapse_workers_generic_worker_federation_endpoints.
# It's intentionally put in a separate variable, to avoid tripping ansible-lint's var-spacing rule.
matrix_synapse_workers_generic_worker_federation_endpoints_regex: '.*(/_matrix/federation|/_matrix/key).*'
# matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints contains the endpoints serviced by the `typing` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-typing-stream
matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints:
- ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/typing
# matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints contains the endpoints serviced by the `to_device` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-to_device-stream
matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/sendToDevice/
# matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints contains the endpoints serviced by the `account_data` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-account_data-stream
matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/.*/tags
- ^/_matrix/client/(r0|v3|unstable)/.*/account_data
# matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints contains the endpoints serviced by the `recepts` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-receipts-stream
matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints:
- ^/_matrix/client/(r0|v3|unstable)/rooms/.*/receipt
- ^/_matrix/client/(r0|v3|unstable)/rooms/.*/read_markers
# matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints contains the endpoints serviced by the `presence` stream writer.
# See: https://matrix-org.github.io/synapse/latest/workers.html#the-presence-stream
matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints:
- ^/_matrix/client/(api/v1|r0|v3|unstable)/presence/
# matrix_synapse_workers_known_stream_writer_stream_types contains the list of stream writer stream types that the playbook recognizes.
# This is used for validation purposes. If adding support for a new type, besides adding it to this list,
# don't forget to actually configure it where appropriate (see worker.yaml.j2`, the nginx proxy configuration, etc).
matrix_synapse_workers_known_stream_writer_stream_types: ['events', 'typing', 'to_device', 'account_data', 'receipts', 'presence']
# matrix_synapse_workers_webserving_stream_writer_types contains a list of stream writer types that serve web (client) requests.
# Not all stream writers serve web requests. Some just perform background tasks.
matrix_synapse_workers_webserving_stream_writer_types: ['typing', 'to_device', 'account_data', 'receipts', 'presence']
# matrix_synapse_workers_systemd_services_list contains a list of systemd services (one for each worker systemd service which serves web requests).
# This list is built during runtime.
# Not all workers serve web requests. Those that don't won't be injected here.
matrix_synapse_webserving_workers_systemd_services_list: []
# matrix_synapse_known_worker_types contains the list of known worker types.
# A worker type is different than a worker app (e.g. `generic_worker`).
# For example, the `stream_writer` worker type is served by the `generic_worker` app, but is a separate type that we recognize.
matrix_synapse_known_worker_types: |
{{
matrix_synapse_workers_avail_list
+
['stream_writer']
}}
# matrix_synapse_known_instance_map_eligible_worker_types contains the list of worker types that are to be injected into `matrix_synapse_instance_map`.
matrix_synapse_known_instance_map_eligible_worker_types:
- stream_writer