Reverse-proxy to Synapse via matrix-synapse-reverse-proxy-companion

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090
This commit is contained in:
Slavi Pantaleev 2022-11-20 16:43:30 +02:00
parent 424de93f82
commit e9e84341a9
23 changed files with 720 additions and 245 deletions

View file

@ -1,3 +1,51 @@
# 2022-11-20
## (Backward Compatibility Break) Changing how reverse-proxying to Synapse works - now via a `matrix-synapse-reverse-proxy-companion` service
**TLDR**: There's now a `matrix-synapse-reverse-proxy-companion` nginx service, which helps with reverse-proxying to Synapse and its various worker processes (if workers are enabled), so that `matrix-nginx-proxy` can be relieved of this role. `matrix-nginx-proxy` still remains as the public SSL-terminating reverse-proxy in the playbook. `matrix-synapse-reverse-proxy-companion` is just one more reverse-proxy thrown into the mix for convenience. People with a more custom reverse-proxying configuration may be affected - see [Webserver configuration](#webserver-configuration) below.
### Background
Previously, `matrix-nginx-proxy` forwarded requests to Synapse directly. When Synapse is running in worker mode, the reverse-proxying configuration is more complicated (different requests need to go to different Synapse worker processes). `matrix-nginx-proxy` had configuration for sending each URL endpoint to the correct Synapse worker responsible for handling it. However, sometimes people like to disable `matrix-nginx-proxy` (for whatever reason) as detailed in [Using your own webserver, instead of this playbook's nginx proxy](docs/configuring-playbook-own-webserver.md).
Because `matrix-nginx-proxy` was so central to request forwarding, when it was disabled and Synapse was running with workers enabled, there was nothing which could forward requests to the correct place anymore.. which caused [problems such as this one affecting Dimension](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090).
### Solution
From now on, `matrix-nginx-proxy` is relieved of its function of reverse-proxying to Synapse and its various worker processes.
This role is now handled by the new `matrix-synapse-reverse-proxy-companion` nginx service and works even if `matrix-nginx-proxy` is disabled.
The purpose of the new `matrix-synapse-reverse-proxy-companion` service is to:
- serve as a companion to Synapse and know how to reverse-proxy to Synapse correctly (no matter if workers are enabled or not)
- provide a unified container address for reaching Synapse (no matter if workers are enabled or not)
- `matrix-synapse-reverse-proxy-companion:8008` for Synapse Client-Server API traffic
- `matrix-synapse-reverse-proxy-companion:8048` for Synapse Server-Server (Federation) API traffic
- simplify `matrix-nginx-proxy` configuration - it now only needs to send requests to `matrix-synapse-reverse-proxy-companion` or `matrix-dendrite`, etc., without having to worry about workers
- allow reverse-proxying to Synapse, even if `matrix-nginx-proxy` is disabled
`matrix-nginx-proxy` still remains as the public SSL-terminating reverse-proxy in the playbook. All traffic goes through it before reaching any of the services.
It's just that now the Synapse traffic is routed through `matrix-synapse-reverse-proxy-companion` like this:
(`matrix-nginx-proxy` -> `matrix-synapse-reverse-proxy-companion` -> (`matrix-synapse` or some Synapse worker)).
Various services (like Dimension, etc.) still talk to Synapse via `matrix-nginx-proxy` (e.g. `http://matrix-nginx-proxy:12080`) preferentially. They only talk to Synapse via the reverse-proxy companion (e.g. `http://matrix-synapse-reverse-proxy-companion:8008`) if `matrix-nginx-proxy` is disabled. Services should not be talking to Synapse (e.g. `https://matrix-synapse:8008` directly anymore), because when workers are enabled, that's the Synapse `master` process and may not be serving all URL endpoints needed by the service.
### Webserver configuration
- if you're using `matrix-nginx-proxy` (`matrix_nginx_proxy_enabled: true`, which is the default for the playbook), you don't need to do anything
- if you're using your own `nginx` webserver running on the server, you shouldn't be affected. The `/matrix/nginx/conf.d` configuration and exposed ports that you're relying on will automatically be updated in a way that should work
- if you're using another local webserver (e.g. Apache, etc.) and haven't changed any ports (`matrix_*_host_bind_port` definitions), you shouldn't be affected. You're likely sending Matrix traffic to `127.0.0.1:8008` and `127.0.0.1:8048`. These ports (`8008` and `8048`) will still be exposed on `127.0.0.1` by default - just not by the `matrix-synapse` container from now on, but by the `matrix-synapse-reverse-proxy-companion` container instead
- if you've been exposing `matrix-synapse` ports (`matrix_synapse_container_client_api_host_bind_port`, etc.) manually, you should consider exposing `matrix-synapse-reverse-proxy-companion` ports instead
- if you're running Traefik and reverse-proxying directly to the `matrix-synapse` container, you should start reverse-proxying to the `matrix-synapse-reverse-proxy-companion` container instead. See [our updated Traefik example configuration](docs/configuring-playbook-own-webserver.md#sample-configuration-for-running-behind-traefik-20). Note: we now recommend calling the federation entry point `federation` (instead of `synapse`) and reverse-proxying the federation traffic via `matrix-nginx-proxy`, instead of sending it directly to Synapse (or `matrix-synapse-reverse-proxy-companion`). This makes the configuration simpler.
# 2022-11-05
## (Backward Compatibility Break) A new default standalone mode for Etherpad

View file

@ -70,7 +70,7 @@ matrix_email2matrix_matrix_mappings:
SkipMarkdown: true
```
You can also set `MatrixHomeserverUrl` to `http://matrix-synapse:8008`, instead of the public `https://matrix.DOMAIN`.
You can also set `MatrixHomeserverUrl` to `http://matrix-synapse-reverse-proxy-companion:8008`, instead of the public `https://matrix.DOMAIN`.
However, that's more likely to break in the future if you switch to another server implementation than Synapse.
Re-run the playbook (`--tags=setup-email2matrix,start`) and try sending an email to `my-mailbox@matrix.DOMAIN`.

View file

@ -46,6 +46,9 @@ matrix_synapse_federation_port_enabled: false
# This removes the `8448` virtual host from the matrix-nginx-proxy reverse-proxy server.
matrix_nginx_proxy_proxy_matrix_federation_api_enabled: false
# This stops the federation port on the synapse-reverse-proxy-companion side (normally `matrix-synapse-reverse-proxy-companion:8048` on the container network).
matrix_synapse_reverse_proxy_companion_federation_api_enabled: false
```
## Changing the federation port from 8448 to a different port to use a CDN that only accepts 443/80 ports

View file

@ -40,8 +40,8 @@ No matter which external webserver you decide to go with, you'll need to:
Here are the variables required for the default configuration (Synapse and Element)
```
matrix_synapse_container_client_api_host_bind_port: '0.0.0.0:8008'
matrix_synapse_container_federation_api_plain_host_bind_port: '0.0.0.0:8048'
matrix_synapse_reverse_proxy_companion_container_client_api_host_bind_port: '0.0.0.0:8008'
matrix_synapse_reverse_proxy_companion_container_federation_api_host_bind_port: '0.0.0.0:8048'
matrix_client_element_container_http_host_bind_port: "0.0.0.0:8765"
```
@ -172,31 +172,24 @@ matrix_nginx_proxy_container_extra_arguments:
# The Nginx proxy container will receive traffic from these subdomains
- '--label "traefik.http.routers.matrix-nginx-proxy.rule=Host(`{{ matrix_server_fqn_matrix }}`,`{{ matrix_server_fqn_element }}`,`{{ matrix_server_fqn_dimension }}`,`{{ matrix_server_fqn_jitsi }}`)"'
# (The 'web-secure' entrypoint must bind to port 443 in Traefik config)
- '--label "traefik.http.routers.matrix-nginx-proxy.entrypoints=web-secure"'
# (The 'default' certificate resolver must be defined in Traefik config)
- '--label "traefik.http.routers.matrix-nginx-proxy.tls.certResolver=default"'
# The Nginx proxy container uses port 8080 internally
- '--label "traefik.http.services.matrix-nginx-proxy.loadbalancer.server.port=8080"'
matrix_synapse_container_extra_arguments:
# May be unnecessary depending on Traefik config, but can't hurt
- '--label "traefik.enable=true"'
# The Synapse container will receive traffic from this subdomain
- '--label "traefik.http.routers.matrix-synapse.rule=Host(`{{ matrix_server_fqn_matrix }}`)"'
# (The 'synapse' entrypoint must bind to port 8448 in Traefik config)
- '--label "traefik.http.routers.matrix-synapse.entrypoints=synapse"'
# Federation
- '--label "traefik.http.routers.matrix-nginx-proxy-federation.rule=Host(`{{ matrix_server_fqn_matrix }}`)"'
# (The 'federation' entrypoint must bind to port 8448 in Traefik config)
- '--label "traefik.http.routers.matrix-nginx-proxy-federation.entrypoints=federation"'
# (The 'default' certificate resolver must be defined in Traefik config)
- '--label "traefik.http.routers.matrix-synapse.tls.certResolver=default"'
- '--label "traefik.http.routers.matrix-nginx-proxy-federation.tls.certResolver=default"'
# The Nginx proxy container uses port `matrix_nginx_proxy_proxy_matrix_federation_port (8448) internally
- '--label "traefik.http.services.matrix-nginx-proxy-federation.loadbalancer.server.port={{ matrix_nginx_proxy_proxy_matrix_federation_port }}"'
- '--label "traefik.http.services.matrix-nginx-proxy-federation.loadbalancer.server.scheme={{ 'https' if matrix_nginx_proxy_https_enabled else 'http' }}"'
# The Synapse container uses port 8048 internally
- '--label "traefik.http.services.matrix-synapse.loadbalancer.server.port=8048"'
matrix_synapse_reverse_proxy_companion_container_labels_traefik_enabled: true
```
This method uses labels attached to the Nginx and Synapse containers to provide the Traefik Docker provider with the information it needs to proxy `matrix.DOMAIN`, `element.DOMAIN`, `dimension.DOMAIN` and `jitsi.DOMAIN`. Some [static configuration](https://docs.traefik.io/v2.0/reference/static-configuration/file/) is required in Traefik; namely, having endpoints on ports 443 and 8448 and having a certificate resolver.
@ -240,7 +233,7 @@ services:
- "--providers.docker.network=traefik"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web-secure.address=:443"
- "--entrypoints.synapse.address=:8448"
- "--entrypoints.federation.address=:8448"
- "--certificatesresolvers.default.acme.tlschallenge=true"
- "--certificatesresolvers.default.acme.email=YOUR EMAIL"
- "--certificatesresolvers.default.acme.storage=/letsencrypt/acme.json"

View file

@ -21,11 +21,11 @@ https://matrix.DOMAIN {
}
# Synapse Client<>Server API
proxy /_matrix matrix-synapse:8008 {
proxy /_matrix matrix-synapse-reverse-proxy-companion:8008 {
transparent
except /_matrix/identity/ /_matrix/client/r0/user_directory/search
}
proxy /_synapse/client matrix-synapse:8008 {
proxy /_synapse/client matrix-synapse-reverse-proxy-companion:8008 {
transparent
}
}

View file

@ -57,12 +57,10 @@ devture_playbook_state_preserver_commit_hash_preservation_dst: "{{ matrix_base_d
matrix_identity_server_url: "{{ ('https://' + matrix_server_fqn_matrix) if matrix_ma1sd_enabled else None }}"
# If Synapse workers are enabled and matrix-nginx-proxy is disabled, certain APIs may not work over 'http://matrix-synapse:{{ matrix_synapse_container_client_api_port }}'.
# This is because we explicitly disable them for the main Synapse process.
matrix_homeserver_container_url: |-
{{
'http://matrix-nginx-proxy:12080' if matrix_nginx_proxy_enabled else {
'synapse': ('http://matrix-synapse:'+ matrix_synapse_container_client_api_port|string),
'synapse': ('http://matrix-synapse-reverse-proxy-companion:8008' if matrix_synapse_reverse_proxy_companion_enabled else 'http://matrix-synapse:'+ matrix_synapse_container_client_api_port|string),
'dendrite': ('http://matrix-dendrite:' + matrix_dendrite_http_bind_port|string),
'conduit': ('http://matrix-conduit:' + matrix_conduit_port_number|string),
}[matrix_homeserver_implementation]
@ -71,7 +69,7 @@ matrix_homeserver_container_url: |-
matrix_homeserver_container_federation_url: |-
{{
'http://matrix-nginx-proxy:12088' if matrix_nginx_proxy_enabled else {
'synapse': ('http://matrix-synapse:'+ matrix_synapse_container_federation_api_plain_port|string),
'synapse': ('http://matrix-synapse-reverse-proxy-companion:8048' if matrix_synapse_reverse_proxy_companion_enabled else 'http://matrix-synapse:'+ matrix_synapse_container_federation_api_plain_port|string),
'dendrite': ('http://matrix-dendrite:' + matrix_dendrite_http_bind_port|string),
'conduit': ('http://matrix-conduit:' + matrix_conduit_port_number|string),
}[matrix_homeserver_implementation]
@ -1720,6 +1718,7 @@ matrix_ma1sd_database_password: "{{ '%s' | format(matrix_homeserver_generic_secr
#
######################################################################
######################################################################
#
# matrix-nginx-proxy
@ -1782,10 +1781,10 @@ matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container: "matrix-ngin
matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container: "127.0.0.1:12088"
matrix_nginx_proxy_proxy_synapse_enabled: "{{ matrix_synapse_enabled }}"
matrix_nginx_proxy_proxy_synapse_client_api_addr_with_container: "matrix-synapse:{{ matrix_synapse_container_client_api_port }}"
matrix_nginx_proxy_proxy_synapse_client_api_addr_sans_container: "127.0.0.1:{{ matrix_synapse_container_client_api_port }}"
matrix_nginx_proxy_proxy_synapse_federation_api_addr_with_container: "matrix-synapse:{{matrix_synapse_container_federation_api_plain_port | string}}"
matrix_nginx_proxy_proxy_synapse_federation_api_addr_sans_container: "127.0.0.1:{{matrix_synapse_container_federation_api_plain_port | string}}"
matrix_nginx_proxy_proxy_synapse_client_api_addr_with_container: "{{ 'matrix-synapse-reverse-proxy-companion:8008' if matrix_synapse_reverse_proxy_companion_enabled else 'matrix-synapse:8008' }}"
matrix_nginx_proxy_proxy_synapse_client_api_addr_sans_container: "127.0.0.1:8008"
matrix_nginx_proxy_proxy_synapse_federation_api_addr_with_container: "{{ 'matrix-synapse-reverse-proxy-companion:8048' if matrix_synapse_reverse_proxy_companion_enabled else 'matrix-synapse:8048' }}"
matrix_nginx_proxy_proxy_synapse_federation_api_addr_sans_container: "127.0.0.1:8048"
matrix_nginx_proxy_proxy_dendrite_enabled: "{{ matrix_dendrite_enabled }}"
matrix_nginx_proxy_proxy_dendrite_client_api_addr_with_container: "matrix-dendrite:{{ matrix_dendrite_http_bind_port | string }}"
@ -1815,24 +1814,14 @@ matrix_nginx_proxy_self_check_validate_certificates: "{{ false if matrix_ssl_ret
# and https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1074
matrix_nginx_proxy_ocsp_stapling_enabled: "{{ matrix_ssl_retrieval_method != 'self-signed' }}"
matrix_nginx_proxy_synapse_workers_enabled: "{{ matrix_synapse_workers_enabled }}"
matrix_nginx_proxy_synapse_workers_list: "{{ matrix_synapse_workers_enabled_list }}"
matrix_nginx_proxy_synapse_generic_worker_client_server_locations: "{{ matrix_synapse_workers_generic_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_generic_worker_federation_locations: "{{ matrix_synapse_workers_generic_worker_federation_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints }}"
matrix_nginx_proxy_synapse_media_repository_locations: "{{matrix_synapse_workers_media_repository_endpoints|default([]) }}"
matrix_nginx_proxy_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_worker_client_server_endpoints|default([]) }}"
matrix_nginx_proxy_systemd_wanted_services_list: |
{{
['matrix-' + matrix_homeserver_implementation + '.service']
+
(matrix_synapse_webserving_workers_systemd_services_list if matrix_homeserver_implementation == 'synapse' and matrix_synapse_workers_enabled else [])
+
(['matrix-synapse-reverse-proxy-companion.service'] if matrix_synapse_reverse_proxy_companion_enabled else [])
+
(['matrix-corporal.service'] if matrix_corporal_enabled else [])
+
(['matrix-ma1sd.service'] if matrix_ma1sd_enabled else [])
@ -2328,16 +2317,6 @@ matrix_synapse_container_image_self_build: "{{ matrix_architecture not in ['arm6
# When ma1sd is enabled, we can use it to validate phone numbers. It's something that the homeserver cannot do by itself.
matrix_synapse_account_threepid_delegates_msisdn: "{{ 'http://matrix-ma1sd:' + matrix_ma1sd_container_port | string if matrix_ma1sd_enabled else '' }}"
# Normally, matrix-nginx-proxy is enabled and nginx can reach Synapse over the container network.
# If matrix-nginx-proxy is not enabled, or you otherwise have a need for it,
# you can expose Synapse's ports to the host.
#
# For exposing the Matrix Client API's port (plain HTTP) to the local host.
matrix_synapse_container_client_api_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:' + matrix_synapse_container_client_api_port | string }}"
#
# For exposing the Matrix Federation API's plain port (plain HTTP) to the local host.
matrix_synapse_container_federation_api_plain_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:' + matrix_synapse_container_federation_api_plain_port | string }}"
#
# For exposing the Matrix Federation API's TLS port (HTTPS) to the internet on all network interfaces.
matrix_synapse_container_federation_api_tls_host_bind_port: "{{ matrix_federation_public_port if (matrix_synapse_federation_enabled and matrix_synapse_tls_federation_listener_enabled) else '' }}"
#
@ -2426,6 +2405,37 @@ matrix_synapse_app_service_runtime_injected_config_files: "{{ matrix_homeserver_
#
######################################################################
######################################################################
#
# matrix-synapse-reverse-proxy-companion
#
######################################################################
matrix_synapse_reverse_proxy_companion_enabled: "{{ matrix_synapse_enabled }}"
matrix_synapse_reverse_proxy_companion_client_api_client_max_body_size_mb: "{{ matrix_synapse_max_upload_size_mb }}"
matrix_synapse_reverse_proxy_companion_container_client_api_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:8008' }}"
matrix_synapse_reverse_proxy_companion_container_federation_api_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:8048' }}"
matrix_synapse_reverse_proxy_companion_synapse_workers_enabled: "{{ matrix_synapse_workers_enabled }}"
matrix_synapse_reverse_proxy_companion_synapse_workers_list: "{{ matrix_synapse_workers_enabled_list }}"
matrix_synapse_reverse_proxy_companion_synapse_generic_worker_client_server_locations: "{{ matrix_synapse_workers_generic_worker_client_server_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_generic_worker_federation_locations: "{{ matrix_synapse_workers_generic_worker_federation_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_typing_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_typing_stream_worker_client_server_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_to_device_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_to_device_stream_worker_client_server_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_account_data_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_account_data_stream_worker_client_server_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_receipts_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_receipts_stream_worker_client_server_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_presence_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_presence_stream_worker_client_server_endpoints }}"
matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations: "{{matrix_synapse_workers_media_repository_endpoints|default([]) }}"
matrix_synapse_reverse_proxy_companion_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_worker_client_server_endpoints|default([]) }}"
######################################################################
#
# /matrix-synapse-reverse-proxy-companion
#
######################################################################
######################################################################
#
# matrix-synapse-admin

View file

@ -58,6 +58,7 @@
- custom/matrix-bot-mjolnir
- custom/matrix-cactus-comments
- custom/matrix-synapse
- custom/matrix-synapse-reverse-proxy-companion
- custom/matrix-dendrite
- custom/matrix-conduit
- custom/matrix-synapse-admin

View file

@ -37,7 +37,7 @@ matrix_corporal_var_dir_path: "{{ matrix_corporal_base_path }}/var"
matrix_corporal_matrix_homeserver_domain_name: "{{ matrix_domain }}"
# Controls where matrix-corporal can reach your Synapse server (e.g. "http://matrix-synapse:{{ matrix_synapse_container_client_api_port }}").
# Controls where matrix-corporal can reach your Synapse server (e.g. "http://matrix-synapse-reverse-proxy-companion:{{ matrix_synapse_container_client_api_port }}").
# If Synapse runs on the same machine, you may need to add its service to `matrix_corporal_systemd_required_services_list`.
matrix_corporal_matrix_homeserver_api_endpoint: ""

View file

@ -639,29 +639,6 @@ matrix_nginx_proxy_proxy_matrix_nginx_status_enabled: false
matrix_nginx_proxy_proxy_matrix_nginx_status_allowed_addresses: ['{{ ansible_default_ipv4.address }}']
# synapse worker activation and endpoint mappings
matrix_nginx_proxy_synapse_workers_enabled: false
matrix_nginx_proxy_synapse_workers_list: []
matrix_nginx_proxy_synapse_generic_worker_client_server_locations: []
matrix_nginx_proxy_synapse_generic_worker_federation_locations: []
matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations: []
matrix_nginx_proxy_synapse_media_repository_locations: []
matrix_nginx_proxy_synapse_user_dir_locations: []
# synapse content caching
matrix_nginx_proxy_synapse_cache_enabled: false
matrix_nginx_proxy_synapse_cache_path: "{{ '/tmp/synapse-cache' if matrix_nginx_proxy_enabled else matrix_nginx_proxy_data_path + '/synapse-cache' }}"
matrix_nginx_proxy_synapse_cache_keys_zone_name: "STATIC"
matrix_nginx_proxy_synapse_cache_keys_zone_size: "10m"
matrix_nginx_proxy_synapse_cache_inactive_time: "48h"
matrix_nginx_proxy_synapse_cache_max_size_mb: 1024
matrix_nginx_proxy_synapse_cache_proxy_cache_valid_time: "24h"
# The amount of worker processes and connections
# Consider increasing these when you are expecting high amounts of traffic
# http://nginx.org/en/docs/ngx_core_module.html#worker_connections

View file

@ -12,17 +12,15 @@
#
- name: Ensure Matrix nginx-proxy paths exist
ansible.builtin.file:
path: "{{ item.path }}"
path: "{{ item }}"
state: directory
mode: 0750
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
with_items:
- {path: "{{ matrix_nginx_proxy_base_path }}", when: true}
- {path: "{{ matrix_nginx_proxy_data_path }}", when: true}
- {path: "{{ matrix_nginx_proxy_confd_path }}", when: true}
- {path: "{{ matrix_nginx_proxy_synapse_cache_path }}", when: "{{ matrix_nginx_proxy_synapse_cache_enabled and not matrix_nginx_proxy_enabled }}"}
when: item.when | bool
- "{{ matrix_nginx_proxy_base_path }}"
- "{{ matrix_nginx_proxy_data_path }}"
- "{{ matrix_nginx_proxy_confd_path }}"
- name: Ensure Matrix nginx-proxy configured (main config override)
ansible.builtin.template:

View file

@ -1,70 +1,5 @@
#jinja2: lstrip_blocks: "True"
{% set generic_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'generic_worker') | list %}
{% set stream_writer_typing_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'typing') | list %}
{% set stream_writer_to_device_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'to_device') | list %}
{% set stream_writer_account_data_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'account_data') | list %}
{% set stream_writer_receipts_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'receipts') | list %}
{% set stream_writer_presence_stream_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'presence') | list %}
{% set media_repository_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'media_repository') | list %}
{% set user_dir_workers = matrix_nginx_proxy_synapse_workers_list | selectattr('type', 'equalto', 'user_dir') | list %}
{% macro render_worker_upstream(name, workers, matrix_nginx_proxy_enabled) %}
{% if workers | length > 0 %}
upstream {{ name }} {
{% for worker in workers %}
{% if matrix_nginx_proxy_enabled %}
server "{{ worker.name }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{% endmacro %}
{% macro render_locations_to_upstream(locations, upstream_name) %}
{% for location in locations %}
location ~ {{ location }} {
proxy_pass http://{{ upstream_name }}$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% endmacro %}
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{% if matrix_nginx_proxy_synapse_cache_enabled %}
proxy_cache_path {{ matrix_nginx_proxy_synapse_cache_path }} levels=1:2 keys_zone={{ matrix_nginx_proxy_synapse_cache_keys_zone_name }}:{{ matrix_nginx_proxy_synapse_cache_keys_zone_size }} inactive={{ matrix_nginx_proxy_synapse_cache_inactive_time }} max_size={{ matrix_nginx_proxy_synapse_cache_max_size_mb }}m;
{% endif %}
# Round Robin "upstream" pools for workers
{% if generic_workers |length > 0 %}
upstream generic_workers_upstream {
# ensures that requests from the same client will always be passed
# to the same server (except when this server is unavailable)
hash $http_x_forwarded_for;
{% for worker in generic_workers %}
{% if matrix_nginx_proxy_enabled %}
server "{{ worker.name }}:{{ worker.port }}";
{% else %}
server "127.0.0.1:{{ worker.port }}";
{% endif %}
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('stream_writer_typing_stream_workers_upstream', stream_writer_typing_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_to_device_stream_workers_upstream', stream_writer_to_device_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_account_data_stream_workers_upstream', stream_writer_account_data_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_receipts_stream_workers_upstream', stream_writer_receipts_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('stream_writer_presence_stream_workers_upstream', stream_writer_presence_stream_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('media_repository_workers_upstream', media_repository_workers, matrix_nginx_proxy_enabled) }}
{{ render_worker_upstream('user_dir_workers_upstream', user_dir_workers, matrix_nginx_proxy_enabled) }}
{% endif %}
server {
listen 12080;
{% if matrix_nginx_proxy_enabled %}
@ -77,71 +12,6 @@ server {
gzip on;
gzip_types text/plain application/json;
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{# Workers redirects BEGIN #}
{% if generic_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappgeneric_worker
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_generic_worker_client_server_locations, 'generic_workers_upstream') }}
{% endif %}
{% if stream_writer_typing_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-typing-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_typing_stream_worker_client_server_locations, 'stream_writer_typing_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_to_device_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-to_device-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_to_device_stream_worker_client_server_locations, 'stream_writer_to_device_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_account_data_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-account_data-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_account_data_stream_worker_client_server_locations, 'stream_writer_account_data_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_receipts_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-receipts-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_receipts_stream_worker_client_server_locations, 'stream_writer_receipts_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_presence_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-presence-stream
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_stream_writer_presence_stream_worker_client_server_locations, 'stream_writer_presence_stream_workers_upstream') }}
{% endif %}
{% if media_repository_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository
{% for location in matrix_nginx_proxy_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;
client_max_body_size {{ matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb }}M;
proxy_max_temp_file_size 0;
{% if matrix_nginx_proxy_synapse_cache_enabled %}
proxy_buffering on;
proxy_cache {{ matrix_nginx_proxy_synapse_cache_keys_zone_name }};
proxy_cache_valid any {{ matrix_nginx_proxy_synapse_cache_proxy_cache_valid_time }};
proxy_force_ranges on;
add_header X-Cache-Status $upstream_cache_status;
{% endif %}
}
{% endfor %}
{% endif %}
{% if user_dir_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#updating-the-user-directory
# If matrix_nginx_proxy_proxy_matrix_user_directory_search_enabled is set, requests may not reach here,
# but could be captured early on (see `matrix-domain.conf.j2`) and forwarded elsewhere (to an identity server, etc.).
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_user_dir_locations, 'user_dir_workers_upstream') }}
{% endif %}
{# Workers redirects END #}
{% endif %}
{% for configuration_block in matrix_nginx_proxy_proxy_synapse_additional_server_configuration_blocks %}
{{- configuration_block }}
{% endfor %}
@ -180,34 +50,6 @@ server {
gzip on;
gzip_types text/plain application/json;
{% if matrix_nginx_proxy_synapse_workers_enabled %}
{% if generic_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappgeneric_worker
{{ render_locations_to_upstream(matrix_nginx_proxy_synapse_generic_worker_federation_locations, 'generic_workers_upstream') }}
{% endif %}
{% if media_repository_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository
{% for location in matrix_nginx_proxy_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;
client_max_body_size {{ matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb }}M;
proxy_max_temp_file_size 0;
{% if matrix_nginx_proxy_synapse_cache_enabled %}
proxy_buffering on;
proxy_cache {{ matrix_nginx_proxy_synapse_cache_keys_zone_name }};
proxy_cache_valid any {{ matrix_nginx_proxy_synapse_cache_proxy_cache_valid_time }};
proxy_force_ranges on;
add_header X-Cache-Status $upstream_cache_status;
{% endif %}
}
{% endfor %}
{% endif %}
{% endif %}
location / {
{% if matrix_nginx_proxy_enabled %}
{# Use the embedded DNS resolver in Docker containers to discover the service #}

View file

@ -22,9 +22,6 @@ ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name
--cap-drop=ALL \
--read-only \
--tmpfs=/tmp:rw,noexec,nosuid,size={{ matrix_nginx_proxy_tmp_directory_size_mb }}m \
{% if matrix_nginx_proxy_synapse_cache_enabled %}
--tmpfs=/tmp/synapse-cache:rw,noexec,nosuid,size={{ matrix_nginx_proxy_tmp_cache_directory_size_mb }}m\
{% endif %}
--network={{ matrix_docker_network }} \
{% if matrix_nginx_proxy_container_http_host_bind_port %}
-p {{ matrix_nginx_proxy_container_http_host_bind_port }}:8080 \

View file

@ -0,0 +1,164 @@
---
# matrix-synapse-reverse-proxy companion is a role which brings up a containerized nginx webserver which helps with reverse-proxying to Synapse.
#
# When Synapse is NOT running in worker-mode, reverse-proxying is relatively simple (everything goes to `matrix-synapse:XXXX`).
#
# When Synapse workers are enabled, however, the reverse-proxying configuration is much more complicated.
# Certain requests need to go to certain workers, etc.
# In the past, the main reverse proxy (`matrix-synapse-reverse-proxy-companion`) was handling request routing to the appropriate workers,
# but that only worked well for external requests (from outside of the Matrix server itself).
#
# Without the help of `matrix-synapse-reverse-proxy-companion`, internal services (like Dimension) that would like to talk to Synapse over the container network
# did not have an endpoint for Synapse that they could be pointed to and have it just work.
# If `matrix-synapse-reverse-proxy-companion` was enabled, Dimension could be pointed to its vhost handling Synapse and routing to the appropriate workers,
# but when `matrix-synapse-reverse-proxy-companion` was disabled, this helpful functionality was not available and the best we could do
# is point Dimension to the main Synapse process at `matrix-synapse:XXXX` itself.
# Doing that breaks requests that need to go to specific workers.
# See: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090
#
# What this role does is, it extracts all the Synapse request routing out of the `matrix-synapse-reverse-proxy-companion` role here,
# and makes the `matrix-synapse-reverse-proxy-companion` container service represent Synapse and route appropriately,
# regardless of whether workers are enabled or disabled.
# All other playbook services can then forget about `matrix-synapse` or `matrix-synapse-whatever-worker`, etc.,
# and just use `matrix-synapse-reverse-proxy-companion` as their request destination.
matrix_synapse_reverse_proxy_companion_enabled: true
matrix_synapse_reverse_proxy_companion_version: 1.23.2-alpine
matrix_synapse_reverse_proxy_companion_base_path: "{{ matrix_synapse_base_path }}/reverse-proxy-companion"
matrix_synapse_reverse_proxy_companion_confd_path: "{{ matrix_synapse_reverse_proxy_companion_base_path }}/conf.d"
# List of systemd services that matrix-synapse-reverse-proxy-companion.service depends on
matrix_synapse_reverse_proxy_companion_systemd_required_services_list: ['docker.service']
# List of systemd services that matrix-synapse-reverse-proxy-companion.service wants
matrix_synapse_reverse_proxy_companion_systemd_wanted_services_list: ['matrix-synapse.service']
# We use an official nginx image, which we fix-up to run unprivileged.
# An alternative would be an `nginxinc/nginx-unprivileged` image, but
# that is frequently out of date.
matrix_synapse_reverse_proxy_companion_container_image: "{{ matrix_container_global_registry_prefix }}nginx:{{ matrix_synapse_reverse_proxy_companion_version }}"
matrix_synapse_reverse_proxy_companion_container_image_force_pull: "{{ matrix_synapse_reverse_proxy_companion_container_image.endswith(':latest') }}"
matrix_synapse_reverse_proxy_companion_container_network: "{{ matrix_docker_network }}"
# A list of additional container networks that matrix-synapse-reverse-proxy-companion would be connected to.
# The playbook does not create these networks, so make sure they already exist.
#
# Use this to expose matrix-synapse-reverse-proxy-companion to another reverse proxy, which runs in a different container network,
# without exposing all other Matrix services to that other reverse-proxy.
#
# For background, see: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1498
matrix_synapse_reverse_proxy_companion_container_additional_networks: []
# Controls whether the matrix-synapse-reverse-proxy-companion container exposes its HTTP Client-Server API port (tcp/8008 in the container).
#
# Takes an "<ip>:<port>" or "<port>" value (e.g. "127.0.0.1:8008"), or empty string to not expose.
matrix_synapse_reverse_proxy_companion_container_client_api_host_bind_port: ''
# Controls whether the matrix-synapse-reverse-proxy-companion container exposes its HTTP Federation (Server-Server) API port (tcp/8048 in the container).
#
# Takes an "<ip>:<port>" or "<port>" value (e.g. "127.0.0.1:8048"), or empty string to not expose.
matrix_synapse_reverse_proxy_companion_container_federation_api_host_bind_port: ''
# The amount of worker processes and connections
# Consider increasing these when you are expecting high amounts of traffic
# http://nginx.org/en/docs/ngx_core_module.html#worker_connections
matrix_synapse_reverse_proxy_companion_worker_processes: auto
matrix_synapse_reverse_proxy_companion_worker_connections: 1024
# Option to disable the access log
matrix_synapse_reverse_proxy_companion_access_log_enabled: true
# The tmpfs at /tmp needs to be large enough to handle multiple concurrent file uploads.
matrix_synapse_reverse_proxy_companion_tmp_directory_size_mb: "{{ (matrix_synapse_reverse_proxy_companion_federation_api_client_max_body_size_mb | int) * 50 }}"
matrix_synapse_reverse_proxy_companion_tmp_cache_directory_size_mb: "{{ (matrix_synapse_reverse_proxy_companion_synapse_cache_max_size_mb | int) * 2 }}"
# A list of strings containing additional configuration blocks to add to the nginx server configuration (nginx.conf).
# for big matrixservers to enlarge the number of open files to prevent timeouts
# matrix_synapse_reverse_proxy_companion_additional_configuration_blocks:
# - 'worker_rlimit_nofile 30000;'
matrix_synapse_reverse_proxy_companion_additional_configuration_blocks: []
# A list of strings containing additional configuration blocks to add to the nginx event server configuration (nginx.conf).
matrix_synapse_reverse_proxy_companion_event_additional_configuration_blocks: []
# A list of strings containing additional configuration blocks to add to the nginx http's server configuration (nginx-http.conf).
matrix_synapse_reverse_proxy_companion_http_additional_server_configuration_blocks: []
# To increase request timeout in NGINX using proxy_read_timeout, proxy_connect_timeout, proxy_send_timeout, send_timeout directives
# Nginx Default: proxy_connect_timeout 60s; #Defines a timeout for establishing a connection with a proxied server
# Nginx Default: proxy_send_timeout 60s; #Sets a timeout for transmitting a request to the proxied server.
# Nginx Default: proxy_read_timeout 60s; #Defines a timeout for reading a response from the proxied server.
# Nginx Default: send_timeout 60s; #Sets a timeout for transmitting a response to the client.
#
# For more information visit:
# http://nginx.org/en/docs/http/ngx_http_proxy_module.html
# http://nginx.org/en/docs/http/ngx_http_core_module.html#send_timeout
# https://www.nginx.com/resources/wiki/start/topics/examples/fullexample2/
#
# Here we are sticking with nginx default values change this value carefully.
matrix_synapse_reverse_proxy_companion_proxy_connect_timeout: 60
matrix_synapse_reverse_proxy_companion_proxy_send_timeout: 60
matrix_synapse_reverse_proxy_companion_proxy_read_timeout: 60
matrix_synapse_reverse_proxy_companion_send_timeout: 60
# For OCSP purposes, we need to define a resolver at the `server{}` level or `http{}` level (we do the latter).
#
# Otherwise, we get warnings like this:
# > [warn] 22#22: no resolver defined to resolve r3.o.lencr.org while requesting certificate status, responder: r3.o.lencr.org, certificate: "/matrix/ssl/config/live/.../fullchain.pem"
#
# We point it to the internal Docker resolver, which likely delegates to nameservers defined in `/etc/resolv.conf`.
matrix_synapse_reverse_proxy_companion_http_level_resolver: 127.0.0.11
matrix_synapse_reverse_proxy_companion_hostname: "matrix-synapse-reverse-proxy-companion"
# matrix_synapse_reverse_proxy_companion_client_api_addr specifies the address where the Client-Server API is
matrix_synapse_reverse_proxy_companion_client_api_addr: 'matrix-synapse:{{ matrix_synapse_container_client_api_port }}'
# This needs to be equal or higher than the maximum upload size accepted by Synapse.
matrix_synapse_reverse_proxy_companion_client_api_client_max_body_size_mb: 50
# matrix_synapse_reverse_proxy_companion_federation_api_enabled specifies whether reverse proxying for the Federation (Server-Server) API should be done
matrix_synapse_reverse_proxy_companion_federation_api_enabled: true
# matrix_synapse_reverse_proxy_companion_federation_api_addr specifies the address where the Federation (Server-Server) API is
matrix_synapse_reverse_proxy_companion_federation_api_addr: 'matrix-synapse:{{ matrix_synapse_container_federation_api_plain_port }}'
matrix_synapse_reverse_proxy_companion_federation_api_client_max_body_size_mb: "{{ (matrix_synapse_reverse_proxy_companion_client_api_client_max_body_size_mb | int) * 3 }}"
# A list of strings containing additional configuration blocks to add to the nginx vhost handling the Synapse Client-Server API
matrix_synapse_reverse_proxy_companion_synapse_client_api_additional_server_configuration_blocks: []
# A list of strings containing additional configuration blocks to add to the nginx vhost handling the Synapse Federation (Server-Server) API
matrix_synapse_reverse_proxy_companion_synapse_federation_api_additional_server_configuration_blocks: []
# synapse worker activation and endpoint mappings
matrix_synapse_reverse_proxy_companion_synapse_workers_enabled: false
matrix_synapse_reverse_proxy_companion_synapse_workers_list: []
matrix_synapse_reverse_proxy_companion_synapse_generic_worker_client_server_locations: []
matrix_synapse_reverse_proxy_companion_synapse_generic_worker_federation_locations: []
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_typing_stream_worker_client_server_locations: []
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_to_device_stream_worker_client_server_locations: []
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_account_data_stream_worker_client_server_locations: []
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_receipts_stream_worker_client_server_locations: []
matrix_synapse_reverse_proxy_companion_synapse_stream_writer_presence_stream_worker_client_server_locations: []
matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations: []
matrix_synapse_reverse_proxy_companion_synapse_user_dir_locations: []
# synapse content caching
matrix_synapse_reverse_proxy_companion_synapse_cache_enabled: false
matrix_synapse_reverse_proxy_companion_synapse_cache_path: /tmp/synapse-cache
matrix_synapse_reverse_proxy_companion_synapse_cache_keys_zone_name: "STATIC"
matrix_synapse_reverse_proxy_companion_synapse_cache_keys_zone_size: "10m"
matrix_synapse_reverse_proxy_companion_synapse_cache_inactive_time: "48h"
matrix_synapse_reverse_proxy_companion_synapse_cache_max_size_mb: 1024
matrix_synapse_reverse_proxy_companion_synapse_cache_proxy_cache_valid_time: "24h"
# Controls whether matrix-synapse-reverse-proxy-companion trusts an upstream server's X-Forwarded-Proto header.
# The `matrix-synapse-reverse-proxy-companion` does not terminate SSL and always expects to be fronted by another reverse-proxy server (`matrix-nginx-proxy`, etc.).
# As such, it trusts the protocol scheme forwarded by the upstream proxy.
matrix_synapse_reverse_proxy_companion_trust_forwarded_proto: true
matrix_synapse_reverse_proxy_companion_x_forwarded_proto_value: "{{ '$http_x_forwarded_proto' if matrix_synapse_reverse_proxy_companion_trust_forwarded_proto else '$scheme' }}"

View file

@ -0,0 +1,6 @@
---
- ansible.builtin.set_fact:
matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-synapse-reverse-proxy-companion.service'] }}"
when: matrix_synapse_reverse_proxy_companion_enabled | bool

View file

@ -0,0 +1,19 @@
---
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/init.yml"
tags:
- always
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/setup_install.yml"
when: run_setup | bool and matrix_synapse_reverse_proxy_companion_enabled | bool
tags:
- setup-all
- setup-synapse-reverse-proxy-companion
- setup-synapse
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
when: run_setup | bool and not matrix_synapse_reverse_proxy_companion_enabled | bool
tags:
- setup-all
- setup-synapse-reverse-proxy-companion
- setup-synapse

View file

@ -0,0 +1,44 @@
---
- name: Ensure mtrix-synapse-reverse-proxy-companion paths exist
ansible.builtin.file:
path: "{{ item }}"
state: directory
mode: 0750
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
with_items:
- "{{ matrix_synapse_reverse_proxy_companion_base_path }}"
- "{{ matrix_synapse_reverse_proxy_companion_confd_path }}"
- name: Ensure matrix-synapse-reverse-proxy-companion configured
ansible.builtin.template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
mode: 0644
with_items:
- src: "{{ role_path }}/templates/nginx/nginx.conf.j2"
dest: "{{ matrix_synapse_reverse_proxy_companion_base_path }}/nginx.conf"
- src: "{{ role_path }}/templates/nginx/conf.d/nginx-http.conf.j2"
dest: "{{ matrix_synapse_reverse_proxy_companion_confd_path }}/nginx-http.conf"
- src: "{{ role_path }}/templates/nginx/conf.d/matrix-synapse-reverse-proxy-companion.conf.j2"
dest: "{{ matrix_synapse_reverse_proxy_companion_confd_path }}/matrix-synapse-reverse-proxy-companion.conf"
- name: Ensure matrix-synapse-reverse-proxy-companion nginx container image is pulled
community.docker.docker_image:
name: "{{ matrix_synapse_reverse_proxy_companion_container_image }}"
source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
force_source: "{{ matrix_synapse_reverse_proxy_companion_container_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_synapse_reverse_proxy_companion_container_image_force_pull }}"
register: result
retries: "{{ devture_playbook_help_container_retries_count }}"
delay: "{{ devture_playbook_help_container_retries_delay }}"
until: result is not failed
- name: Ensure matrix-synapse-reverse-proxy-companion.service installed
ansible.builtin.template:
src: "{{ role_path }}/templates/systemd/matrix-synapse-reverse-proxy-companion.service.j2"
dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-synapse-reverse-proxy-companion.service"
mode: 0644

View file

@ -0,0 +1,30 @@
---
- name: Check existence of matrix-synapse-reverse-proxy-companion service
ansible.builtin.stat:
path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-synapse-reverse-proxy-companion.service"
register: matrix_synapse_reverse_proxy_companion_service_stat
- when: matrix_synapse_reverse_proxy_companion_service_stat.stat.exists | bool
block:
- name: Ensure matrix-synapse-reverse-proxy-companion.service is stopped
ansible.builtin.service:
name: matrix_synapse_reverse_proxy_companion_service_stat
state: stopped
enabled: false
daemon_reload: true
register: stopping_result
- name: Ensure matrix-synapse-reverse-proxy-companion.service doesn't exist
ansible.builtin.file:
path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-synapse-reverse-proxy-companion.service"
state: absent
- name: Ensure systemd reloaded after matrix-synapse-reverse-proxy-companion.service removal
ansible.builtin.service:
daemon_reload: true
- name: Ensure matrix-synapse-reverse-proxy-companion data deleted
ansible.builtin.file:
path: "{{ matrix_synapse_reverse_proxy_companion_base_path }}"
state: absent

View file

@ -0,0 +1,208 @@
#jinja2: lstrip_blocks: "True"
{% set generic_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'generic_worker') | list %}
{% set stream_writer_typing_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'typing') | list %}
{% set stream_writer_to_device_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'to_device') | list %}
{% set stream_writer_account_data_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'account_data') | list %}
{% set stream_writer_receipts_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'receipts') | list %}
{% set stream_writer_presence_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'presence') | list %}
{% set media_repository_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'media_repository') | list %}
{% set user_dir_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'user_dir') | list %}
{% macro render_worker_upstream(name, workers) %}
{% if workers | length > 0 %}
upstream {{ name }} {
{% for worker in workers %}
server "{{ worker.name }}:{{ worker.port }}";
{% endfor %}
}
{% endif %}
{% endmacro %}
{% macro render_locations_to_upstream(locations, upstream_name) %}
{% for location in locations %}
location ~ {{ location }} {
proxy_pass http://{{ upstream_name }}$request_uri;
proxy_set_header Host $host;
}
{% endfor %}
{% endmacro %}
{% if matrix_synapse_reverse_proxy_companion_synapse_workers_enabled %}
{% if matrix_synapse_reverse_proxy_companion_synapse_cache_enabled %}
proxy_cache_path {{ matrix_synapse_reverse_proxy_companion_synapse_cache_path }} levels=1:2 keys_zone={{ matrix_synapse_reverse_proxy_companion_synapse_cache_keys_zone_name }}:{{ matrix_synapse_reverse_proxy_companion_synapse_cache_keys_zone_size }} inactive={{ matrix_synapse_reverse_proxy_companion_synapse_cache_inactive_time }} max_size={{ matrix_synapse_reverse_proxy_companion_synapse_cache_max_size_mb }}m;
{% endif %}
# Round Robin "upstream" pools for workers
{% if generic_workers |length > 0 %}
upstream generic_workers_upstream {
# ensures that requests from the same client will always be passed
# to the same server (except when this server is unavailable)
hash $http_x_forwarded_for;
{% for worker in generic_workers %}
server "{{ worker.name }}:{{ worker.port }}";
{% endfor %}
}
{% endif %}
{{ render_worker_upstream('stream_writer_typing_stream_workers_upstream', stream_writer_typing_stream_workers) }}
{{ render_worker_upstream('stream_writer_to_device_stream_workers_upstream', stream_writer_to_device_stream_workers) }}
{{ render_worker_upstream('stream_writer_account_data_stream_workers_upstream', stream_writer_account_data_stream_workers) }}
{{ render_worker_upstream('stream_writer_receipts_stream_workers_upstream', stream_writer_receipts_stream_workers) }}
{{ render_worker_upstream('stream_writer_presence_stream_workers_upstream', stream_writer_presence_stream_workers) }}
{{ render_worker_upstream('media_repository_workers_upstream', media_repository_workers) }}
{{ render_worker_upstream('user_dir_workers_upstream', user_dir_workers) }}
{% endif %}
server {
listen 8008;
server_name {{ matrix_synapse_reverse_proxy_companion_hostname }};
server_tokens off;
root /dev/null;
gzip on;
gzip_types text/plain application/json;
{% if matrix_synapse_reverse_proxy_companion_synapse_workers_enabled %}
{# Workers redirects BEGIN #}
{% if generic_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappgeneric_worker
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_generic_worker_client_server_locations, 'generic_workers_upstream') }}
{% endif %}
{% if stream_writer_typing_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-typing-stream
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_stream_writer_typing_stream_worker_client_server_locations, 'stream_writer_typing_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_to_device_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-to_device-stream
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_stream_writer_to_device_stream_worker_client_server_locations, 'stream_writer_to_device_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_account_data_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-account_data-stream
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_stream_writer_account_data_stream_worker_client_server_locations, 'stream_writer_account_data_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_receipts_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-receipts-stream
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_stream_writer_receipts_stream_worker_client_server_locations, 'stream_writer_receipts_stream_workers_upstream') }}
{% endif %}
{% if stream_writer_presence_stream_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#the-presence-stream
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_stream_writer_presence_stream_worker_client_server_locations, 'stream_writer_presence_stream_workers_upstream') }}
{% endif %}
{% if media_repository_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository
{% for location in matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;
client_max_body_size {{ matrix_synapse_reverse_proxy_companion_client_api_client_max_body_size_mb }}M;
proxy_max_temp_file_size 0;
{% if matrix_synapse_reverse_proxy_companion_synapse_cache_enabled %}
proxy_buffering on;
proxy_cache {{ matrix_synapse_reverse_proxy_companion_synapse_cache_keys_zone_name }};
proxy_cache_valid any {{ matrix_synapse_reverse_proxy_companion_synapse_cache_proxy_cache_valid_time }};
proxy_force_ranges on;
add_header X-Cache-Status $upstream_cache_status;
{% endif %}
}
{% endfor %}
{% endif %}
{% if user_dir_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#updating-the-user-directory
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_user_dir_locations, 'user_dir_workers_upstream') }}
{% endif %}
{# Workers redirects END #}
{% endif %}
{% for configuration_block in matrix_synapse_reverse_proxy_companion_synapse_client_api_additional_server_configuration_blocks %}
{{- configuration_block }}
{% endfor %}
{# Everything else just goes to the API server ##}
location / {
{# Use the embedded DNS resolver in Docker containers to discover the service #}
resolver {{ matrix_synapse_reverse_proxy_companion_http_level_resolver }} valid=5s;
set $backend "{{ matrix_synapse_reverse_proxy_companion_client_api_addr }}";
proxy_pass http://$backend;
proxy_set_header Host $host;
client_body_buffer_size 25M;
client_max_body_size {{ matrix_synapse_reverse_proxy_companion_client_api_client_max_body_size_mb }}M;
proxy_max_temp_file_size 0;
}
}
{% if matrix_synapse_reverse_proxy_companion_federation_api_enabled %}
server {
listen 8048;
server_name {{ matrix_synapse_reverse_proxy_companion_hostname }};
server_tokens off;
root /dev/null;
gzip on;
gzip_types text/plain application/json;
{% if matrix_synapse_reverse_proxy_companion_synapse_workers_enabled %}
{% if generic_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappgeneric_worker
{{ render_locations_to_upstream(matrix_synapse_reverse_proxy_companion_synapse_generic_worker_federation_locations, 'generic_workers_upstream') }}
{% endif %}
{% if media_repository_workers | length > 0 %}
# https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository
{% for location in matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations %}
location ~ {{ location }} {
proxy_pass http://media_repository_workers_upstream$request_uri;
proxy_set_header Host $host;
client_body_buffer_size 25M;
client_max_body_size {{ matrix_synapse_reverse_proxy_companion_federation_api_client_max_body_size_mb }}M;
proxy_max_temp_file_size 0;
{% if matrix_synapse_reverse_proxy_companion_synapse_cache_enabled %}
proxy_buffering on;
proxy_cache {{ matrix_synapse_reverse_proxy_companion_synapse_cache_keys_zone_name }};
proxy_cache_valid any {{ matrix_synapse_reverse_proxy_companion_synapse_cache_proxy_cache_valid_time }};
proxy_force_ranges on;
add_header X-Cache-Status $upstream_cache_status;
{% endif %}
}
{% endfor %}
{% endif %}
{% endif %}
{% for configuration_block in matrix_synapse_reverse_proxy_companion_synapse_federation_api_additional_server_configuration_blocks %}
{{- configuration_block }}
{% endfor %}
location / {
{# Use the embedded DNS resolver in Docker containers to discover the service #}
resolver {{ matrix_synapse_reverse_proxy_companion_http_level_resolver }} valid=5s;
set $backend "{{ matrix_synapse_reverse_proxy_companion_federation_api_addr }}";
proxy_pass http://$backend;
proxy_set_header Host $host;
client_body_buffer_size 25M;
client_max_body_size {{ matrix_synapse_reverse_proxy_companion_federation_api_client_max_body_size_mb }}M;
proxy_max_temp_file_size 0;
}
}
{% endif %}

View file

@ -0,0 +1,13 @@
#jinja2: lstrip_blocks: "True"
# The default is aligned to the CPU's cache size,
# which can sometimes be too low.
# Thus, we ensure a larger bucket size value is used.
server_names_hash_bucket_size 64;
{% if matrix_synapse_reverse_proxy_companion_http_level_resolver %}
resolver {{ matrix_synapse_reverse_proxy_companion_http_level_resolver }};
{% endif %}
{% for configuration_block in matrix_synapse_reverse_proxy_companion_http_additional_server_configuration_blocks %}
{{- configuration_block }}
{% endfor %}

View file

@ -0,0 +1,66 @@
#jinja2: lstrip_blocks: "True"
# This is a custom nginx configuration file that we use in the container (instead of the default one),
# because it allows us to run nginx with a non-root user.
#
# For this to work, the default vhost file (`/etc/nginx/conf.d/default.conf`) also needs to be removed.
#
# The following changes have been done compared to a default nginx configuration file:
# - various temp paths are changed to `/tmp`, so that a non-root user can write to them
# - the `user` directive was removed, as we don't want nginx to switch users
worker_processes {{ matrix_synapse_reverse_proxy_companion_worker_processes }};
error_log /var/log/nginx/error.log warn;
pid /tmp/nginx.pid;
{% for configuration_block in matrix_synapse_reverse_proxy_companion_additional_configuration_blocks %}
{{- configuration_block }}
{% endfor %}
events {
worker_connections {{ matrix_synapse_reverse_proxy_companion_worker_connections }};
{% for configuration_block in matrix_synapse_reverse_proxy_companion_event_additional_configuration_blocks %}
{{- configuration_block }}
{% endfor %}
}
http {
proxy_temp_path /tmp/proxy_temp;
client_body_temp_path /tmp/client_temp;
fastcgi_temp_path /tmp/fastcgi_temp;
uwsgi_temp_path /tmp/uwsgi_temp;
scgi_temp_path /tmp/scgi_temp;
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
{% if matrix_synapse_reverse_proxy_companion_access_log_enabled %}
access_log /var/log/nginx/access.log main;
{% else %}
access_log off;
{% endif %}
proxy_connect_timeout {{ matrix_synapse_reverse_proxy_companion_proxy_connect_timeout }};
proxy_send_timeout {{ matrix_synapse_reverse_proxy_companion_proxy_send_timeout }};
proxy_read_timeout {{ matrix_synapse_reverse_proxy_companion_proxy_read_timeout }};
send_timeout {{ matrix_synapse_reverse_proxy_companion_send_timeout }};
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
server_tokens off;
#gzip on;
{# Map directive needed for proxied WebSocket upgrades #}
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
include /etc/nginx/conf.d/*.conf;
}

View file

@ -0,0 +1,53 @@
#jinja2: lstrip_blocks: "True"
[Unit]
Description=Synapse reverse-proxy companion
{% for service in matrix_synapse_reverse_proxy_companion_systemd_required_services_list %}
Requires={{ service }}
After={{ service }}
{% endfor %}
{% for service in matrix_synapse_reverse_proxy_companion_systemd_wanted_services_list %}
Wants={{ service }}
{% endfor %}
DefaultDependencies=no
[Service]
Type=simple
Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} kill matrix-synapse-reverse-proxy-companion 2>/dev/null || true'
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-synapse-reverse-proxy-companion 2>/dev/null || true'
ExecStart={{ devture_systemd_docker_base_host_command_docker }} run \
--rm \
--name=matrix-synapse-reverse-proxy-companion \
--log-driver=none \
--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
--cap-drop=ALL \
--read-only \
--tmpfs=/tmp:rw,noexec,nosuid,size={{ matrix_synapse_reverse_proxy_companion_tmp_directory_size_mb }}m \
{% if matrix_synapse_reverse_proxy_companion_synapse_cache_enabled %}
--tmpfs=/tmp/synapse-cache:rw,noexec,nosuid,size={{ matrix_synapse_reverse_proxy_companion_tmp_cache_directory_size_mb }}m\
{% endif %}
--network={{ matrix_synapse_reverse_proxy_companion_container_network }} \
{% if matrix_synapse_reverse_proxy_companion_container_client_api_host_bind_port %}
-p {{ matrix_synapse_reverse_proxy_companion_container_client_api_host_bind_port }}:8008 \
{% endif %}
{% if matrix_synapse_reverse_proxy_companion_container_federation_api_host_bind_port %}
-p {{ matrix_synapse_reverse_proxy_companion_container_federation_api_host_bind_port }}:8048 \
{% endif %}
--mount type=bind,src={{ matrix_synapse_reverse_proxy_companion_base_path }}/nginx.conf,dst=/etc/nginx/nginx.conf,ro \
--mount type=bind,src={{ matrix_synapse_reverse_proxy_companion_confd_path }},dst=/etc/nginx/conf.d,ro \
{{ matrix_synapse_reverse_proxy_companion_container_image }}
{% for network in matrix_synapse_reverse_proxy_companion_container_additional_networks %}
ExecStartPost={{ devture_systemd_docker_base_host_command_sh }} -c 'attempt=0; while [ $attempt -le 29 ]; do attempt=$(( $attempt + 1 )); if [ "`docker inspect -f {{ '{{.State.Running}}' }} matrix-synapse-reverse-proxy-companion 2> /dev/null`" = "true" ]; then break; fi; sleep 1; done; {{ devture_systemd_docker_base_host_command_docker }} network connect {{ network }} matrix-synapse-reverse-proxy-companion'
{% endfor %}
ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} kill matrix-synapse-reverse-proxy-companion 2>/dev/null || true'
ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-synapse-reverse-proxy-companion 2>/dev/null || true'
ExecReload={{ devture_systemd_docker_base_host_command_docker }} exec matrix-synapse-reverse-proxy-companion /usr/sbin/nginx -s reload
Restart=always
RestartSec=30
SyslogIdentifier=matrix-synapse-reverse-proxy-companion
[Install]
WantedBy=multi-user.target

View file

@ -24,3 +24,6 @@
- {'old': 'matrix_container_retries_delay', 'new': 'devture_playbook_help_container_retries_delay'}
- {'old': 'matrix_geturl_retries_count', 'new': 'devture_playbook_help_geturl_retries_count'}
- {'old': 'matrix_geturl_retries_delay', 'new': 'devture_playbook_help_geturl_retries_delay'}
- {'old': 'matrix_nginx_proxy_synapse_cache_path', 'new': 'matrix_synapse_reverse_proxy_companion_synapse_cache_path'}
- {'old': 'matrix_nginx_proxy_synapse_cache_enabled', 'new': 'matrix_synapse_reverse_proxy_companion_synapse_cache_enabled'}