Commit graph

37 commits

Author SHA1 Message Date
Slavi Pantaleev 5f3f460cda Restore support for appservice and user_dir workers 2022-09-15 10:06:56 +03:00
Slavi Pantaleev 226c550ffa Add support for stream writer Synapse workers
As stream writer workers are also powered by the `generic_worker`
Synapse app, this necessitated that we provide means for distinguishing
between them and regular `generic_workers`.

I've also taken the time to optimize nginx configuration generation
(more Jinja2 macro usage, less duplication).

Worker names have also changed.
Workers are now named sequentially like this:
- `matrix-synapse-worker-0-generic`
- `matrix-synapse-worker-1-stream-writer-typing`
- `matrix-synapse-worker-2-pusher`

instead of `matrix-synapse-worker_generic_worker-18111` (indexed with a
port number).

People who modify `matrix_synapse_workers_enabled_list` directly will
need to adjust their configuration.
2022-09-15 08:10:04 +03:00
Slavi Pantaleev f12206676f Upgrade Synapse (v1.66.0 -> 1.67.0) and remove frontend_proxy workers
`frontend_proxy` workers have been superseded by `generic_worker` workers.
Related to https://github.com/matrix-org/synapse/pull/13645
2022-09-13 15:45:50 +03:00
Slavi Pantaleev c1849ae888 Fix the remaining var-spacing ansible-lint errors
Reference: https://ansible-lint.readthedocs.io/en/latest/default_rules/#var-spacing
2022-07-18 15:33:41 +03:00
Slavi Pantaleev cac9bf2637 Fix syntax error regression 2022-07-18 15:13:09 +03:00
Slavi Pantaleev ddf18eadc7 More ansible-lint fixes 2022-07-18 13:01:17 +03:00
Slavi Pantaleev 6a573399ae Upgrade Synapse (v1.60.0 -> v1.61.0) 2022-06-14 17:15:27 +03:00
Slavi Pantaleev 78204619ea Stop using deprecated (in Synapse v1.59) user_dir and appservice workers
Source: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types

As an alternative, we should probably find a way to run one or a few
more generic workers (which will handle appservice and user_dir stuff) and
update `homeserver.yaml` so that it would point to the name of these workers using
`notify_appservices_from_worker` and `update_user_directory_from_worker` options.

For now, this solves the deprecation, so we can have a peace of mind
going forward.

We're force-setting these worker counts to 0, so that we can clean up
existing homeservers which use these worker types. In the future, these
options will either be removed or repurposed (so that they transparently
create more generic workers that handle user_dir/appservice loads).
2022-05-31 16:35:50 +03:00
Slavi Pantaleev 83b7fcee45 Do not proxy some endpoints to the generic Synapse worker
These endpoints should not be proxied to a generic Synapse worker
without other preparation (setting up stream writers, sending traffic
to a specific stream writer, etc.).

Disabling them for now. In the future, we'd like to fix up our awk
script to disable them automatically.

This is a fix up for 058fedff91
2022-05-07 09:39:19 +02:00
Slavi Pantaleev 058fedff91 Fix "endpoint seems conditional" determination in workers-doc-to-yaml.awk"
This prevented us from keeping our workers reverse-proxying definitions
updated since Synapse v1.54.0.

The last `workers.md` file we could parse is at commit
02632b3504ad4512c5f5a4f859b3fe326b19c788.
Parsing regressed at commit c56bfb08bc071368db23f3b1c593724eb4f205f0,
because the introduction message for `synapse.app.generic_worker` said
"If":

> If a worker is set up to handle a..

.. which made the AWK script think that definitions below were
conditional (which they're not in this case).

This patch fixes up the regex for determining if a line is conditional
or not, so that it doesn't trip up. Hopefully, it doesn't miss something
important.
2022-05-06 09:08:45 +02:00
Slavi Pantaleev ef7acce94b Upgrade Synapse (1.52.0 -> 1.53.0) 2022-02-22 15:34:10 +02:00
Marko Weltzer 7e5b88c3b7 fix: all praise the allmighty yamllinter 2022-02-05 21:32:54 +01:00
Slavi Pantaleev 5be1310541 Upgrade Synapse (1.48.0 -> 1.49.0) 2021-12-14 16:54:35 +02:00
Aaron Raimist 0071828503
Run workers-doc-to-yaml.sh
1472958e25 reverted some of the v3 changes. I'm not sure why. Running the `workers-doc-to-yaml.sh` script now puts them back 🤷‍♂️.
2021-12-08 08:37:55 +00:00
Slavi Pantaleev 1472958e25 Upgrade Synapse (1.47.1 -> 1.48.0) 2021-11-30 16:35:23 +02:00
Slavi Pantaleev 3b27ce2ff6
Merge pull request #1404 from aaronraimist/v3
Allow workers to serve new v3 APIs
2021-11-19 10:54:47 +02:00
Slavi Pantaleev e1a6d1e4b2 Upgrade Synapse (1.46.0 -> 1.47.0)
We had to remove UID/GID environment variables that we used to pass
to the Synapse container, because it was causing a problem after
https://github.com/matrix-org/synapse/pull/11209

We were using both `--user` and UID/GID environment variables until now.
2021-11-17 17:21:15 +02:00
Aaron Raimist f8fe68b385
Allow workers to serve new v3 APIs
1f196f59cb
2021-11-17 14:54:49 +00:00
boris runakov 1ec67f49b0 replaced 8008 where possible 2021-11-15 22:43:05 +02:00
Slavi Pantaleev 7934cde90f Upgrade Synapse (1.42.0 -> 1.43.0)
Related to:

- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1287
- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1288

.. but does it in a more complete way
2021-09-21 18:17:50 +03:00
Slavi Pantaleev 0faad74a44 Upgrade Synapse (1.41.1 -> 1.42.0) 2021-09-07 20:44:18 +03:00
Wolfgang Winter ee75d35193 Endpoint changes for Client and media API due to migration to 1.41.0 2021-08-24 21:46:38 +02:00
Slavi Pantaleev 3dd32d2512 Update worker endpoints
Should have been part of 9daeb39710
2021-06-15 19:28:21 +03:00
Slavi Pantaleev f6b371164c Remove useless variable 2021-04-23 07:07:18 +03:00
Slavi Pantaleev 3f426de599 Upgrade Synapse (1.30.1 -> 1.31.0) 2021-04-06 16:00:10 +03:00
Slavi Pantaleev ae091d7b2d Upgrade Synapse (v1.27.0 -> v1.28.0) 2021-02-25 13:40:35 +02:00
Slavi Pantaleev 2f732e4234 Update Synapse worker endpoints 2021-02-19 11:36:14 +02:00
Slavi Pantaleev e7f3f7c431 Enable /devices endpoint for generic workers 2021-01-27 22:18:47 +02:00
Slavi Pantaleev da50fb27a0 Whitelist /_matrix/key requests for going to generic workers on the federation port 2021-01-25 09:46:50 +02:00
Slavi Pantaleev 5ca68210cd Do not handle /_matrix/federation on client-server port, nor /_matrix/client stuff on federation port
I guess it didn't hurt to do it until now, but it's not great serving
federation APIs on the client-server API port, etc.

matrix-corporal doesn't work yet (still something to be solved in the
future), but its firewalling operations will also be sabotaged
by Client-Server APIs being served on the federation port (it's a way to get around its firewalling).
2021-01-24 22:22:57 +02:00
Marcel Partap c8f051a42d Track workers endpoint list in repo instead of regenerating on user side 2021-01-23 14:44:36 +01:00
Slavi Pantaleev 9952ec6c16 Upgrade Synapse (v1.18.0 -> v1.19.0) 2020-08-17 17:02:40 +03:00
Slavi Pantaleev 10b3ceff72 Make Matrix federation port configurable
Fixes #523 (Github Issue).
2020-06-09 08:29:03 +03:00
Slavi Pantaleev b440d5b73c Remove some fact definitions during runtime 2019-08-22 08:00:02 +03:00
Slavi Pantaleev bb816df557 Move mautrix telegram and whatsapp into separate roles
The goal is to move each bridge into its own separate role.
This commit starts off the work on this with 2 bridges:
- mautrix-telegram
- mautrix-whatsapp

Each bridge's role (including these 2) is meant to:

- depend only on the matrix-base role

- integrate nicely with the matrix-synapse role (if available)

- integrate nicely with the matrix-nginx-proxy role (if available and if
required). mautrix-telegram bridge benefits from integrating with
it.

- not break if matrix-synapse or matrix-nginx-proxy are not used at all

This has been provoked by #174 (Github Issue).
2019-05-14 23:47:22 +09:00
Slavi Pantaleev a43bcd81fe Rename some variables 2019-02-28 11:51:09 +02:00
Slavi Pantaleev c10182e5a6 Make roles more independent of one another
With this change, the following roles are now only dependent
on the minimal `matrix-base` role:
- `matrix-corporal`
- `matrix-coturn`
- `matrix-mailer`
- `matrix-mxisd`
- `matrix-postgres`
- `matrix-riot-web`
- `matrix-synapse`

The `matrix-nginx-proxy` role still does too much and remains
dependent on the others.

Wiring up the various (now-independent) roles happens
via a glue variables file (`group_vars/matrix-servers`).
It's triggered for all hosts in the `matrix-servers` group.

According to Ansible's rules of priority, we have the following
chain of inclusion/overriding now:
- role defaults (mostly empty or good for independent usage)
- playbook glue variables (`group_vars/matrix-servers`)
- inventory host variables (`inventory/host_vars/matrix.<your-domain>`)

All roles default to enabling their main component
(e.g. `matrix_mxisd_enabled: true`, `matrix_riot_web_enabled: true`).
Reasoning: if a role is included in a playbook (especially separately,
in another playbook), it should "work" by default.

Our playbook disables some of those if they are not generally useful
(e.g. `matrix_corporal_enabled: false`).
2019-01-16 18:05:48 +02:00