Commit graph

530 commits

Author SHA1 Message Date
Slavi Pantaleev 5163aa643a Upgrade Synapse (1.51.0 -> 1.52.0)
This also removes the `matrix_synapse_version_arm64` variable we've
been dragging around for a long time.

Since https://github.com/matrix-org/synapse/pull/11810, a multiarch Synapse
container image (for AMD64 and ARM64) is released at the same time.
2022-02-08 15:13:41 +02:00
Marko Weltzer 819574b8ba
Merge branch 'spantaleev:master' into master 2022-02-05 21:37:53 +01:00
Marko Weltzer 7e5b88c3b7 fix: all praise the allmighty yamllinter 2022-02-05 21:32:54 +01:00
Slavi Pantaleev 86c36523df Replace ExecStopPost with ExecStop
Reverts b1b4ba501f, 90c9801c56, a3c84f78ca, ..

I haven't really traced it (yet), but on some servers, I'm observing
`ansible-playbook ... --tags=start` completing very slowly, waiting
to stop services. I can't reproduce this on all Matrix servers I manage.
I suspect that either the systemd version is to blame or that some
specific service is not responding well to some `docker kill/rm` command.

`ExecStop` seems to work great in all cases and it's what we've been
using for a very long time, so I'm reverting to that.
2022-02-05 12:13:36 +02:00
Slavi Pantaleev ad082b3b1b Fix self-building for Synapse v1.51.0 (requires BuildKit)
Synapse v1.51.0 requires to be built with BuildKit since
https://github.com/matrix-org/synapse/pull/11691

The `docker_image` Ansible module does not support BuildKit
(https://github.com/ansible-collections/community.general/issues/514),
so we had to switch to a `docker build` call.
2022-01-26 08:41:34 +02:00
Slavi Pantaleev cf46b7fed5 Upgrade Synapse (1.50.2 -> 1.51.0) 2022-01-25 14:19:40 +02:00
Catalan Lover f59f903c04
Update Synapse from 1.50.1 to 1.50.2
Fixes a issue with room version 1.

Merging has to wait until ARM images are built ofc.
2022-01-24 14:47:41 +01:00
Slavi Pantaleev bcfae766a1
Merge pull request #1567 from aaronraimist/antispam
Upgrade synapse-simple-antispam (0.0.3 -> 0.0.7)
2022-01-19 16:56:10 +02:00
Aaron Raimist ff94d815e1
Upgrade synapse-simple-antispam (0.0.3 -> 0.0.7) 2022-01-19 14:35:55 +00:00
Wm Salt Hale ad8d4740a7
Merge branch 'spantaleev:master' into default_room_version_9 2022-01-18 14:56:25 -08:00
Slavi Pantaleev 4cd44f117d Upgrade Synapse (1.50.0 -> 1.50.1)
v1.50.0 was found to be buggy for people using a `webclient` listener.
This is fixed in v1.50.1.

We don't use such a listener, so we weren't affected anyway.
2022-01-18 20:45:12 +02:00
Slavi Pantaleev 5f22371c23 Upgrade Synapse (1.49.2 -> 1.50.0) 2022-01-18 16:15:52 +02:00
Slavi Pantaleev 4e4fb98a65 Do not install fuse unless necessary
Discussed here: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1510
2022-01-08 14:14:46 +02:00
Slavi Pantaleev 2bd33e5cf2 Make --tags=register-user work for both Dendrite/Synapse
Also get rid of `--tags=update-user-password` in the
`matrix-dendrite` role, as what we had doesn't work.

We may be able to do it with some Ansible helper or something else.
For now, we'll omit this feature.
2022-01-07 15:59:35 +02:00
Slavi Pantaleev b1b4ba501f Replace ExecStop with ExecStopPost
ExecStopPost should allow us to clean up (docker kill + docker rm)
even if the ExecStart (docker run ..) command failed, and not just after
a graceful service stop was initiated.

Source: https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecStopPost=
2022-01-04 17:27:25 +02:00
Slavi Pantaleev 280c6c5424
Add |to_json 2021-12-27 21:45:13 +02:00
Alejo Diaz 297ed9ce36 Add encryption_enabled_by_default_for_room_type
This commit simply add encryption_enabled_by_default_for_room_type
variable.

Signed-off-by: Alejo Diaz <xlejo@protonmail.com>
2021-12-27 14:20:23 -03:00
Catalan Lover eb0c332f80
Update Synapse Version from 1.49.1 to 1.49.2
This upgrade is technically not needed due to 1.49.1 and 1.49.2 being identical with a lone fix to Debian packaging being the only change. 

Still some might want us to be on the absolutely latest version even tho these 2 are practically identical.

ARM64 has yet to be built so this has to wait for that before merge.
2021-12-21 19:29:56 +01:00
Catalan Lover 6c9f6c28de
Update Synapse from 1.49.0 to 1.49.1
This update fixes a sync issue that would cause some users to experience sync issues that could cause initial sync to completely fail.
2021-12-21 14:15:13 +01:00
Slavi Pantaleev 01b6bba9d7 Make Synapse's url_preview_accept_language default to 'en-US, en'
Related to https://github.com/matrix-org/synapse/issues/11604

Getting an upstream fix is preferable. In any case, it's probably nice
to have this defined explicitly in our configuration. This way, people
can more easily discover that they can override the URL preview
language.
2021-12-18 15:17:14 +02:00
Slavi Pantaleev 4625b34acc Fix failure trying to stop orphaned systemd services
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1461
2021-12-17 17:18:35 +02:00
Wm Salt Hale 3001b2d32d
Merge branch 'spantaleev:master' into default_room_version_9 2021-12-14 18:19:26 -08:00
Slavi Pantaleev 5be1310541 Upgrade Synapse (1.48.0 -> 1.49.0) 2021-12-14 16:54:35 +02:00
SkepticalWaves bbab82ee64
Fix email sending broken by #1406 2021-12-11 10:26:55 -05:00
Aaron Raimist 0071828503
Run workers-doc-to-yaml.sh
1472958e25 reverted some of the v3 changes. I'm not sure why. Running the `workers-doc-to-yaml.sh` script now puts them back 🤷‍♂️.
2021-12-08 08:37:55 +00:00
Slavi Pantaleev 1472958e25 Upgrade Synapse (1.47.1 -> 1.48.0) 2021-11-30 16:35:23 +02:00
Slavi Pantaleev 6b07ee3b58 Upgrade Synapse (1.47.0 -> 1.47.1) - security fixes
Learn more here: https://github.com/matrix-org/synapse/releases/tag/v1.47.1
2021-11-23 14:50:07 +02:00
Slavi Pantaleev 3b27ce2ff6
Merge pull request #1404 from aaronraimist/v3
Allow workers to serve new v3 APIs
2021-11-19 10:54:47 +02:00
rakshazi d41e9230da
expose smtp_user and smtp_pass to ansible configs (role: matrix-synapse) 2021-11-17 21:34:46 +02:00
Slavi Pantaleev e1a6d1e4b2 Upgrade Synapse (1.46.0 -> 1.47.0)
We had to remove UID/GID environment variables that we used to pass
to the Synapse container, because it was causing a problem after
https://github.com/matrix-org/synapse/pull/11209

We were using both `--user` and UID/GID environment variables until now.
2021-11-17 17:21:15 +02:00
Aaron Raimist f8fe68b385
Allow workers to serve new v3 APIs
1f196f59cb
2021-11-17 14:54:49 +00:00
boris runakov d3a9ec98de refactoring 2021-11-16 21:03:21 +02:00
boris runakov 1ec67f49b0 replaced 8008 where possible 2021-11-15 22:43:05 +02:00
Slavi Pantaleev c1bc7b9f93 Rename variables to prevent confusion
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1397
and https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1399
2021-11-15 14:56:11 +02:00
Slavi Pantaleev ba48aa70f7 Fix variable name typo
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1397
2021-11-15 14:52:08 +02:00
Slavi Pantaleev dc4452ac21
Merge branch 'master' into matrix-federation-api-port 2021-11-15 14:49:03 +02:00
Slavi Pantaleev edf63bfdd7
Add some to_json invocations 2021-11-15 14:48:25 +02:00
boris runakov 8c3e25de1b renamed var to matrix_synapse_container_federation_api_port 2021-11-15 13:01:22 +02:00
b 07496069c8 rellocating variables for consistency 2021-11-15 12:07:54 +02:00
b afccc2b11f make 8448 configurable instead of hard coded 2021-11-14 23:32:25 +02:00
b 7756cc4c8e replace port 8048 with matrix_synapse_container_default_federation_port 2021-11-14 20:30:13 +02:00
Slavi Pantaleev 735c966ab6 Disable systemd services when stopping to uninstall them
Until now, we were leaving services "enabled"
(symlinks in /etc/systemd/system/multi-user.target.wants/).

We clean these up now. Broken symlinks may still exist in older
installations that enabled/disabled services. We're not taking care
to fix these up. It's just a cosmetic defect anyway.
2021-11-10 17:39:21 +02:00
Slavi Pantaleev 7b8b595e81 Upgrade Synapse (1.45.1 -> 1.46.0) 2021-11-02 17:42:13 +02:00
Slavi Pantaleev 5dc2868269 Upgrade Synapse (1.45.0 -> 1.45.1) 2021-10-20 15:08:07 +03:00
Slavi Pantaleev 1dab178a44 Upgrade Synapse (1.44.0 -> 1.45.0) 2021-10-19 16:25:00 +03:00
Wm Salt Hale a1cbd5459c
Update Synapse default room version (6 -> 9)
From the [Synapse 1.43.0 release highlights](https://matrix.org/blog/2021/09/21/synapse-1-43-0-released):
> Asks clients to prefer [room version 9](https://github.com/matrix-org/matrix-doc/pull/3375) when creating restricted rooms ([#10772](https://github.com/matrix-org/synapse/issues/10772)), via the API defined in [MSC3244: room version capabilities](https://github.com/matrix-org/matrix-doc/pull/3244).
2021-10-12 09:49:16 -07:00
Slavi Pantaleev 278bbae4d5 Upgrade Synapse (1.43.0 -> 1.44.0) 2021-10-05 17:13:21 +03:00
Slavi Pantaleev 7934cde90f Upgrade Synapse (1.42.0 -> 1.43.0)
Related to:

- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1287
- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1288

.. but does it in a more complete way
2021-09-21 18:17:50 +03:00
Slavi Pantaleev 0faad74a44 Upgrade Synapse (1.41.1 -> 1.42.0) 2021-09-07 20:44:18 +03:00
Catalan Lover 777ba6bc5a
Upgrade to Synaspe v 1.41.1 (Security Update)
Synapse 1.41.1 Patches 2 exploits that can reveal information about rooms an user is not supposed to have access to information about.
2021-08-31 14:54:23 +02:00
Slavi Pantaleev a4db9557db Update homeserver.yaml to match the one in Synapse v1.41.0
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1247
2021-08-25 09:51:50 +03:00
Slavi Pantaleev bb0e59de48
Merge pull request #1250 from wol-win/worker_trial
Endpoint changes for Client and media API due to migration to 1.41.0
2021-08-25 09:37:43 +03:00
Wolfgang Winter ee75d35193 Endpoint changes for Client and media API due to migration to 1.41.0 2021-08-24 21:46:38 +02:00
Catalan Lover 1e070f1e56
Update Synapse from 1.40.0 to 1.41.0 2021-08-24 19:09:28 +02:00
Toni Spets bce94c5860 Allow configuring synapse database transaction limit 2021-08-10 21:09:28 +03:00
Slavi Pantaleev 05ad620386 Update homeserver.yaml to keep up with Synapse v1.40.0
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1225
2021-08-10 17:36:44 +03:00
WobbelTheBear 2469ada520
Update Synapse (1.39.0 -> 1.40.0) 2021-08-10 15:32:07 +02:00
Slavi Pantaleev 179775b92d Upgrade Synapse (1.38.1 -> 1.39.0) 2021-07-29 18:51:22 +03:00
Slavi Pantaleev 112d8d8243 Upgrade Synapse (1.38.0 -> 1.38.1) 2021-07-22 19:48:29 +03:00
Janar Juusu c198320f65
feat: update synapse to 1.38.0 2021-07-13 20:56:47 +03:00
Neutron 1ad9bba88f Skip importing validate_config task when Synapse is disabled 2021-07-02 20:58:57 +03:00
Davy Landman c9d73c6606
Updating to latest synapse release (performance regression)
https://github.com/matrix-org/synapse/releases/tag/v1.37.1
2021-06-30 16:15:00 +02:00
Slavi Pantaleev 3da4b684a5 Upgrade Synapse (1.36.0 -> 1.37.0) 2021-06-29 13:55:09 +03:00
Slavi Pantaleev 3dd32d2512 Update worker endpoints
Should have been part of 9daeb39710
2021-06-15 19:28:21 +03:00
Slavi Pantaleev 9daeb39710 Upgrade Synapse (1.35.1 -> 1.36.0) 2021-06-15 19:25:41 +03:00
Slavi Pantaleev fa76c1ee5b Do not run self-build Ansible version-check, if component not enabled
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1108
2021-06-08 08:59:02 +03:00
Slavi Pantaleev ab08a4f60e Upgrade Synapse (1.35.0 -> 1.35.1) 2021-06-03 16:27:15 +03:00
Slavi Pantaleev b1f1c28ef0 Upgrade Synapse (1.34.0 -> 1.35.0) 2021-06-01 19:14:59 +03:00
pushytoxin bee14550ab Fix local/bin scripts autocompletion by adding rx perms to everyone
It's mildly annoying when trying to execute these scripts while logged
in as a regular user, as the missing execute permissions will hinder
autocompletion even when trying to use with sudo.

These shell scripts don't contain secrets, but may fail when ran by a
regular user. The failure is due to the lack of access to the /matrix
directory, and does not result in any damage.
2021-05-28 10:39:27 +02:00
Slavi Pantaleev b3351d2a53
Merge pull request #1083 from haghighi-ahmad/active-directory-support-for-ldap_auth_provider
Synapse LDAP auth: add support for Active Directory
2021-05-26 10:53:27 +03:00
Slavi Pantaleev d61fe94bae
Fix incorrect path in Mjolnir uninstallation tasks 2021-05-26 10:52:15 +03:00
BG 763952395b Adding mjolnir antispam synapse modul. 2021-05-25 16:43:30 +02:00
Michael-GMH 6f40d78353 fix random edits to upstream 2021-05-25 21:25:40 +08:00
Michael-GMH 85777e8f96 merge with upstream 2021-05-25 21:08:00 +08:00
Ahmad Haghighi 209d59070e Avoiding if(s), fix #1083
Conversation: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1083#discussion_r638671860

Signed-off-by: Ahmad Haghighi <haghighi@fedoraproject.org>
2021-05-25 15:41:58 +04:30
Ahmad Haghighi ee088d5d46 Synapse LDAP auth: add support for Active Directory
Signed-off-by: Ahmad Haghighi <haghighi@fedoraproject.org>
2021-05-25 15:36:41 +04:30
Slavi Pantaleev 47b4608b96 Fail in a friendlier way when trying to self-build on Ansible <= 2.8
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1070

Related discussion here: 1ab507349c (commitcomment-51108407)
2021-05-21 11:15:05 +03:00
Slavi Pantaleev 1ab507349c Fix self-building for various components on Ansible < 2.8
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1070
2021-05-20 08:43:20 +03:00
Slavi Pantaleev d156c8caa2 Upgrade Synapse (1.33.2 -> 1.34.0) 2021-05-17 14:58:07 +03:00
sakkiii 8fc55b30c5
Upgrade Synapse (1.33.1 -> 1.33.2)
This release fixes a denial of service attack (CVE-2021-29471) against Synapse's push rules implementation. Server admins are encouraged to upgrade.

Ref: https://github.com/matrix-org/synapse/releases/tag/v1.33.2
2021-05-11 19:06:30 +05:30
Michael-GMH 4e6f6e179b GMH 0.4.6 update 2021-05-10 18:50:10 +08:00
Slavi Pantaleev 61220ea487 Upgrade Synapse (1.33.0 -> 1.33.1) 2021-05-06 20:47:09 +03:00
Slavi Pantaleev d4d1e2e922 Upgrade Synapse (1.32.2 -> 1.33.0) 2021-05-05 19:18:53 +03:00
Slavi Pantaleev f6b371164c Remove useless variable 2021-04-23 07:07:18 +03:00
Slavi Pantaleev e3fa3e12bc Upgrade Synapse (1.31 -> 1.32.2) 2021-04-22 14:22:07 +03:00
Slavi Pantaleev 378fabf177 Revert "Upgrade Synapse (1.31 -> 1.32.1)"
This reverts commit 1fb54a37cb.

Seems like it's been pulled or something. It used to exist, but not
anymore. Not sure what's going on.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1017

Related to
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-21 23:36:58 +03:00
Slavi Pantaleev 1fb54a37cb Upgrade Synapse (1.31 -> 1.32.1)
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-21 18:47:15 +03:00
Slavi Pantaleev ca786cc343 Revert "Upgrade Synapse (1.31 -> 1.32)"
This reverts commit f825c7c263.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-20 23:40:55 +03:00
Slavi Pantaleev f825c7c263 Upgrade Synapse (1.31 -> 1.32) 2021-04-20 17:47:34 +03:00
Ahmad Haghighi e335f3fc77 rename matrix_global_registry to matrix_container_global_registry_prefix related to #990
Signed-off-by: Ahmad Haghighi <haghighi@fedoraproject.org>
2021-04-12 17:23:55 +04:30
Ahmad Haghighi f52a8b6484 use custom docker registry 2021-04-12 17:23:55 +04:30
Slavi Pantaleev 4830b7d830 Upgrade Synapse for ARM64 (1.30.1 -> 1.31.0) 2021-04-06 17:22:25 +03:00
Slavi Pantaleev 3f426de599 Upgrade Synapse (1.30.1 -> 1.31.0) 2021-04-06 16:00:10 +03:00
Slavi Pantaleev f183add44d
Merge pull request #977 from aaronraimist/simple-antispam
Upgrade synapse-simple-antispam (0.0.1 -> 0.0.3)
2021-04-03 08:45:14 +03:00
Aaron Raimist c43bd412dd
Upgrade synapse-simple-antispam (0.0.1 -> 0.0.3) 2021-04-02 18:08:08 -05:00
Slavi Pantaleev 49868db3de Upgrade Synapse for ARM64 (1.30.0 -> 1.30.1) 2021-03-26 16:48:15 +02:00
Slavi Pantaleev 94487dc6a7 Upgrade Synapse for amd64 (1.30.0 -> 1.30.1) 2021-03-26 15:37:11 +02:00
Slavi Pantaleev d09609daa8 Fix Jinja2 syntax error
Fixes a regression introduced in ffe649a240
2021-03-22 17:13:10 +02:00
Slavi Pantaleev 6a3433fbad Update Synapse for ARM64 (1.29.0 -> 1.30.0)
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/958
2021-03-22 16:43:23 +02:00
Slavi Pantaleev ffe649a240 Update homeserver.yaml to keep up with Synapse v1.30.0
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/958
2021-03-22 16:43:10 +02:00
rakshazi 74106f2a80
Updated synapse 1.29.0 -> 1.30.0 2021-03-22 14:03:42 +00:00
Slavi Pantaleev f99dcd611f Pass proper UID/GID to Synapse
Fixes a regression caused by a5ee39266c.

If the user id and group id were different than 991:991
(which used to be a hardcoded default for us long ago),
there was a mismatch between what Synapse was trying to use (991:991)
and what it was actually started with (in `--user=..`). It was then
trying to change ownership, which was failing.

This was mostly affecting newer installations which were not using the
991:991 defaults we had long ago (since a1c5a197a9).
2021-03-19 16:44:10 +02:00
Slavi Pantaleev a5ee39266c Go through start.py when launching Synapse
This allows us to benefit from helpful things it does for us,
like enabling jemalloc: https://github.com/matrix-org/synapse/pull/8553

We weren't going through `start.py` before, because it was causing some
conflict with our `docker run --user=...` stuff, but it doesn't seem
to be a problem anymore.

Having done this, we won't need to do things like
https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/941
anymore.
2021-03-19 08:16:59 +02:00
Pablo Montepagano 52fe8a05b0 Adding vars to synapse for private servers. 2021-03-14 00:39:44 -03:00
Slavi Pantaleev 9b72384df7 Upgrade Synapse (1.28.0 -> 1.29.0) 2021-03-08 17:24:09 +02:00
Slavi Pantaleev ccf5915874 Upgrade Synapse for ARM64 (v1.26.0 -> v1.28.0) 2021-02-25 19:09:46 +02:00
Slavi Pantaleev ae091d7b2d Upgrade Synapse (v1.27.0 -> v1.28.0) 2021-02-25 13:40:35 +02:00
Slavi Pantaleev 2ef1d9c537 Make healthchecks work for Synapse worker containers
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/456
2021-02-24 07:59:14 +02:00
rakshazi 2f887f292c
added "matrix_%SERVICE%_version" variable to all roles, use it in "matrix_%SERVICE%_docker_image" var (preserving backward-compatibility) 2021-02-20 19:08:28 +02:00
Slavi Pantaleev d94d0e2ca5
Merge pull request #456 from eMPee584/synapse-workers
Synapse workers
2021-02-19 11:40:36 +02:00
Slavi Pantaleev 9dc87bb948 Add Synapse worker presets for easier configuration
Adding more presets in the future would be nice.
2021-02-19 11:38:47 +02:00
Slavi Pantaleev 2f732e4234 Update Synapse worker endpoints 2021-02-19 11:36:14 +02:00
Slavi Pantaleev 217b4a8808 Release Synapse v1.27.0 to ARM32 via self-building
Related to: https://matrix.org/blog/2021/02/18/synapse-1-27-0-released#dropping-armv7-docker-images
2021-02-19 09:10:16 +02:00
Slavi Pantaleev d6c4d41c2b Define instanceId property on workers
This give us the possibility to run multiple instances of
workers that that don't expose a port.

Right now, we don't support that, but in the future we could
run multiple `federation_sender` or `pusher` workers, without
them fighting over naming (previously, they'd all be named
something like `matrix-synapse-worker-pusher-0`, because
they'd all define `port` as `0`).
2021-02-18 18:19:51 +02:00
Slavi Pantaleev d33483b8ce Document that Synapse pusher worker instances are shardable
Related to:
- https://github.com/matrix-org/synapse/pull/9407
- https://github.com/matrix-org/synapse/pull/7855
2021-02-16 17:45:41 +02:00
Slavi Pantaleev daae74b074 Merge branch 'master' into synapse-workers 2021-02-16 17:31:40 +02:00
Slavi Pantaleev 521160c12f Upgrade Synapse (v1.26.0 -> v1.27.0) 2021-02-16 17:30:48 +02:00
Slavi Pantaleev 61e427d690 Do not let people enable more than 1 federation_sender worker 2021-02-15 11:37:03 +02:00
Slavi Pantaleev 85a05f38e8 Allow Synapse worker list to be generated dynamically
This leads to much easier management and potential safety
features (validation). In the future, we could try to avoid port
conflicts as well, but it didn't seem worth the effort to do it now.
Our port ranges seem large enough.

This can also pave the way for a "presets" feature
(similar to `matrix_nginx_proxy_ssl_presets`) which makes it even easier
for people to configure worker counts.
2021-02-15 11:25:35 +02:00
Slavi Pantaleev 43059bb040 Fix metrics listeners for Synapse workers
`::` leads to errors like:

> socket.gaierror: [Errno -9] Address family for hostname not supported
2021-02-15 11:19:07 +02:00
Slavi Pantaleev 453a4ec2d8 Relocate tasks related to Synapse workers 2021-02-15 11:18:47 +02:00
Slavi Pantaleev 5cfeae806b Merge branch 'master' into synapse-workers 2021-02-14 13:00:57 +02:00
Slavi Pantaleev 7e8e95a09a Make S3-mounting path configurable
This will make data migration easier.
2021-02-09 22:05:07 +02:00
Béla Becker 2edc9cb83c Name the Synapse database on state compression import
Fixes:
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/833
2021-01-28 17:54:02 +01:00
Slavi Pantaleev e7f3f7c431 Enable /devices endpoint for generic workers 2021-01-27 22:18:47 +02:00
Slavi Pantaleev 1cd2a218de Merge branch 'master' into synapse-workers 2021-01-27 21:41:54 +02:00
Slavi Pantaleev c6feb0b99e Upgrade Synapse (v1.25.0 -> v1.26.0) 2021-01-27 21:41:47 +02:00
Slavi Pantaleev d98a1ceadd Merge branch 'master' into synapse-workers 2021-01-27 10:27:17 +02:00
Slavi Pantaleev 512f42aa76 Do not report docker kill/rm attempts as errors
These are just defensive cleanup tasks that we run.
In the good case, there's nothing to kill or remove, so they trigger an
error like this:

> Error response from daemon: Cannot kill container: something: No such container: something

and:

> Error: No such container: something

People often ask us if this is a problem, so instead of always having to
answer with "no, this is to be expected", we'd rather eliminate it now
and make logs cleaner.

In the event that:
- a container is really stuck and needs cleanup using kill/rm
- and cleanup fails, and we fail to report it because of error
suppression (`2>/dev/null`)

.. we'd still get an error when launching ("container name already in use .."),
so it shouldn't be too hard to investigate.
2021-01-27 10:22:46 +02:00
Slavi Pantaleev a535226210 Stop/disable unnecessary worker services before deleting them 2021-01-25 15:20:37 +02:00
Slavi Pantaleev 66cdc7bf5a Clean up worker.yaml generation a bit and make it more flexible 2021-01-25 13:02:01 +02:00
Slavi Pantaleev 1462409b34 Fix worker listening addresses
Not specifying bind addresses for the worker resulted in this warning:

> synapse.app - 47 - WARNING - None - Failed to listen on 0.0.0.0, continuing because listening on [::]

Additionally, metrics listening only on 127.0.0.1 seems like a no-op.
Only having it accessible from within the container is likely not what
we intend. Changed that to all interfaces as well.

Whether it actually gets exposed or not depends on the systemd service
and `matrix_synapse_workers_container_host_bind_address`.
2021-01-25 12:29:47 +02:00
Slavi Pantaleev 01747c8cc4 Prevent Synapse warning about enabling metric listeners with enable_metrics: false
> synapse.app.generic_worker - 606 - WARNING - None - Metrics listener configured, but enable_metrics is not True!
2021-01-25 12:24:12 +02:00
Slavi Pantaleev 70796703d3 Run Synapse workers in their own containers
This switches the `docker exec` method of spawning
Synapse workers inside the `matrix-synapse` container with
dedicated containers for each worker.

We also have dedicated systemd services for each worker,
so this are now:
- more consistent with everything else (we don't use systemd
instantiated services anywhere)
- we don't need the "parse systemd instance name into worker name +
port" part
- we don't need to keep track of PIDs manually
- we don't need jq (less depenendencies)
- workers dying would be restarted by systemd correctly, like any other
service
- `docker ps` shows each worker separately and we can observe resource
usage
2021-01-25 12:14:46 +02:00
Slavi Pantaleev da50fb27a0 Whitelist /_matrix/key requests for going to generic workers on the federation port 2021-01-25 09:46:50 +02:00
Slavi Pantaleev 63301b0ef1 Improvements around Synapse worker/metrics ports exposure
There was a `matrix_nginx_proxy_enabled|default(False)` check, but:
- it didn't seem to work reliably for some reason (hmm)
- referring to a `matrix_nginx_proxy_*` variable from within the
  `matrix-synapse` role is not ideal
- exposing always happened on `127.0.0.1`, which may not be good enough
  for some rarer setups (where the own webserver is external to the host)
2021-01-25 08:25:43 +02:00
Slavi Pantaleev 5ca68210cd Do not handle /_matrix/federation on client-server port, nor /_matrix/client stuff on federation port
I guess it didn't hurt to do it until now, but it's not great serving
federation APIs on the client-server API port, etc.

matrix-corporal doesn't work yet (still something to be solved in the
future), but its firewalling operations will also be sabotaged
by Client-Server APIs being served on the federation port (it's a way to get around its firewalling).
2021-01-24 22:22:57 +02:00
Slavi Pantaleev cc5cf0d725 Load roles/matrix-synapse/vars/workers.yml earlier to not break --tags=setup-nginx-proxy
If we load it at runtime, during matrix-synapse role execution,
it's good enough for matrix-synapse and all roles after that,
but.. it breaks when someone uses `--tags=setup-nginx-proxy` alone.

The downside of including this vars file like this in `setup.yml`
is that the variables contained in it cannot be overriden by the user
(in their inventory's `vars.yml`).
... but it's not like overriding these variables was possible anyway
when including them at runtime.
2021-01-24 20:19:55 +02:00
Marcel Partap edc21f15e5 Restrict publishing worker (metrics) ports to localhost 2021-01-24 08:53:09 +01:00
Marcel Partap 183adec3d8 Merge remote-tracking branch 'origin/master' into synapse-workers 2021-01-23 15:04:11 +01:00
Marcel Partap c8f051a42d Track workers endpoint list in repo instead of regenerating on user side 2021-01-23 14:44:36 +01:00
Marcel Partap f2c7d79238 Drop probably incorrect comment from synapse homeserver.yaml.j2 2021-01-23 14:44:36 +01:00
Slavi Pantaleev 95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
Slavi Pantaleev 1692a28fe4 Work around annoying Docker warning about undefined $HOME
> WARNING: Error loading config file: .dockercfg: $HOME is not defined

.. which appeared in Docker 20.10.
2021-01-15 00:23:01 +02:00
Slavi Pantaleev 05ca9357a8 Add .service suffix to systemd units list
We'll be adding `.timer` units later on, so it's good to be
more explicit.
2021-01-14 23:02:10 +02:00
Slavi Pantaleev b15da29ebb Bump Synapse to v1.25.0 for ARM 2021-01-14 10:41:47 +02:00
Slavi Pantaleev 24100342e1 Tell people that federation_ip_range_blacklist is gone
Related to d5945c6e78
2021-01-13 13:47:51 +02:00
Slavi Pantaleev d5945c6e78 Upgrade Synapse (v1.24.0 -> v1.25.0) for amd64 2021-01-13 13:02:49 +02:00
Marcel Partap cd8100544b Merge remote-tracking branch 'origin/master' into synapse-workers
Sync with upstream
2021-01-08 20:58:50 +01:00
Slavi Pantaleev 6e1dfb62f0 Rename some doc files and commands related to importing
Since we'll likely have generic SQLite database importing
via [pgloader](https://pgloader.io/) for migrating bridge
databases from SQLite to Postgres, we'd rather avoid
calling the "import Synapse SQLite database" command
as just `--tags=import-sqlite-db`.

Similarly, for the media store, we'd like to mention that it's
related to Synapse as well.

We'd like to be more explicit, so as to be less confusing,
especially in light of other homeserver implementations
coming in the future.
2020-12-14 01:51:00 +02:00