Commit graph

151 commits

Author SHA1 Message Date
Slavi Pantaleev ba51997f7b (BC Break) Redo how metrics are exposed to external Prometheus servers 2022-06-23 17:55:07 +03:00
Slavi Pantaleev 323f5aa60d Synchronize homeserver.yaml config with the one from Synapse 1.61.0 2022-06-17 15:26:23 +03:00
Slavi Pantaleev 78204619ea Stop using deprecated (in Synapse v1.59) user_dir and appservice workers
Source: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types

As an alternative, we should probably find a way to run one or a few
more generic workers (which will handle appservice and user_dir stuff) and
update `homeserver.yaml` so that it would point to the name of these workers using
`notify_appservices_from_worker` and `update_user_directory_from_worker` options.

For now, this solves the deprecation, so we can have a peace of mind
going forward.

We're force-setting these worker counts to 0, so that we can clean up
existing homeservers which use these worker types. In the future, these
options will either be removed or repurposed (so that they transparently
create more generic workers that handle user_dir/appservice loads).
2022-05-31 16:35:50 +03:00
Slavi Pantaleev 677a2fc503 Fix compatibility with ansible=6 / ansible-core=2.13
Details here: https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_6.html#id36

Basically:

```yaml
- name: Prior to 2.13
  debug:
    msg: '[1] + {{ [2] }}'

- name: 2.13 and forward
  debug:
    msg: '{{ [1] + [2] }}'
```

Interestingly, we had been using the new/safe syntax in lofs of places.

We were using the broken one in many others though. Hopefully all
instances were fixed by this patch.
2022-05-18 15:43:39 +03:00
Aine 502ea21fba
add retires to all get_url actions 2022-04-19 22:01:14 +03:00
Lunar 471806e7bd
Increase default async time for rust-synapse-compress-state
Increase the async timeout value defaults, as larger Matrix servers need more time to complete.
2022-04-17 20:27:04 -05:00
Slavi Pantaleev 2df993977a Ensure git cloning when self-building is done with the matrix user, not root
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1749
2022-04-14 08:52:37 +03:00
Slavi Pantaleev 85627b59ad Make synapse-compress-state in-container binary path configurable
This makes it easier to use another container image for
rust-synapse-compress-state, which may be storing the binary at another
path.
2022-03-26 10:04:21 +02:00
Aine 2da3768b20
Added retries to the docker pulls (#1701) 2022-03-17 17:37:11 +02:00
Jim Myhrberg eeca3c8dca
fix: avoid yaml being wrapped at column 80 via to_nice_yaml
The `to_nice_yaml` helper will by default wrap any string YAML values on
the first space after column 80. This can in worst case yield invalid
YAML syntax. More details in Ansible's documentation here:

https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json

In short, you need to explicitly provide a custom width argument of a
high number of some kind to avoid the line wrapping.
2022-03-16 01:10:26 +00:00
Slavi Pantaleev f44ca0c7c2 Add support for matrix_encryption_disabler
Related to https://github.com/matrix-org/synapse/issues/4401

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1621
2022-02-12 09:25:24 +02:00
Slavi Pantaleev 5a69c899a3 Upgrade matrix-synapse-shared-secret-auth (1.0.2 -> 2.0.2)
For now, we disable the new `com.devture.shared_secret_auth` login type
by default, because it causes problems with Element:
https://github.com/vector-im/element-web/issues/19605

This also becomes the first module to use the new Synapse module system
that got introduced in Synapse v1.46.0.

Despite these upgrades, things should remain functionally identical
as far as bridges, matrix-corporal or other consumers are concerned.
2022-02-11 10:23:50 +02:00
Slavi Pantaleev e0df99a7de Fix typo 2022-02-09 14:03:06 +02:00
Marko Weltzer d27e623c71 fix: manually merge upstream 2022-02-09 09:01:56 +01:00
Slavi Pantaleev 5163aa643a Upgrade Synapse (1.51.0 -> 1.52.0)
This also removes the `matrix_synapse_version_arm64` variable we've
been dragging around for a long time.

Since https://github.com/matrix-org/synapse/pull/11810, a multiarch Synapse
container image (for AMD64 and ARM64) is released at the same time.
2022-02-08 15:13:41 +02:00
Marko Weltzer 7e5b88c3b7 fix: all praise the allmighty yamllinter 2022-02-05 21:32:54 +01:00
Slavi Pantaleev ad082b3b1b Fix self-building for Synapse v1.51.0 (requires BuildKit)
Synapse v1.51.0 requires to be built with BuildKit since
https://github.com/matrix-org/synapse/pull/11691

The `docker_image` Ansible module does not support BuildKit
(https://github.com/ansible-collections/community.general/issues/514),
so we had to switch to a `docker build` call.
2022-01-26 08:41:34 +02:00
Aaron Raimist ff94d815e1
Upgrade synapse-simple-antispam (0.0.3 -> 0.0.7) 2022-01-19 14:35:55 +00:00
Slavi Pantaleev 4e4fb98a65 Do not install fuse unless necessary
Discussed here: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1510
2022-01-08 14:14:46 +02:00
Slavi Pantaleev 2bd33e5cf2 Make --tags=register-user work for both Dendrite/Synapse
Also get rid of `--tags=update-user-password` in the
`matrix-dendrite` role, as what we had doesn't work.

We may be able to do it with some Ansible helper or something else.
For now, we'll omit this feature.
2022-01-07 15:59:35 +02:00
Slavi Pantaleev 4625b34acc Fix failure trying to stop orphaned systemd services
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1461
2021-12-17 17:18:35 +02:00
Slavi Pantaleev e1a6d1e4b2 Upgrade Synapse (1.46.0 -> 1.47.0)
We had to remove UID/GID environment variables that we used to pass
to the Synapse container, because it was causing a problem after
https://github.com/matrix-org/synapse/pull/11209

We were using both `--user` and UID/GID environment variables until now.
2021-11-17 17:21:15 +02:00
Slavi Pantaleev 735c966ab6 Disable systemd services when stopping to uninstall them
Until now, we were leaving services "enabled"
(symlinks in /etc/systemd/system/multi-user.target.wants/).

We clean these up now. Broken symlinks may still exist in older
installations that enabled/disabled services. We're not taking care
to fix these up. It's just a cosmetic defect anyway.
2021-11-10 17:39:21 +02:00
Neutron 1ad9bba88f Skip importing validate_config task when Synapse is disabled 2021-07-02 20:58:57 +03:00
Slavi Pantaleev fa76c1ee5b Do not run self-build Ansible version-check, if component not enabled
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1108
2021-06-08 08:59:02 +03:00
pushytoxin bee14550ab Fix local/bin scripts autocompletion by adding rx perms to everyone
It's mildly annoying when trying to execute these scripts while logged
in as a regular user, as the missing execute permissions will hinder
autocompletion even when trying to use with sudo.

These shell scripts don't contain secrets, but may fail when ran by a
regular user. The failure is due to the lack of access to the /matrix
directory, and does not result in any damage.
2021-05-28 10:39:27 +02:00
Slavi Pantaleev d61fe94bae
Fix incorrect path in Mjolnir uninstallation tasks 2021-05-26 10:52:15 +03:00
BG 763952395b Adding mjolnir antispam synapse modul. 2021-05-25 16:43:30 +02:00
Michael-GMH 85777e8f96 merge with upstream 2021-05-25 21:08:00 +08:00
Michael-GMH 4e6f6e179b GMH 0.4.6 update 2021-05-10 18:50:10 +08:00
Slavi Pantaleev e3fa3e12bc Upgrade Synapse (1.31 -> 1.32.2) 2021-04-22 14:22:07 +03:00
Slavi Pantaleev 378fabf177 Revert "Upgrade Synapse (1.31 -> 1.32.1)"
This reverts commit 1fb54a37cb.

Seems like it's been pulled or something. It used to exist, but not
anymore. Not sure what's going on.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1017

Related to
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-21 23:36:58 +03:00
Slavi Pantaleev 1fb54a37cb Upgrade Synapse (1.31 -> 1.32.1)
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-21 18:47:15 +03:00
Slavi Pantaleev ca786cc343 Revert "Upgrade Synapse (1.31 -> 1.32)"
This reverts commit f825c7c263.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1010
2021-04-20 23:40:55 +03:00
Slavi Pantaleev f825c7c263 Upgrade Synapse (1.31 -> 1.32) 2021-04-20 17:47:34 +03:00
Slavi Pantaleev 9dc87bb948 Add Synapse worker presets for easier configuration
Adding more presets in the future would be nice.
2021-02-19 11:38:47 +02:00
Slavi Pantaleev d6c4d41c2b Define instanceId property on workers
This give us the possibility to run multiple instances of
workers that that don't expose a port.

Right now, we don't support that, but in the future we could
run multiple `federation_sender` or `pusher` workers, without
them fighting over naming (previously, they'd all be named
something like `matrix-synapse-worker-pusher-0`, because
they'd all define `port` as `0`).
2021-02-18 18:19:51 +02:00
Slavi Pantaleev 61e427d690 Do not let people enable more than 1 federation_sender worker 2021-02-15 11:37:03 +02:00
Slavi Pantaleev 85a05f38e8 Allow Synapse worker list to be generated dynamically
This leads to much easier management and potential safety
features (validation). In the future, we could try to avoid port
conflicts as well, but it didn't seem worth the effort to do it now.
Our port ranges seem large enough.

This can also pave the way for a "presets" feature
(similar to `matrix_nginx_proxy_ssl_presets`) which makes it even easier
for people to configure worker counts.
2021-02-15 11:25:35 +02:00
Slavi Pantaleev 453a4ec2d8 Relocate tasks related to Synapse workers 2021-02-15 11:18:47 +02:00
Slavi Pantaleev 5cfeae806b Merge branch 'master' into synapse-workers 2021-02-14 13:00:57 +02:00
Slavi Pantaleev 7e8e95a09a Make S3-mounting path configurable
This will make data migration easier.
2021-02-09 22:05:07 +02:00
Béla Becker 2edc9cb83c Name the Synapse database on state compression import
Fixes:
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/833
2021-01-28 17:54:02 +01:00
Slavi Pantaleev a535226210 Stop/disable unnecessary worker services before deleting them 2021-01-25 15:20:37 +02:00
Slavi Pantaleev 70796703d3 Run Synapse workers in their own containers
This switches the `docker exec` method of spawning
Synapse workers inside the `matrix-synapse` container with
dedicated containers for each worker.

We also have dedicated systemd services for each worker,
so this are now:
- more consistent with everything else (we don't use systemd
instantiated services anywhere)
- we don't need the "parse systemd instance name into worker name +
port" part
- we don't need to keep track of PIDs manually
- we don't need jq (less depenendencies)
- workers dying would be restarted by systemd correctly, like any other
service
- `docker ps` shows each worker separately and we can observe resource
usage
2021-01-25 12:14:46 +02:00
Slavi Pantaleev cc5cf0d725 Load roles/matrix-synapse/vars/workers.yml earlier to not break --tags=setup-nginx-proxy
If we load it at runtime, during matrix-synapse role execution,
it's good enough for matrix-synapse and all roles after that,
but.. it breaks when someone uses `--tags=setup-nginx-proxy` alone.

The downside of including this vars file like this in `setup.yml`
is that the variables contained in it cannot be overriden by the user
(in their inventory's `vars.yml`).
... but it's not like overriding these variables was possible anyway
when including them at runtime.
2021-01-24 20:19:55 +02:00
Marcel Partap 183adec3d8 Merge remote-tracking branch 'origin/master' into synapse-workers 2021-01-23 15:04:11 +01:00
Marcel Partap c8f051a42d Track workers endpoint list in repo instead of regenerating on user side 2021-01-23 14:44:36 +01:00
Slavi Pantaleev 95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
Slavi Pantaleev 05ca9357a8 Add .service suffix to systemd units list
We'll be adding `.timer` units later on, so it's good to be
more explicit.
2021-01-14 23:02:10 +02:00