Commit graph

262 commits

Author SHA1 Message Date
teutat3s 2531e82d5b
Merge branch 'master' into pub.solar 2022-10-19 01:30:25 +02:00
Slavi Pantaleev 30745db6d5 Add storage-class support to the s3_media_upload command and mention matrix-synapse-s3-storage-provider-migrate script 2022-10-14 18:29:30 +03:00
Slavi Pantaleev de8fd519c5 Document matrix-synapse-s3-storage-provider-migrate.{service,timer} 2022-10-14 18:14:34 +03:00
Slavi Pantaleev fe96475595 Remove some useless PG_ variables fron the s3-storage-provider env-file
These Postgres configuration variables are part of the `database.yaml` file
and don't need to be here.
2022-10-14 18:01:51 +03:00
Slavi Pantaleev ac7cb3619d Add support for synapse-s3-storage-provider 2022-10-14 17:41:41 +03:00
Slavi Pantaleev 881fdd28f0 Add support for Synapse container image customization
This allows people to augment the Synapse image with custom tools and
addons without having to rebuild it from scratch.

If customizations are enabled, the playbook will build a new
`localhost/matrixdotorg/synapse:VERSION-customized` image
on top of the default one (`FROM matrixdotorg/synapse:VERSION`)
and with custom Dockerfile build steps.

For servers that self-build the Synapse image, the Synapse image will be
built first, before proceding to extend it the same way.

In the future, we'll also have easy to enable Dockerfile build steps
for modules that the playbook supports.
2022-10-14 16:37:54 +03:00
Slavi Pantaleev 0da068c7e6 Add support for configuring Synapse's media_storage_providers
In the future, we'll also inject media storage providers that we support
into the `matrix_synapse_media_storage_providers_auto` list.
2022-10-14 16:28:47 +03:00
teutat3s 71d239a28e
Merge branch 'master' into pub.solar 2022-09-18 13:54:11 +02:00
Slavi Pantaleev 109e1addb8 Improve reliability when using more than 1 media_repository worker 2022-09-15 10:45:03 +03:00
Slavi Pantaleev 8887aed500 Do not add federation_sender_instances/pusher_instances configuration if not necessary
This keeps the configuration cleaner and hopefully prevents odd issues
if Synapse (by any chance) happens to interpret `pusher_instances: []`
as "no pushers", instead of as "push from the master process".

I haven't confirmed what an empty pushers/federation-senders list means,
so it's safer to just avoid it.
2022-09-15 10:34:02 +03:00
Slavi Pantaleev a1fb082618 Add support for running background tasks on a worker 2022-09-15 10:32:51 +03:00
Slavi Pantaleev 5f3f460cda Restore support for appservice and user_dir workers 2022-09-15 10:06:56 +03:00
Slavi Pantaleev 8cd7c1ec2f Relocate enable_media_repo and make it configurable via matrix_synapse_enable_media_repo
.. although.. manual configuration is discouraged in most cases.
2022-09-15 08:37:23 +03:00
Slavi Pantaleev b842447047 Add support for multiple pusher workers 2022-09-15 08:32:55 +03:00
Slavi Pantaleev 22cf259155 Put common worker configuration options at the top 2022-09-15 08:14:47 +03:00
Slavi Pantaleev ec654ca91e Add support for multiple federation sender workers 2022-09-15 08:13:08 +03:00
Slavi Pantaleev 226c550ffa Add support for stream writer Synapse workers
As stream writer workers are also powered by the `generic_worker`
Synapse app, this necessitated that we provide means for distinguishing
between them and regular `generic_workers`.

I've also taken the time to optimize nginx configuration generation
(more Jinja2 macro usage, less duplication).

Worker names have also changed.
Workers are now named sequentially like this:
- `matrix-synapse-worker-0-generic`
- `matrix-synapse-worker-1-stream-writer-typing`
- `matrix-synapse-worker-2-pusher`

instead of `matrix-synapse-worker_generic_worker-18111` (indexed with a
port number).

People who modify `matrix_synapse_workers_enabled_list` directly will
need to adjust their configuration.
2022-09-15 08:10:04 +03:00
Slavi Pantaleev ef523d68a4 Suppress errors in docker kill/rm calls for Synapse workers
We do the same everywhere else. This is an omission.
2022-09-14 08:13:01 +03:00
Slavi Pantaleev f12206676f Upgrade Synapse (v1.66.0 -> 1.67.0) and remove frontend_proxy workers
`frontend_proxy` workers have been superseded by `generic_worker` workers.
Related to https://github.com/matrix-org/synapse/pull/13645
2022-09-13 15:45:50 +03:00
teutat3s 53dea38606
Merge branch 'master' into pub.solar 2022-09-04 19:29:32 +02:00
Slavi Pantaleev 8e0e9fa878 Deprecate matrix_synapse_account_threepid_delegates_email before Synapse v1.66.0
This is done in anticipation of this option's removal in the
upcoming Synapse v1.66.0 release (likely tomorrow).

See: https://matrix-org.github.io/synapse/v1.66/upgrade.html#delegation-of-email-validation-no-longer-supported
2022-08-30 18:51:35 +03:00
teutat3s 5102ed3098
Merge branch 'master' into pub.solar 2022-08-27 13:53:39 +02:00
Shaleen Jain f674afe5e8
appservice: add and use homeserver_container_* vars (#2045)
* appservice: add and use matrix_homeserver_* vars

* appservice: use the new vars

* Apply suggestions from code review

Co-authored-by: Slavi Pantaleev <slavi@devture.com>

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2022-08-24 08:38:12 +03:00
Benjamin Castellan 22db7914e5 added rc_invites configuration for synapse 2022-08-17 13:34:33 +02:00
teutat3s 8b057ab29b
Merge branch 'master' into pub.solar 2022-07-21 16:44:48 +02:00
Slavi Pantaleev 2a131fd507
Merge pull request #1954 from aaronraimist/stats
Update telemetry documentation to more closely match upstream
2022-07-19 18:38:29 +03:00
Aaron Raimist 817830bb3d
Update telemetry documentation to more closely match upstream
Synapse no longer describes the stats as anonymized since the `server_name` is included.
https://github.com/matrix-org/synapse/pull/13321
2022-07-19 14:58:19 +00:00
teutat3s f6a1d4391e
Merge branch 'master' into pub.solar 2022-07-19 09:56:48 +02:00
Slavi Pantaleev d073c7ecb3 More ansible-lint fixes 2022-07-18 13:01:19 +03:00
teutat3s 2925553b8f
Merge branch 'master' into pub.solar 2022-06-28 16:32:45 +02:00
Slavi Pantaleev ba51997f7b (BC Break) Redo how metrics are exposed to external Prometheus servers 2022-06-23 17:55:07 +03:00
David Mehren f6a73231ab
Synapse workers should respect X-Forwarded headers
Currently, Synapse workers ignore the X-Forwarded headers, which leads to internal Docker IP addresses randomly appearing in the users' device list.

This adds the `x_forwarded: true` option to the worker config, fixing the issue.
2022-06-18 16:13:08 +02:00
Slavi Pantaleev 38027e72f6 Fix "object of type 'NoneType' has no len()" error
Fixup for 5eff67371d - https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1884
2022-06-17 15:45:29 +03:00
Slavi Pantaleev 7440dd34fb
Merge pull request #1884 from etkecc/master
add synapse media_retention
2022-06-17 15:31:55 +03:00
Slavi Pantaleev 5987589436
Use |to_json 2022-06-17 15:30:22 +03:00
Slavi Pantaleev 323f5aa60d Synchronize homeserver.yaml config with the one from Synapse 1.61.0 2022-06-17 15:26:23 +03:00
Aine 5eff67371d
add synapse media_retention 2022-06-17 14:32:17 +03:00
Slavi Pantaleev a2c3ee417b
Stop using deprecated (in Synapse v1.59) user_dir and appservice workers
Source: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types

As an alternative, we should probably find a way to run one or a few
more generic workers (which will handle appservice and user_dir stuff) and
update `homeserver.yaml` so that it would point to the name of these workers using
`notify_appservices_from_worker` and `update_user_directory_from_worker` options.

For now, this solves the deprecation, so we can have a peace of mind
going forward.

We're force-setting these worker counts to 0, so that we can clean up
existing homeservers which use these worker types. In the future, these
options will either be removed or repurposed (so that they transparently
create more generic workers that handle user_dir/appservice loads).
2022-06-11 17:58:01 +02:00
Vicx bf240c5045
Fix synapse LDAP simple bind config
When using LDAP simple bind, the `bind_dn` and `bind_password`
configuration values must not be present.
2022-06-11 17:58:00 +02:00
Slavi Pantaleev 29f2bfee8f
Upgrade Synapse (1.58.1 -> 1.59.0) 2022-06-11 17:57:57 +02:00
Slavi Pantaleev 78204619ea Stop using deprecated (in Synapse v1.59) user_dir and appservice workers
Source: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types

As an alternative, we should probably find a way to run one or a few
more generic workers (which will handle appservice and user_dir stuff) and
update `homeserver.yaml` so that it would point to the name of these workers using
`notify_appservices_from_worker` and `update_user_directory_from_worker` options.

For now, this solves the deprecation, so we can have a peace of mind
going forward.

We're force-setting these worker counts to 0, so that we can clean up
existing homeservers which use these worker types. In the future, these
options will either be removed or repurposed (so that they transparently
create more generic workers that handle user_dir/appservice loads).
2022-05-31 16:35:50 +03:00
Slavi Pantaleev 63c755b115
Merge pull request #1837 from Lyokovic/vicx/fix-ldap-simple-bind
Fix synapse LDAP simple bind config
2022-05-24 10:40:28 +03:00
Vicx a906fad12e Fix synapse LDAP simple bind config
When using LDAP simple bind, the `bind_dn` and `bind_password`
configuration values must not be present.
2022-05-20 19:09:16 +02:00
Slavi Pantaleev f972a80224 Upgrade Synapse (1.58.1 -> 1.59.0) 2022-05-17 13:08:17 +03:00
teutat3s e0b27ae3cc
matrix-synapse: allow unsafe locale in DB 2022-04-18 18:29:22 +02:00
Yan Minagawa f6cb59116b This adds a variable for requiring MSC3231 token for registration 2022-04-12 14:31:49 +07:00
Slavi Pantaleev 0364c6c634 Suppress old container cleanup (kill/rm) failures
People often report and ask about these "failures".
More-so previously, when the `docker kill/rm` output was collected,
but it still happens now when people do `systemctl status
matrix-something` and notice that it says "FAILURE".

Suppressing to avoid further time being wasted on saying "this is
expected".
2022-04-11 09:05:33 +03:00
Aine 7559eb99a7
Update Synapse 1.55.2 -> 1.56.0 2022-04-05 20:48:15 +03:00
Slavi Pantaleev d04162b275 Upgrade Synapse (1.54.0 -> 1.55.0)
`roles/matrix-synapse/vars/workers.yml` has not been updated here,
because running `roles/matrix-synapse/files/workers-doc-to-yaml.awk`
seems to cause some trouble with the current `workers.md` file
available at https://github.com/matrix-org/synapse/raw/master/docs/workers.md,
namely lots of:

> FIXME: ADDITIONAL CONDITIONS REQUIRED: to be enabled manually

lines and commented out regex lines.

This is something that remains to be investigated/fixed.
2022-03-22 19:41:30 +02:00
Jim Myhrberg eeca3c8dca
fix: avoid yaml being wrapped at column 80 via to_nice_yaml
The `to_nice_yaml` helper will by default wrap any string YAML values on
the first space after column 80. This can in worst case yield invalid
YAML syntax. More details in Ansible's documentation here:

https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json

In short, you need to explicitly provide a custom width argument of a
high number of some kind to avoid the line wrapping.
2022-03-16 01:10:26 +00:00