Commit graph

626 commits

Author SHA1 Message Date
Slavi Pantaleev ca69fce648 Add missing group vars for matrix-synapse-auto-compressor 2023-03-12 10:18:33 +02:00
Slavi Pantaleev b28d779c6c Add matrix-synapse-auto-compressor section in group_vars/matrix_servers 2023-03-12 09:48:46 +02:00
Slavi Pantaleev 023fe3ea08 Add sliding-sync support
This allows people to try out the new Element X clients, which need to
run against the sliding-sync proxy (https://github.com/matrix-org/sliding-sync).

Supersedes https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2515

The code is based on the existing PR (#2515), but heavily reworked. Major changes:

- lots of internal refactoring and variable renaming

- fixed self-building to support non-amd64 architectures

- changed to talk to the homeserver locally, over the container network (not
  publicly)

- no more matrix-nginx-proxy support due to complexity (see below)

- no more `matrix_server_fqn_sliding_sync_proxy` in favor of
  `matrix_sliding_sync_hostname` and `matrix_sliding_sync_path_prefix`

- runs on `matrix.DOMAIN/sliding-sync` by default, so it can tried
  easily without having to create new DNS records
2023-03-07 11:57:56 +02:00
Slavi Pantaleev 30f1034767 Remove matrix_playbook_traefik_role_enabled variable and devture-traefik references
The variable was necessary when multiple playbooks could have
potentially tried to manage a shared `devture-traefik.serivce` systemd service
and shared `/devture-traefik` directory.

Since adcc6d9723, we use our own `/matrix/traefik`
(`matrix-traefik.service`) installation and no conflicts can arise.
It's safe to always enable the role, just like we do with all the other roles.
2023-03-06 09:51:14 +02:00
Slavi Pantaleev adcc6d9723 Relocate Traefik (to matrix-traefik.service && /matrix/traefik base path)
The migration is automatic. Existing users should experience a bit of
downtime until the playbook runs to completion, but don't need to do
anything manually.

This change is provoked by https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2535

While my statements there ("Traefik is a shared component among
sibling/related playbooks and should retain its global
non-matrix-prefixed name and path") do make sense, there's another point
of view as well.

With the addition of docker-socket-proxy support in bf2b540807,
we potentially introduced another non-`matrix-`-prefixed systemd service
and global path (`/devture-container-socket-proxy`). It would have
started to become messy.

Traefik always being called `devture-traefik.service` and using the `/devture-traefik` path
has the following downsides:

- different playbooks may write to the same place, unintentionally,
  before you disable the Traefik role in some of them.
  If each playbook manages its own installation, no such conflicts
  arise and you'll learn about the conflict when one of them starts its
  Traefik service and fails because the ports are already in use

- the data is scattered - backing up `/matrix` is no longer enough when
  some stuff lives in `/devture-traefik` or `/devture-container-socket-proxy` as well;
  similarly, deleting `/matrix` is no longer enough to clean up

For this reason, the Traefik instance managed by this playbook
will now be called `matrix-traefik` and live under `/matrix/traefik`.

This also makes it obvious to users running multiple playbooks, which
Traefik instance (powered by which playbook) is the active one.
Previously, you'd look at `devture-traefik.service` and wonder which
role was managing it.
2023-03-06 09:34:31 +02:00
Slavi Pantaleev bf2b540807 Harden Traefik security by accessing the Docker API through docker-socket-proxy
With these changes, we:

- install https://github.com/Tecnativa/docker-socket-proxy via the
  https://github.com/devture/com.devture.ansible.role.container_socket_proxy Ansible role

- make Traefik access the Docker API via TCP by connecting to this
  socket proxy

- .. which allows us to run the Traefik container with less privileges
  (non-`root`, dropped capabilities)
2023-03-06 09:11:02 +02:00
Slavi Pantaleev 10b5350370 Add Traefik support to Go-NEB bot
Completely untested.
2023-03-03 10:40:45 +02:00
Slavi Pantaleev f8966cd8da Default etherpad_hostname to matrix_server_fqn_etherpad for backward compatibility 2023-03-03 09:47:13 +02:00
Slavi Pantaleev 124fbeda04 Switch to using an external Etherpad role
This new role also adds native Traefik support and support for other
(non-`amd64`) architectures via self-building.
2023-03-02 22:50:13 +02:00
Slavi Pantaleev b0845984b3 Only enable Traefik certs dumper if the ACME certificate resolver for Traefik is enabled
If someone disables ACME, then they're using their own certificates
somehow. There's nothing to dump from an `acme.json` file.
2023-03-01 09:45:16 +02:00
Slavi Pantaleev f7149103e4 Remove matrix_playbook_traefik_certs_dumper_role_enabled in favor of just devture_traefik_certs_dumper_enabled
We don't need these 2 roughly-the-same settings related to the
traefik-certs-dumper role.

For Traefik, it makes sense, because it's a component used by the
various related playbooks and they could step onto each other's toes
if the role is enabled, but Traefik is disabled (in that case, uninstall
tasks will run).

As for Traefik certs dumper, the other related playbooks don't have it,
so there's no conflict. Even if they used it, each one would use its own
instance (different `devture_traefik_certs_dumper_identifier`), so there
wouldn't be a conflict and uninstall tasks can run without any danger.
2023-03-01 09:31:48 +02:00
Slavi Pantaleev b388a01ab7 Wire all certResolver variables to devture_traefik_certResolver_primary
This allows people wishing to change or unset the resolver,
to have a single variable which they can toggle.

Unsetting the resolver is useful for using your own certificates
(not coming from a certificate resolver).
2023-02-27 17:09:19 +02:00
Slavi Pantaleev 9e7415afa2 Ensure Buscarron is part of the Postgres network 2023-02-27 17:07:44 +02:00
Slavi Pantaleev 058a54fd05 Add native Traefik support to Dimension 2023-02-26 23:06:36 +02:00
Slavi Pantaleev b84f25309b Add matrix_homeserver_container_network 2023-02-26 22:09:37 +02:00
Slavi Pantaleev d20ff688db Add native Traefik support to Sygnal 2023-02-26 11:03:42 +02:00
Slavi Pantaleev 348dd8e76b Remove double space 2023-02-25 19:37:35 +02:00
Slavi Pantaleev 725b2beed7 Add native Traefik support to Buscarron 2023-02-25 15:50:48 +02:00
Slavi Pantaleev bc5dda2b3a Reorder some Buscarron default variables and fix some typos
Fixes a regression introduced in 0220c851e8
2023-02-25 15:11:23 +02:00
Slavi Pantaleev 0220c851e8 Add multiple container networks support to Buscarron 2023-02-25 15:03:03 +02:00
Slavi Pantaleev 233e253264 Add native Traefik support to rageshake 2023-02-25 13:46:42 +02:00
Slavi Pantaleev 306679103b Require self-building of rageshake for arm64
There are no arm64 images published.. yet
2023-02-25 12:32:19 +02:00
Slavi Pantaleev 55f43dcc6d Fixup matrix-rageshake section in group vars 2023-02-25 12:09:23 +02:00
Benjamin Kampmann 40f037b36d Add rageshake server 2023-02-24 16:55:49 +01:00
Slavi Pantaleev 5e7f30a129 Fix appservice-discord/appservice-slack/appservice-webhooks port troubles with external reverse-proxy
Continuation of 6cda711
2023-02-19 11:20:58 +02:00
Slavi Pantaleev 632026513e Add matrix_synapse_uid, matrix_synapse_gid and matrix_synapse_username 2023-02-17 17:16:50 +02:00
Slavi Pantaleev 990a6369e1 Switch to using an external Redis role 2023-02-17 16:23:59 +02:00
Slavi Pantaleev 964aa0e84d Switch to using an external Ntfy role
The newly extracted role also has native Traefik support,
so we no longer need to rely on `matrix-nginx-proxy` for
reverse-proxying to Ntfy.

The new role uses port `80` inside the container (not `8080`, like
before), because that's the default assumption of the officially
published container image. Using a custom port (like `8080`), means the
default healthcheck command (which hardcodes port `80`) doesn't work.
Instead of fiddling to override the healthcheck command, we've decided
to stick to the default port instead. This only affects the
inside-the-container port, not any external ports.

The new role also supports adding the network ranges of the container's
multiple additional networks as "exempt hosts". Previously, only one
network's address range was added to "exempt hosts".
2023-02-17 09:54:33 +02:00
Slavi Pantaleev e80b98c3ad Do not mount SSL certificates into Coturn if TLS is disabled for it 2023-02-16 09:22:29 +02:00
Slavi Pantaleev bb7895678c Fix typo 2023-02-15 11:48:27 +02:00
Slavi Pantaleev 7c5826f1c3 Break dependency between matrix-prometheus-nginxlog-exporter and the Grafana role
Wiring happens via `group_vars/matrix_servers` now.
2023-02-15 10:52:25 +02:00
Slavi Pantaleev 1006b8d899 Replace matrix-grafana with an external role 2023-02-15 10:32:24 +02:00
Slavi Pantaleev 94124263a7 Add matrix_prometheus_container_network/matrix_prometheus_container_additional_networks 2023-02-15 08:56:11 +02:00
Slavi Pantaleev c85d48c45c Remove Traefik labels for Hydrogen & Cinny from matrix-nginx-proxy
Related to 6a52be7987 and 28e7ef9c71f02
2023-02-14 22:46:34 +02:00
Slavi Pantaleev f28e7ef9c7 Add (native) Traefik support to matrix-client-cinny
Previously, it had to go through matrix-nginx-proxy.
It's exposed to Traefik directly via container labels now
2023-02-14 11:29:53 +02:00
Slavi Pantaleev 6a52be7987 Add (native) Traefik support to matrix-client-hydrogen
Previously, it had to go through matrix-nginx-proxy.
It's exposed to Traefik directly via container labels now

Serving at a path other than `/` doesn't work well yet.
2023-02-14 09:58:35 +02:00
Slavi Pantaleev e51e4eec09 Add (native) Traefik support to matrix-client-element
Previously, it had to go through matrix-nginx-proxy.
It's exposed to Traefik directly via container labels now
2023-02-13 19:03:20 +02:00
Slavi Pantaleev b2d8718233 Fix synapse-admin reverse-proxying regression for "playbook-managed-nginx"
Regression since 3d9aa8387e
2023-02-13 16:08:59 +02:00
Slavi Pantaleev 6cda711c0b Fix incorrect host_bind_port syntax (extra :) affecting certain deployments
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2474

Seems like this affected all "own webserver" deployments, which required
port exposure.

`playbook-managed-traefik` and `playbook-managed-nginx` were not affected.
2023-02-13 15:38:24 +02:00
Slavi Pantaleev 3d9aa8387e Add (native) Traefik support to synapse-admin
Previously, it had to go through matrix-nginx-proxy.
It's exposed to Traefik directly via container labels now.
2023-02-13 15:08:42 +02:00
Aine 33b4f7031b
restore borg prefixes 2023-02-13 10:44:42 +00:00
Slavi Pantaleev 266195ab45 Upgrade backup_borg (v1.2.3-1.7.5-1 -> v1.2.3-1.7.6-0)
Supersedes https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2472
2023-02-13 12:26:49 +02:00
Slavi Pantaleev 23f7720247 Add missing backup_borg_base_path override 2023-02-13 11:44:19 +02:00
Slavi Pantaleev 38904c08b0 Wire backup_borg_username
It's probably unnecessary, as this user is only used in the borg container
internally, but.. It doesn't hurt to set it to `matrix`.
2023-02-13 11:01:54 +02:00
Slavi Pantaleev 78c35136b2 Replace matrix-backup-borg with an external role 2023-02-13 10:53:11 +02:00
Slavi Pantaleev 972043cfaf Fix trying to start devture-traefik when not necessarily enabled
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2465
2023-02-12 17:14:25 +02:00
Slavi Pantaleev f1a1ce8a91
Merge pull request #2464 from spantaleev/traefik
Reverse-proxy configuration changes and initial Traefik support
2023-02-12 16:05:56 +02:00
Slavi Pantaleev b3f6436a0d Do not enable the Traefik role when reverse-proxy = other-traefik-container 2023-02-12 15:50:18 +02:00
Catalan Lover be471250dd
Move services that crash without hs connectivity to after proxy. 2023-02-11 17:58:19 +01:00
Slavi Pantaleev 94be74e633 Improve traefik-certs-dumper defaults for other-traefik-container setups
We'd like to auto-enable traefik-certs-dumper for these setups.

`devture_traefik_certs_dumper_ssl_dir_path` will be empty though,
so the role's validation will point people in the right direction.
2023-02-11 08:54:07 +02:00
Slavi Pantaleev f37a7a21f1 Delay Postmoogle startup to help Traefik-based setups 2023-02-11 08:53:32 +02:00
Slavi Pantaleev 8309a21303 Rename reverse proxy types and fix Hookshot http/https urlPrefix issue 2023-02-11 08:44:11 +02:00
Slavi Pantaleev 97f65e8dff Minor fixes to allow for Traefik without SSL 2023-02-10 19:36:06 +02:00
Slavi Pantaleev 28d2eb593c Add matrix_playbook_reverse_proxy_type variable which influences all other services 2023-02-10 16:04:34 +02:00
Slavi Pantaleev 06ccd71edc Merge branch 'master' into traefik 2023-02-10 14:37:59 +02:00
Slavi Pantaleev 01ccec2dbe Merge branch 'master' into pr-jitsi-matrix-authentication 2023-02-10 14:12:47 +02:00
Slavi Pantaleev 7cdf59d79b
Merge pull request #2451 from FSG-Cat/draupnir
Add Draupnir support to the project.
2023-02-10 11:43:30 +02:00
Slavi Pantaleev a5683a6449 Upgrade com.devture.ansible.role.traefik and rename some variables 2023-02-09 10:12:09 +02:00
Catalan Lover 7b42ff4b75
Finalise moving draupnir to a fully testable state. 2023-02-08 18:55:08 +01:00
Slavi Pantaleev 88a26758e1 Merge branch 'master' into traefik 2023-02-08 18:48:10 +02:00
Slavi Pantaleev c71567477a Stop using deprecated matrix_bot_postmoogle_domain variable in group vars 2023-02-08 18:48:01 +02:00
Slavi Pantaleev 1338963b6c Add support for obtaining additional SSL certificates via Traefik 2023-02-08 18:47:19 +02:00
Slavi Pantaleev 9a71a5696b Allow Postmoogle to work with SSL certificates extracted from Traefik 2023-02-08 16:45:03 +02:00
Slavi Pantaleev ddf6b2d4ee Handle matrix_playbook_reverse_proxy_type being "none" when deciding on Coturn certificate parameters 2023-02-08 16:24:43 +02:00
Slavi Pantaleev d44d4b637f Allow Coturn to work with SSL certificates extracted from Traefik 2023-02-08 16:06:46 +02:00
Slavi Pantaleev c07630ed51 Add com.devture.ansible.role.traefik_certs_dumper role
With this, other roles (like Coturn, Postmoogle) will be able
to use SSL certificates extracted from Traefik
via https://github.com/ldez/traefik-certs-dumper
2023-02-08 16:05:38 +02:00
Paul N 96dd86d33b Set default values where sensible and remove unnecessary conditionals in .env.j2.
Check for empty string instead of Null to verify if an openid_server_name is pinned.
2023-02-06 15:26:08 +01:00
Paul N d67d8c07f5 Remove remnant comment. 2023-02-06 15:26:08 +01:00
jakicoll 6499b6536a Decoupling: Do not use variables user-verification-service role inside the jitsi role. 2023-02-06 15:18:25 +01:00
Paul N 1d99f17b4a Disable matrix-user-verification-service in group_vars and update docs accordingly. 2023-02-06 13:23:11 +01:00
Paul N 50c1e9d695 Set matrix_user_verification_service_uvs_homeserver_url in the role defaults and updated docs accordingly. 2023-02-06 13:14:34 +01:00
Paul N 07d9ea5e87 Stick to port 3003 instead of changing the port based on the status of grafana. 2023-02-06 13:06:35 +01:00
jakicoll 0e0ae2f3e6 Assign default log level in role instead of matrix_servers file. 2023-02-06 13:04:06 +01:00
jakicoll f53731756d Change comment
Applying the assumption, that synapse is always managed by this playbook.
2023-02-06 12:15:54 +01:00
Slavi Pantaleev 8155f780e5 Add support for reverse-proxying Matric (Client & Federation) via Traefik 2023-02-06 13:08:11 +02:00
jakicoll 94830b582b Wording: change collection -> playbook 2023-02-06 11:58:50 +01:00
Slavi Pantaleev f983604695 Initial work on Traefik support
This gets us started on adding a Traefik role and hooking Traefik:

- directly to services which support Traefik - we only have a few of
  these right now, but the list will grow

- to matrix-nginx-proxy for most services that integrate with
  matrix-nginx-proxy right now

Traefik usage should be disabled by default for now and nothing should
change for people just yet.

Enabling these experiments requires additional configuration like this:

```yaml
devture_traefik_ssl_email_address: '.....'

matrix_playbook_traefik_role_enabled: true
matrix_playbook_traefik_labels_enabled: true

matrix_ssl_retrieval_method: none

matrix_nginx_proxy_https_enabled: false

matrix_nginx_proxy_container_http_host_bind_port: ''
matrix_nginx_proxy_container_federation_host_bind_port: ''

matrix_nginx_proxy_trust_forwarded_proto: true

matrix_nginx_proxy_x_forwarded_for: '$proxy_add_x_forwarded_for'

matrix_coturn_enabled: false
```

What currently works is:
reverse-proxying for all nginx-proxy based services **except** for the Matrix homeserver
(both Client-Server an Federation traffic for the homeserver don't work yet)
2023-02-06 10:34:51 +02:00
Slavi Pantaleev be78b74fbd Switch from matrix-prometheus-postgres-exporter to an external prometheus_postgres_exporter role 2023-02-05 10:32:09 +02:00
Slavi Pantaleev d7c0239e40 Enable metrics endpoint for mautrix bridges by default when Prometheus is enabled
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2427

This just enables the endpoint, which is somewhat helpful, but not
really enough to scrape them. Ideally, we'd be injecting these targets
into the Prometheus scrape config too.
For now, registering targets with Prometheus is very manual
(`matrix_prometheus_scraper_postgres_enabled`, `matrix_prometheus_scraper_hookshot_enabled`, ..).
This should be redone - e.g. a new `matrix_prometheus_scrape_config_jobs_auto` variable,
which is dynamically built in `group_vars/matrix_servers`.
2023-01-30 08:53:28 +02:00
Slavi Pantaleev 9ed2e04d80 Switch from matrix-prometheus-node-exporter to an external prometheus_node_exporter role 2023-01-21 11:07:04 +02:00
Slavi Pantaleev 4e40ac5ad8
Merge pull request #2227 from xangelix/add-matrix-mautrix-slack-role
Add matrix-bridge-mautrix-slack role
2023-01-11 10:35:45 +02:00
Slavi Pantaleev ddfab60427 Enable self-building for chatgpt for arm32 2023-01-10 17:20:50 +02:00
Slavi Pantaleev 8d3ce50d1b Disable chatgpt from group_vars/matrix_servers by default 2023-01-10 17:20:33 +02:00
bertybuttface 0ec1868b95 Add matrix-bot-chatgpt.
Co-Authored-By: Slavi Pantaleev <slavi@devture.com>
2023-01-10 13:57:38 +00:00
Thomas Baer c86720eeae
convert to list from generator
selectattr() returns a generator object, an iterator. This leads to an exception later, lists can't concated to iterators, only to other lists. So '| list' converts the iterator to a list and the script runs happily.
2023-01-05 14:10:06 +01:00
jakicoll 42e4e50f5b Matrix Authentication Support for Jitsi
This extends the collection with support for seamless authentication at the Jitsi server using Matrix OpenID.

1. New role for installing the [Matrix User Verification Service](https://github.com/matrix-org/matrix-user-verification-service)
2. Changes to Jitsi role: Installing Jitsi Prosody Mods and configuring Jitsi Auth
3. Changes to Jitsi and nginx-proxy roles: Serving .well-known/element/jitsi from jitsi.DOMAIN
4. We updated the Jitsi documentation on authentication and added documentation for the user verification service.
2023-01-04 14:27:16 +01:00
Cody Wyatt Neiman 2e0dfb2dc1
Update slack bridge implementation 2023-01-02 20:07:04 -05:00
Cody Wyatt Neiman 784e5492d5
Add matrix-bridge-mautrix-slack role 2023-01-02 19:13:17 -05:00
Samuel Meenzen 0179b0f165
Remove conduit workaround
Conduit update 0.5.0 fixed the issue, so this is no longer needed.
2022-12-21 18:28:34 +01:00
Matthew Cengia 3453fff901
Use upstream Docker image for amd64 rather than self-build 2022-12-11 21:25:43 +11:00
Slavi Pantaleev da82c3bd4f
Merge pull request #2327 from ikkemaniac/fix-nginxlog-prometheus
fix: nginxlog prometheus config port
2022-12-08 13:15:34 +02:00
ikkemaniac e6fc6b7a86 fix: nginxlog prometheus config port 2022-12-08 01:10:05 +01:00
ikkemaniac 8ef6341fd7 fix: systemd entry 2022-12-08 00:02:54 +01:00
ikkemaniac 8ebf18a885
add prometheus-nginxlog-exporter role (#2315)
* add prometheus-nginxlog-exporter role

* Rename matrix_prometheus_nginxlog_exporter_container_url to matrix_prometheus_nginxlog_exporter_container_hostname

* avoid referencing variables from other roles, handover info using group_vars/matrix_servers

* fix: stop service when uninstalling

fix: typo

move available arch's into a var

fix: text

* fix: prometheus enabled condition

Co-authored-by: ikkemaniac <ikkemaniac@localhost>
2022-12-07 16:58:36 +02:00
Slavi Pantaleev 0a018ac22b Add internal Postgres instance (if enabled) to postgres-backup dependencies 2022-11-30 11:22:00 +02:00
Slavi Pantaleev d5ea17d66f Make postgres-backup priority start later 2022-11-30 11:18:39 +02:00
Slavi Pantaleev 4eed49f931 Replace custom/matrix-postgres-backup role with galaxy/com.devture.ansible.role.postgres_backup
This role is usable on its own and it's not tied to Matrix, so
extracting it out into an independent role that we install via
ansible-galaxy makes sense.

This also fixes the confusion from the other day, where
`matrix_postgres_*` had to be renamed to `devture_postgres_*`
(unless it was about `matrix_postgres_backup_*`).
We now can safely say that ALL `matrix_postgres_*` variables need to be
renamed.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2305
2022-11-30 11:01:19 +02:00
Slavi Pantaleev de979bc6a2 Upgrade com.devture.ansible.role.postgres 2022-11-30 09:42:06 +02:00
Slavi Pantaleev 4b2d30a474 Fix matrix_dendrite_client_api_turn_shared_secret not being defined
Regression since https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2290
2022-11-28 18:33:18 +02:00
Slavi Pantaleev 81054bb19c Upgrade com.devture.ansible.role.postgres 2022-11-28 09:05:22 +02:00