Commit graph

107 commits

Author SHA1 Message Date
Slavi Pantaleev 8710883064
Merge pull request #743 from pushytoxin/docker_network
Drop the old workaround for an Ansible bug that has been fixed three years ago
2021-01-03 08:49:09 +02:00
Slavi Pantaleev d08b27784f Fix systemd services autostart problem with Docker 20.10
The Docker 19.04 -> 20.10 upgrade contains the following change
in `/usr/lib/systemd/system/docker.service`:

```
-BindsTo=containerd.service
-After=network-online.target firewalld.service containerd.service
+After=network-online.target firewalld.service containerd.service multi-user.target
-Requires=docker.socket
+Requires=docker.socket containerd.service
Wants=network-online.target
```

The `multi-user.target` requirement in `After` seems to be in conflict
with our `WantedBy=multi-user.target` and `After=docker.service` /
`Requires=docker.service` definitions, causing the following error on
startup for all of our systemd services:

> Job matrix-synapse.service/start deleted to break ordering cycle starting with multi-user.target/start

A workaround which appears to work is to add `DefaultDependencies=no`
to all of our services.
2020-12-10 11:43:20 +02:00
Béla Becker 6f9b4bd9ac Drop workaround for old Ansible docker_network bug 2020-12-05 19:02:10 +01:00
Slavi Pantaleev be5263f397 Move self-building git repository URLs to variables (stop hardcoding) 2020-11-28 21:34:14 +02:00
Slavi Pantaleev 75f9fde7a4 Remove some more -v usage
Continuation of 1fca917ad1.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/722
2020-11-25 10:49:59 +02:00
Slavi Pantaleev 5eed874199 Improve self-building experience (avoid conflict with pullable images)
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/716

This patch makes us use more fully-qualified container image names
(either prefixed with docker.io/ or with localhost/).

The latter happens when self-building is enabled.

We've recently had issues where if an image was removed manually
and the service was restarted (making `docker run` fetch it from Docker Hub, etc.),
we'd end up with a pulled image, even though we're aiming for a self-built one.
Re-running the playbook would then not do a rebuild, because:
- the image with that name already exists (even though it's something
else)
- we sometimes had conditional logic where we'd build only if the git
repo changed

By explicitly changing the name of the images (prefixing with localhost/),
we avoid such confusion and the possibility that we'd automatically pul something
which is not what we expect.

Also, I've removed that condition where building would happen on git
changes only. We now always build (unless an image with that name
already exists). We just force-build when the git repo changes.
2020-11-14 23:00:49 +02:00
Slavi Pantaleev 2a1ec38e3a Stop using Ansible's cron module
This is mainly to address SSL renewal not working for us due to:
- https://github.com/ansible/ansible/issues/71213
- https://github.com/ansible/ansible/pull/71207

Using the cron module was hacky anyway. We shouldn't need an extra
level of buggy abstraction to manage a cronjob file.
2020-09-06 10:49:19 +03:00
Slavi Pantaleev de0efe96e7 Fix incorrect when statement 2020-07-17 08:59:00 +03:00
Slavi Pantaleev 8b59402f79 Upgrade Coturn (4.5.1.2 -> 4.5.1.3)
4.5.1.3 fixes a security vulnerability:
https://github.com/coturn/coturn/security/advisories/GHSA-c8r8-8vp5-6gcm
2020-06-30 14:28:41 +03:00
Chris van Dijk 74df10633a Remove hardcoded command paths in playbook cron usage 2020-05-27 23:14:58 +02:00
Chris van Dijk 6334f6c1ea Remove hardcoded command paths in systemd unit files
Depending on the distro, common commands like sleep and chown may either
be located in /bin or /usr/bin.

Systemd added path lookup to ExecStart in v239, allowing only the
command name to be put in unit files and not the full path as
historically required. At least Ubuntu 18.04 LTS is however still on
v237 so we should maintain portability for a while longer.
2020-05-27 23:14:54 +02:00
Slavi Pantaleev 36c61b5b4e Introduce a separate group variable (matrix_user_groupname)
Related to #485 (Github Pull Request).
2020-05-06 10:02:47 +03:00
Dan Arnfield e750c6c6ff Update coturn (4.5.1.1 -> 4.5.1.2) 2020-05-04 09:46:45 -05:00
Chris van Dijk 7585bcc4ac Allow the matrix user username and groupname to be configured separately
No migration steps should be required.
2020-05-01 19:59:32 +02:00
mooomooo eebc6e13f8 Made directory variables for /etc/systemd/system , /etc/cron.d , /usr/local/bin 2020-03-24 11:27:58 -07:00
Slavi Pantaleev 027056e027 Fix weird path creation
Fixes #403 (Github Issue).
2020-03-18 18:24:30 +02:00
Slavi Pantaleev 3cee815baf Fix some typos 2020-03-15 11:34:35 +02:00
Slavi Pantaleev 2b85fde103 Rename some variables for consistency 2020-03-15 10:15:27 +02:00
Slavi Pantaleev 8fe97abe7d Wire matrix_container_images_self_build to self_build variables via group_vars/matrix_servers
This keeps the roles cleaner and more independent of matrix-base,
which may be important for people building their own playbook
out of the individual roles and not using the matrix-base role.
2020-03-15 10:10:41 +02:00
Horvath Gergely 2d537484d5 introduce variable 2020-03-14 19:16:29 +01:00
Horvath Gergely 3c8535c3bc check ansible version for self-building in every role 2020-03-08 19:17:10 +01:00
Horvath Gergely a5d94eec0b refactor variable names 2020-03-08 00:28:14 +01:00
Horvath Gergely 310aa685f9 refactor based on Slavi's requests 2020-03-08 00:24:00 +01:00
Horvath Gergely 697f86d06f minor fix 2020-02-19 22:26:43 +01:00
Horvath Gergely 7c4a86bc6b add coturn support for raspberry pi 2020-02-19 22:18:17 +01:00
Slavi Pantaleev 9a33e5c7ad Make it possible to control Coturn ports and listen interfaces
Related to #330 (Github Issue).
2019-12-20 12:21:43 +02:00
gusttt 25262fa0e1 Disable docker network tasks in check mode to allow running the playbook in check mode (--check --diff) 2019-09-17 22:24:38 +02:00
Slavi Pantaleev 7d3adc4512 Automatically force-pull :latest images
We do use some `:latest` images by default for the following services:
- matrix-dimension
- Goofys (in the matrix-synapse role)
- matrix-bridge-appservice-irc
- matrix-bridge-appservice-discord
- matrix-bridge-mautrix-facebook
- matrix-bridge-mautrix-whatsapp

It's terribly unfortunate that those software projects don't release
anything other than `:latest`, but that's how it is for now.

Updating that software requires that users manually do `docker pull`
on the server. The playbook didn't force-repull images that it already
had.

With this patch, it starts doing so. Any image tagged `:latest` will be
force re-pulled by the playbook every time it's executed.

It should be noted that even though we ask the `docker_image` module to
force-pull, it only reports "changed" when it actually pulls something
new. This is nice, because it lets people know exactly when something
gets updated, as opposed to giving the indication that it's always
updating the images (even though it isn't).
2019-06-10 14:30:28 +03:00
Slavi Pantaleev ab59cc50bd Add support for more flexible container port exposing
Fixes #171 (Github Issue).
2019-05-25 07:41:08 +09:00
Dan Arnfield 9c23d877fe Fix docker_image option for ansible < 2.8 2019-05-22 05:43:33 -05:00
Dan Arnfield fa38c84be2 Fix casting int to string warning 2019-05-21 10:37:05 -05:00
Dan Arnfield db15791819 Add source option to docker_image to fix deprecation warning 2019-05-21 10:29:12 -05:00
Dan Arnfield 3982f114af Fix CONDITIONAL_BARE_VARS deprecation warning in ansible 2.8 2019-05-21 10:25:59 -05:00
Slavi Pantaleev ae7c8d1524 Use SyslogIdentifier to improve logging
Reasoning is the same as for matrix-org/synapse#5023.

For us, the journal used to contain `docker` for all services, which
is not very helpful when looking at them all together (`journalctl -f`).
2019-05-16 09:43:46 +09:00
Hugues De Keyzer c451025134 Fix indentation in templates
Use Jinja2 lstrip_blocks option in templates to ensure consistent
indentation in generated files.
2019-05-07 21:23:35 +02:00
Sylvia van Os 75b1528d13 Add the possibility to pass extra flags to the docker container 2019-04-30 16:35:18 +02:00
Slavi Pantaleev af1c9ae59d Do not force firewalld on people
In most cases, there's not really a need to touch the system
firewall, as Docker manages iptables by itself
(see https://docs.docker.com/network/iptables/).

All ports exposed by Docker containers are automatically whitelisted
in iptables and wired to the correct container.

This made installing firewalld and whitelisting ports pointless,
as far as this playbook's services are concerned.

People that wish to install firewalld (for other reasons), can do so
manually from now on.

This is inspired by and fixes #97 (Github Issue).
2019-04-03 11:37:20 +03:00
Slavi Pantaleev 59e37105e8 Add TLS support to Coturn 2019-03-19 10:24:39 +02:00
Slavi Pantaleev 018aeed5e9 Add support for mounting additional volumes to matrix-coturn 2019-03-19 09:16:30 +02:00
Slavi Pantaleev 24cf27c60c Isolate Coturn from services in the default Docker network
Most (all?) of our Matrix services are running in the `matrix` network,
so they were safe -- not accessible from Coturn to begin with.

Isolating Coturn into its own network is a security improvement
for people who were starting other services in the default
Docker network. Those services were potentially reachable over the
private Docker network from Coturn.

Discussed in #120 (Github Pull Request)
2019-03-18 17:41:14 +02:00
Slavi Pantaleev c6858d2a08 Define matrix_coturn_turn_external_ip_address in the playbook group vars
This is more explicit than hiding it in the role defaults.

People who reuse the roles in their own playbook (and not only) may
incorrectly define `ansible_host` to be a hostname or some local address.

Making it more explicit is more likely to prevent such mistakes.
2019-03-18 17:04:40 +02:00
Stuart Mumford e367a2d0de
Add nulls for quotas as well 2019-03-18 11:58:52 +00:00
Stuart Mumford 9d236c5466
Add defaults for ips 2019-03-18 11:44:40 +00:00
Stuart Mumford c0dc56324a
Add config options to turnserver.conf 2019-03-18 11:18:30 +00:00
Slavi Pantaleev 2d56ff0afa Skip some uninstall tasks if not necessary to run 2019-03-13 07:40:51 +02:00
Slavi Pantaleev b066f8a0d8 Do not try to start matrix-coturn.service if not enabled 2019-03-13 07:36:28 +02:00
Slavi Pantaleev aae8757027 Update coturn (4.5.1.0 -> 4.5.1.1) 2019-03-06 19:59:40 +02:00
Slavi Pantaleev 45618679f5 Reload systemd services when they get updated
Fixes #69 (Github Issue)
2019-03-03 11:55:15 +02:00
Slavi Pantaleev a43bcd81fe Rename some variables 2019-02-28 11:51:09 +02:00
Slavi Pantaleev 08635666df Do not attempt to start coturn TLS listeners
We don't provide certificates, so it fails anyway,
but we'd rather suppress the warnings about it too.
2019-02-07 13:20:30 +02:00
Slavi Pantaleev f5cd916de8 Update coturn (4.5.0.8 -> 4.5.1.0) 2019-02-07 13:15:59 +02:00
Slavi Pantaleev 0be7b25c64 Make (most) containers run with a read-only filesystem 2019-01-29 18:52:02 +02:00
Slavi Pantaleev b77b967171 Merge branch 'master' into non-root-containers 2019-01-29 18:00:11 +02:00
Slavi Pantaleev cbc1cdbbf0 Do not try to load certificates
Seems like we unintentionally removed the mounting of certificates
(the `/matrix-config` mount) as part of splitting the playbook into
roles in 51312b8250.

It appears that those certificates weren't necessary for coturn to
funciton though, so we might just get rid of the configuration as well.
2019-01-29 17:56:40 +02:00
Slavi Pantaleev 316d653d3e Drop capabilities in containers
We run containers as a non-root user (no effective capabilities).

Still, if a setuid binary is available in a container image, it could
potentially be used to give the user the default capabilities that the
container was started with. For Docker, the default set currently is:
- "CAP_CHOWN"
- "CAP_DAC_OVERRIDE"
- "CAP_FSETID"
- "CAP_FOWNER"
- "CAP_MKNOD"
- "CAP_NET_RAW"
- "CAP_SETGID"
- "CAP_SETUID"
- "CAP_SETFCAP"
- "CAP_SETPCAP"
- "CAP_NET_BIND_SERVICE"
- "CAP_SYS_CHROOT"
- "CAP_KILL"
- "CAP_AUDIT_WRITE"

We'd rather prevent such a potential escalation by dropping ALL
capabilities.

The problem is nicely explained here: https://github.com/projectatomic/atomic-site/issues/203
2019-01-28 11:22:54 +02:00
Slavi Pantaleev c10182e5a6 Make roles more independent of one another
With this change, the following roles are now only dependent
on the minimal `matrix-base` role:
- `matrix-corporal`
- `matrix-coturn`
- `matrix-mailer`
- `matrix-mxisd`
- `matrix-postgres`
- `matrix-riot-web`
- `matrix-synapse`

The `matrix-nginx-proxy` role still does too much and remains
dependent on the others.

Wiring up the various (now-independent) roles happens
via a glue variables file (`group_vars/matrix-servers`).
It's triggered for all hosts in the `matrix-servers` group.

According to Ansible's rules of priority, we have the following
chain of inclusion/overriding now:
- role defaults (mostly empty or good for independent usage)
- playbook glue variables (`group_vars/matrix-servers`)
- inventory host variables (`inventory/host_vars/matrix.<your-domain>`)

All roles default to enabling their main component
(e.g. `matrix_mxisd_enabled: true`, `matrix_riot_web_enabled: true`).
Reasoning: if a role is included in a playbook (especially separately,
in another playbook), it should "work" by default.

Our playbook disables some of those if they are not generally useful
(e.g. `matrix_corporal_enabled: false`).
2019-01-16 18:05:48 +02:00
Slavi Pantaleev 51312b8250 Split playbook into multiple roles
As suggested in #63 (Github issue), splitting the
playbook's logic into multiple roles will be beneficial for
maintainability.

This patch realizes this split. Still, some components
affect others, so the roles are not really independent of one
another. For example:
- disabling mxisd (`matrix_mxisd_enabled: false`), causes Synapse
and riot-web to reconfigure themselves with other (public)
Identity servers.

- enabling matrix-corporal (`matrix_corporal_enabled: true`) affects
how reverse-proxying (by `matrix-nginx-proxy`) is done, in order to
put matrix-corporal's gateway server in front of Synapse

We may be able to move away from such dependencies in the future,
at the expense of a more complicated manual configuration, but
it's probably not worth sacrificing the convenience we have now.

As part of this work, the way we do "start components" has been
redone now to use a loop, as suggested in #65 (Github issue).
This should make restarting faster and more reliable.
2019-01-12 18:01:10 +02:00