Commit graph

735 commits

Author SHA1 Message Date
Slavi Pantaleev 21a108262f Remove some leftover debug statements 2018-08-17 09:30:41 +03:00
Slavi Pantaleev 25becc63d5 Minor fixups for the matrix-synapse-rest-auth handling 2018-08-17 09:25:54 +03:00
Slavi Pantaleev ea43d46b70 Add matrix-synapse-rest-auth support 2018-08-17 09:02:17 +03:00
Slavi Pantaleev df79901f8b Improve compression support 2018-08-17 08:00:38 +03:00
Slavi Pantaleev a302a7d748 Only run federation on 8448 and client on 80
This disables federation on the 80 port, as it's
not necessary. We also disable the old Angular webclient.

For the federation port (8448), we disable the client APIs
as those are not necessary. Those can even cause trouble
if one doesn't know about them and thinks that guarding the client
APIs at the 80 port is enough.
2018-08-17 07:55:58 +03:00
Slavi Pantaleev 74093dfb15 Add mxisd Identity Server support 2018-08-15 10:46:13 +03:00
Slavi Pantaleev 617712000e Minor wording improvements 2018-08-15 10:19:31 +03:00
Slavi Pantaleev 1c71cb110e Change SSL certificate obtaining a bit 2018-08-14 14:50:10 +03:00
Slavi Pantaleev 21da2f572b Add email-sending support 2018-08-14 14:47:44 +03:00
Slavi Pantaleev cab54879d1 Update dependencies 2018-08-11 09:44:11 +03:00
Slavi Pantaleev 084a0a0e53 Minor consistency improvement 2018-08-08 10:47:03 +03:00
Slavi Pantaleev 700602eed3 Rename a bunch of playbook variables for better consistency 2018-08-08 09:17:18 +03:00
Slavi Pantaleev 336785d1ed Rename Ansible playbook tag (setup-main -> setup-all) 2018-08-08 09:03:37 +03:00
Slavi Pantaleev 3fd6fd647f Put all containers in their own isolated Docker network (matrix)
Moving away from using the default bridge network to using our own.
This isolates our services from other Docker containers running
on the default network on the same host.

The benefits are that:

- isolation is a little better - we no longer share a default
bridge network with any other containers that might be running on the host

- there are no longer hard dependencies - we do service discovery
by DNS name, and not via explicit `--link` usage during container start,
so containers can start out of order and fail without bringing down others
with them
(`matrix-nginx-proxy` can continue running, even if one of the other services dies)

In the future, when other services get introduced,
the increased resilience and simplicity will help as well.
2018-08-08 08:57:48 +03:00
Slavi Pantaleev b88fe971d6 Fix matrix-nginx-proxy.service dependency on riot-web, if riot-web disabled 2018-08-07 15:39:57 +03:00
Slavi Pantaleev cdf4eefdf9 Fix typos 2018-08-07 15:08:46 +03:00
Slavi Pantaleev f6950612a5 Upgrade dependencies 2018-08-02 21:26:25 +03:00
Slavi Pantaleev 6cb14be162 Upgrade dependencies 2018-07-30 16:18:17 +03:00
Slavi Pantaleev ae7e8e61c6 Update dependencies 2018-07-20 08:28:02 -04:00
Slavi Pantaleev e4d0a68460 Update riot-web (0.15.5 -> 0.15.6) 2018-06-30 18:51:25 +03:00
Slavi Pantaleev 839b401b28 Set up Synapse configuration using a template (not line/regexp replacements)
Until now, we were starting from a fresh configuration, as generated
by Synapse and manipulating it with regex and line replacements,
until we made it work.

This is more fragile and less predictable, so we're moving to a static
configuration file generated from a Jinja template.

The upside is that configuration will be stable and predictable.

The downside of this new approach is that any manual configuration changes
after the playbook is done, will be thrown away on future playbook
invocations.

There are 2 ways to work around the need for manual configuration
changes though:
- making them part of this playbook and its default template
configuration files (which benefits everyone)
- going your own way for a given host and overriding the template files
that gets used (that is, the
`matrix_synapse_template_synapse_homeserver` or
`matrix_synapse_template_synapse_log` variables)
2018-06-26 21:05:59 +03:00
Slavi Pantaleev add8169c33 Remove deprecated "ssl" directive from nginx configuration 2018-06-26 20:43:57 +03:00
Slavi Pantaleev 053328be08 Fix nginx failing to start on certain low-cache CPUs 2018-06-26 20:40:48 +03:00
Slavi Pantaleev 1725c3e698 Upgrade riot-web (0.15.4 -> 0.15.5) 2018-06-21 09:44:38 +03:00
Slavi Pantaleev 6335485ad3 Upgrade Synapse (0.31.1 -> 0.31.2) 2018-06-15 00:00:01 +03:00
Slavi Pantaleev a5877eadd4 Upgrade Synapse (0.31.0 -> 0.31.1) 2018-06-08 18:25:12 +03:00
Slavi Pantaleev 4ebf6bbf27 Upgrade Synapse (0.30.0 -> 0.31.0) 2018-06-07 10:14:36 +03:00
Slavi Pantaleev ff8f1bc65f Upgrade Docker images 2018-06-06 17:35:29 +03:00
Slavi Pantaleev 1670a20937 Clean up riot-web UI (disable custom URLs and guest usage)
This playbook does not set up guest access in Synapse anyway,
so until the need comes (or someone asks for it), guest access
is removed from riot-web's UI too.

As for supporting custom URLs, this is also not something
that seems like it'd be useful to most deployments.
2018-05-30 15:36:09 +03:00
Slavi Pantaleev 5399e2b6bb Do not require (but want) matrix-coturn.service in matrix-synapse
It's not really a requirement, as it can function without it.
Also, restarting matrix-coturn doesn't need to restart matrix-synapse.
2018-05-29 13:38:41 +03:00
Slavi Pantaleev 62d1b13c91 Minor improvements to the Postgres-upgrade tasks 2018-05-29 11:23:17 +03:00
Slavi Pantaleev d107ab2540 Add support for upgrading Postgres
Since cbee084ac1, this playbook supports Postgres 10.x,
but keeps existing Postgres-9.x installs on 9.x.

This playbook can now also be ran with `--tags=upgrade-postgres`
to make it upgrade from Postgres 9.x to 10.x (or other versions
in the future).
2018-05-28 20:40:42 +03:00
Slavi Pantaleev cbee084ac1 Use Postgres 10.x by default (only for new installs)
This playbook just tries to avoid trying to setup a Postgres 10
database with existing 9.x files, as that makes Postgres complain.

Due to this, existing installs (still on 9.x) are detected
and left on Postgres 9.x.
They need to be upgraded to Postgres 10.x manually.
2018-05-28 20:16:02 +03:00
Slavi Pantaleev f1b4730e82 Update Docker images 2018-05-26 12:25:09 +03:00
Slavi Pantaleev b3e62126db Switch Docker image to official one
Switching from from avhost/docker-matrix (silviof/docker-matrix)
to matrixdotorg/synapse.

The avhost/docker-matrix (silviof/docker-matrix) image used to bundle
in the coturn STUN/TURN server, so as part of the move,
we're separating this to a separately-ran service
(matrix-coturn.service, powered by instrumentisto/coturn-docker-image)
2018-05-25 21:58:53 +03:00
Slavi Pantaleev 3af3ef48fc Make .log.config modifications respect whitespace
A `.log.config` file may be generated with a different
level of indentation depending on which (Docker image, etc.)
generates it.

With this patch, we tolerate different levels of indentation
(2 spaces, 4 spaces, etc.) and don't break the configuration.
2018-05-25 13:15:17 +03:00
Slavi Pantaleev 67a98e51d9 Make the riot-web container run without root privileges 2018-05-14 14:31:43 +03:00
Slavi Pantaleev bd580d3b9a Update dependencies 2018-05-14 14:31:00 +03:00
Slavi Pantaleev a367172b67 Update dependencies 2018-04-28 13:38:44 +03:00
Slavi Pantaleev 7de11261b1 Update Docker images 2018-04-11 18:51:32 +02:00
Slavi Pantaleev af54d60b0f Update Docker images 2018-04-03 18:49:05 +03:00
Slavi Pantaleev 5d9ddd1627 Update Docker images 2018-03-16 10:22:09 +02:00
Slavi Pantaleev efc78fb9d3 Switch from s3fs to Goofys
Improves performance of media store operations.
2018-02-20 21:36:08 +02:00
Slavi Pantaleev db686c3f8e Update dependencies 2018-02-13 23:19:50 +02:00
Slavi Pantaleev bfca91ac1f Switch Matrix Docker images (silviof -> AVENTER-UG)
Silvio announced that he's no longer maintaining his images,
so we're jumping to AVENTER-UG's fork.
2018-01-10 22:11:32 +02:00
Slavi Pantaleev 534f78f9d0 Update Docker image versions 2017-12-07 22:53:43 +02:00
Slavi Pantaleev d14ef08d5b Fix SSL certificate renewal for the custom-proxy-server case
When using matrix-nginx-proxy, the file permissions are organized
in a way that matrix-nginx-proxy could read the challenge files
produced by acmetool.

However, when another own/external webserver was used (like nginx
with our generated sample configuration), this could not work.
From on we're proxying the HTTP requests to port :402 in such a case,
which fixes the problem.
2017-12-01 12:07:27 +01:00
Slavi Pantaleev f476e49e64 Make SSL renewal time configurable and nginx-proxy reload time adequate
The matrix-nginx-proxy was reloaded on the 3rd day of the month (`15 4 3 * *`),
which makes no sense - it's too infrequently.

It's in line with the renewal time now (+5 minutes).
2017-11-11 10:38:38 +02:00
Slavi Pantaleev 57e4f12ad3 Add support for using a pre-configured Macaroon secret key 2017-10-24 15:29:19 +09:00
Slavi Pantaleev dd5cabf658 Make /matrix owned by matrix:matrix 2017-10-24 13:01:11 +09:00
Slavi Pantaleev 19e191f0bd Do not assign Docker container name for certificate renewal
Prevents clashes with other services like this one.
2017-10-16 08:42:27 +03:00
Slavi Pantaleev 1c2d59ae91 Stop using patched synapse_port_db script
The non-working script is supposed to be fixed
by https://github.com/matrix-org/synapse/pull/2375

To have it work, we'd need an updated Docker image
of `silviof/matrix-riot-docker:latest`, which is not yet available
at the time of this commit.

Still, the previous patched synapse_port_db didn't work well either,
so it's not like we're regressing much by getting rid of it.
2017-10-14 09:58:06 +03:00
Slavi Pantaleev 767b321f60 Do not mount certain Docker volumes as read-write unnecessarily 2017-10-01 11:36:30 +03:00
Slavi Pantaleev 2906ec3045 Fix SSL-renewal problem caused by incorrect permissions 2017-10-01 11:26:20 +03:00
Slavi Pantaleev 3a5f82267b Do not use Let's Encrypt certificate for Synapse's federation port
As described here (
https://github.com/matrix-org/synapse/issues/2438#issuecomment-327424711
), using own SSL certificates for the federation port is more fragile,
as renewing them could cause federation outages.

The recommended setup is to use the self-signed certificates generated
by Synapse.

On the 443 port (matrix-nginx-proxy) side, we still use the Let's Encrypt
certificates, which ensures API consumers work without having to trust
"our own CA".

Having done this, we also don't need to ever restart Synapse anymore,
as no new SSL certificates need to be applied there.

It's just matrix-nginx-proxy that needs to be restarted, and it doesn't
even need a full restart as an "nginx reload" does the job of swithing
to the new SSL certificates.
2017-09-23 15:29:15 +03:00
Slavi Pantaleev 6962bfcc42 Add support for not taking over a server (no matrix-nginx-proxy) and disabling Riot 2017-09-12 12:41:44 +03:00
Slavi Pantaleev cb323f5b4c Move SSL certificates from /etc/pki/acmetool-certs to /matrix/ssl
Moving keeps everything in the /matrix directory, so that we
wouldn't contaminate anything else on the system or risk
clashing with something else.

Also retrieving certificates separately for the Riot and Matrix domains,
which should help in multiple ways:

- allows them to be very different (completely separate base domain..)

- allows for Riot to be disabled for the playbook some time later
  and still have the code not break
2017-09-11 23:50:14 +03:00
Slavi Pantaleev ded7c274f6 Add support for Debian (9+) and Ubuntu (16.04+) 2017-09-11 23:24:05 +03:00
Slavi Pantaleev 13ab9eb238 Do not touch hostname and timezone
Let's let the admin set them as they wish.
We don't care what they are anyway.

If other things run on the same server,
it's also better not to hijack these for our
own purposes, especially when we don't need to.

The timedatectl call also seems to fail on Ubuntu 17.04
for some reason (missing timezones information file?).
2017-09-11 22:55:05 +03:00
Slavi Pantaleev 7c049be11a Update postgres and nginx 2017-09-11 22:16:51 +03:00
Slavi Pantaleev f422e379c2 Do not try to start postgres when it's external 2017-09-11 00:58:22 +03:00
Slavi Pantaleev ab1a9fd87e Add support for using an external PostgreSQL server 2017-09-08 17:24:27 +03:00
Slavi Pantaleev f6be25a6ae Fix idempotency problem when getting rid of S3 setup 2017-09-08 16:34:40 +03:00
Slavi Pantaleev ac59192696 Do not leave containers behind after matrix-postgres-cli usage 2017-09-08 14:18:12 +03:00
Slavi Pantaleev 49e5dad86d Do not do the S3 setup so early
It was never intended to be there, but was while testing/development
and got forgotten later.
2017-09-08 10:50:31 +03:00
Slavi Pantaleev 0f43abb91d Do not assume /usr/local/bin is always on the PATH 2017-09-08 10:47:12 +03:00
Slavi Pantaleev 9c68b057b0 Add support for storing Matrix Synapse's media_store to Amazon S3 2017-09-07 18:26:41 +03:00
Slavi Pantaleev 9b97ab6a90 Do not wastefully preserve owner/group when importing media store files 2017-09-07 12:27:32 +03:00
Slavi Pantaleev 0f723c9574 Ensure media store files are owned by the correct user/group after importing (recursively) 2017-09-07 12:24:04 +03:00
Slavi Pantaleev a6760f4469 Ensure media store files are owned by the correct user/group after importing 2017-09-07 12:23:22 +03:00
Slavi Pantaleev ea91ef7fb2 Move media_store & logs out of /data. Allow logging to be configured
The goal is to allow these to be on separate partitions
(including remote ones in the future).

Because the `silviof/docker-matrix` image chowns
everything to MATRIX_UID:MATRIX_GID on startup,
we definitely don't want to include `media_store` in it.
If it's on a remote FS, it would cause a slow startup.

Also, adding some safety checks to the "import media store"
task, after passing a wrong path to it on multiple occassions and
wondering what's wrong.

Also, making logging configurable. The default of keeping 10x100MB
log files is likely excessive and people may want to change that.
2017-09-07 12:12:31 +03:00
Slavi Pantaleev 2bb8bb96d4 Add support for configuring max_upload_size 2017-08-30 12:07:03 +03:00
Slavi Pantaleev b046052aed Switch from playbook vars to role defaults
By using role defauts, we can have inventory variables
which overide the defaults.
2017-08-30 12:05:13 +03:00
Slavi Pantaleev ce3c31eb41 Adjust x_forwarded setting for the plain (8008) port
Port 8008 is forwarded in our case, so unless we adjust
`x_forwaded` for it, Docker's local network IPs are
logged/displayed for devices.

The TLS port (8448) is not proxied in our setup,
so its `x_forwarded` setting remains `false`.
2017-08-12 18:32:24 +03:00
Slavi Pantaleev 1cd227b699 Increase max body size for the nginx proxy
Otherwise, we can't support large media file uploads.
2017-08-12 15:39:21 +03:00
Slavi Pantaleev 887d14884e Do not generate a new Matrix Synapse config, if one already exists
Otherwise certains values in the config file,
such as `macaroon_secret_key`, would be regenerated,
which is not something that we want.

If `macaroon_secret_key` is regenerated, all users'
auth tokens will become invalid (effectively logging out
all users).
2017-08-12 11:06:57 +03:00
Slavi Pantaleev 7d74dced28 Disable TURN for guests and enable URL previews 2017-08-12 10:36:34 +03:00
Slavi Pantaleev fc3386dff7 Fix matrix-synapse-register-user for when the non-admin case
It was trying to omit the `-a` flag, but that wasn't enough,
because the underlying `register_new_matrix_user` command
prompts interactively if it doesn't see the `-a` flag
(it doesn't default to non-admin).

We need to answer such interactive prompts.
2017-08-11 08:21:34 +03:00
Slavi Pantaleev b074030d59 Fix problem detecting matrix-nginx-proxy state when it is not installed 2017-08-08 14:31:01 +03:00
Slavi Pantaleev 02bdb7c7bc Ensure firewalld is installed and Docker launcher after firewalld
Some CentOS 7 hosts may not have firewalld installed.
We shouldn't expect it to be, but should ensure by ourselves that it is.

Docker likes to mess around with iptables forwarding rules,
so it ought to start after firewalld.
2017-08-08 14:29:14 +03:00
Slavi Pantaleev a3ecb7bfd9 Add http->http redirection for Matrix/Riot
Doing this means that matrix-nginx-proxy now starts
occupying port 80, which necessitates that
SSL renewal happens slightly differently.
2017-08-06 19:10:50 +03:00
Slavi Pantaleev 0d1a2071de Ensure playbook works if running at SSL-renewal time
matrix-nginx-proxy will be occupying port 80 soon,
so that we can be more user-friendly and have
http->https forwarding for the Riot hostname.

During the playbook run, acmetool also expects to use
port 80 for domain verification.

During an initial playbook run, this wouldn't cause trouble
because matrix-nginx-proxy is not installed yet.

However, on subsequent playbook runs, it would cause trouble.

This ensures that if matrix-nginx-proxy is available
and running, it would be stopped before running acmetool
and started right after.
2017-08-06 18:40:16 +03:00
Slavi Pantaleev 13eaee2463 Enable TURN support and UDP for STUN 2017-08-06 18:26:06 +03:00
Slavi Pantaleev 81077e6cdf Allow regular users to be created as well (not only admins) 2017-08-01 11:11:29 +03:00
Slavi Pantaleev 87f5883f24 Initial commit 2017-07-31 23:08:20 +03:00