Commit graph

563 commits

Author SHA1 Message Date
Slavi Pantaleev e805044b80 Delete scripts when uninstalling Postgres 2019-07-08 09:38:36 +03:00
Dan Arnfield f83bbf8525 Bump nginx version 2019-07-04 22:59:21 -05:00
Slavi Pantaleev 17cd52ced6 Make Synapse log messages a bit prettier
ef5e4ad061 intentionally makes us conform to
the logging format suggested by the official Docker image.

Reverting this part, because it's uglier.

This likely should be fixed upstream as well though.
2019-07-04 18:19:52 +03:00
Slavi Pantaleev ef5e4ad061 Make Synapse not log to text files
Somewhat related to #213 (Github Pull Request).

We've been moving in the opposite direction for quite a long time.
All services should just leave logging to systemd's journald.
2019-07-04 17:46:31 +03:00
Slavi Pantaleev b84139088c Fix password providers not working on Synapse v1.1.0
Fixes a regression introduced during the upgrade to
Synapse v1.1.0 (in 2b3865ceea).

Since Synapse v1.1.0 upgraded to Python 3.7
(https://github.com/matrix-org/synapse/pull/5546),
we need to use a different modules directory when mounting
password provider modules.
2019-07-04 17:28:38 +03:00
Slavi Pantaleev 73158e6c2f Fix unintentionally inverted boolean
Fixes a problem introduced by da6edc9cba.

Related to #145 (Github Pull Request).
2019-07-04 17:27:42 +03:00
Slavi Pantaleev da6edc9cba Add support for disabling Synapse's local database for user auth
This is a new feature of Synapse v1.1.0.

Discussed in #145 (Github Pull Request).
2019-07-04 17:11:51 +03:00
Slavi Pantaleev 2b3865ceea Upgrade Synapse (1.0.0 -> 1.1.0) 2019-07-04 16:58:45 +03:00
Slavi Pantaleev 8529efcd1c Make Discord bridge configuration playbook-managed
Well, `config.yaml` has been playbook-managed for a long time.
It's now extended to match the default sample config of the Discord
bridge.

With this patch, we also make `registration.yaml` playbook-managed,
which leads us to consistency with all other bridges.

Along with that, we introduce `./config` and `./data` separation,
like we do for the other bridges.
2019-06-26 10:35:00 +03:00
Slavi Pantaleev 918526c5fe Update riot-web (1.2.1 -> 1.2.2) 2019-06-25 14:42:54 +03:00
Dan Arnfield 1eaa7b6967 Update postgres versions to latest 2019-06-24 13:11:23 -05:00
Dan Arnfield ae3a1bb148 Update nginx to 1.17.0 2019-06-24 13:10:58 -05:00
Slavi Pantaleev 37c8b96d06 Use stricter regex in bridges' registration.yaml
I've been thinking of doing before, but haven't.

Now that the Whatsapp bridge does it (since 4797469383),
it makes sense to do it for all other bridges as well.
(Except for the IRC bridge - that one manages most of registration.yaml by itself)
2019-06-24 07:50:51 +03:00
Slavi Pantaleev c876a7df1d Use |regex_escape in Whatsapp registration.yaml
Doesn't matter much, but it makes it consistent with the other bridges.
2019-06-24 07:49:19 +03:00
Slavi Pantaleev 3ff57ed74d Use container network for communication between homeserver and Whatsapp bridge 2019-06-24 07:48:56 +03:00
Slavi Pantaleev 6e26d286af
Merge pull request #207 from tommes0815/whatsapp-config-playbook-managed
Whatsapp config playbook managed
2019-06-24 07:44:26 +03:00
Slavi Pantaleev 62509e4849
Fix indentation consistency 2019-06-24 07:42:39 +03:00
Thomas Kuehne 4797469383 Make WhatsApp bridge configuration playbook-managed
- following spantaleev transition of the telegram brigde
- adding a validate_config task
2019-06-24 00:16:04 +02:00
Lee Verberne 9195ef4c07 Disable appservice-irc log files
appservice-irc doesn't have permission to create files in its project
directory and the intention is to log to the console, anyway. By
commenting out the file names, appservice-irc won't attempt to open the
files.
2019-06-22 08:39:24 +02:00
Slavi Pantaleev 174a6fcd1b Make IRC bridge configuration entirely managed by the playbook 2019-06-19 12:29:44 +03:00
Slavi Pantaleev 668f98a2d3 Escape domain in bridge registration regex 2019-06-19 10:40:59 +03:00
Slavi Pantaleev 380714d290 Talk to Telegram bridge over container network 2019-06-19 10:10:17 +03:00
Slavi Pantaleev f994e40bb7 Extend IRC bridge configuration with some additional options 2019-06-19 09:28:41 +03:00
Slavi Pantaleev 6b023d09d4 Use container network address for communication between IRC bridge and homeserver
This means we need to explicitly specify a `media_url` now,
because without it, `url` would be used for building public URLs to
files/images. That doesn't work when `url` is not a public URL.
2019-06-19 09:21:13 +03:00
Slavi Pantaleev 169b09f0ed Fix token mismatch error for the Telegram bridge
Regression since 4e8543ce21
2019-06-15 12:01:52 +03:00
Slavi Pantaleev 4e8543ce21 Make Telegram bridge configuration playbook-managed 2019-06-15 09:43:43 +03:00
Slavi Pantaleev 2902b53267 Minor fixes for consistency 2019-06-15 09:42:40 +03:00
Slavi Pantaleev 00383a73ac Make running --tags=setup-synapse only not fail to register bridges
Until now, if `--tags=setup-synapse` was used, bridge tasks would not
run and bridges would fail to register with the `matrix-synapse` role.
This means that Synapse's configuration would be generated with an empty
list of appservices (`app_service_config_files: []`).

.. and then bridges would fail, because Synapse would not be aware of
there being any bridges.

From now on, bridges always run their init tasks and always register
with Synapse.

For the Telegram bridge, the same applies to registering with
matrix-nginx-proxy. Previously, running `--tags=setup-nginx-proxy` would
get rid of the Telegram endpoint configuration for the same reason.
Not anymore.
2019-06-14 10:19:52 +03:00
Slavi Pantaleev d8a4007220 Upgrade exim (4.91 -> 4.92)
Note: https://www.us-cert.gov/ncas/current-activity/2019/06/13/Exim-Releases-Security-Patches

That said, I don't believe we've been affected.
Not in a bad way at least, because:
- we run exim as non-root and capabilities dropped
- we run exim in a private Docker network with known trusted relayers
(Synapse and mxisd)
2019-06-14 08:07:54 +03:00
Slavi Pantaleev 3956b300ed Disable riot-web's welcome bot
I've not found this welcome bot to work at all in my previous attempts.
It would simply not reply, even though federation works.

It seems like this is also a potential privacy issue, as per
https://gist.github.com/maxidorius/5736fd09c9194b7a6dc03b6b8d7220d0
2019-06-14 07:49:46 +03:00
Slavi Pantaleev 2e16257e50 Do not ask for _matrix._tcp SRV records anymore
With most people on Synapse v0.99+ and Synapse v1.0 now available,
we should no longer try to be backward compatible with Synapse 0.34,
because this just complicates the instructions for no good reason.
2019-06-12 14:51:10 +03:00
Slavi Pantaleev d8afb241ca
Merge pull request #201 from aaronraimist/default-room-version
Allow default room version to be configured
2019-06-12 09:17:45 +03:00
Slavi Pantaleev f4574961c7
Prevent double-quotes around default room version
Using `|to_json` on a string is expected to correctly wrap it in quotes (e.g. `"4"`).
Wrapping it explicitly in double-quotes results in undesirable double-quoting (`""4""`).
2019-06-12 09:17:35 +03:00
Aaron Raimist 483bdd8c01
Allow default room version to be configured 2019-06-11 21:18:06 -05:00
Aaron Raimist d262028d82
Upgrade mxisd (1.4.4 -> 1.4.5) 2019-06-11 20:19:15 -05:00
Slavi Pantaleev e4068e55ee Upgrade Synapse (0.99.5.2 -> 1.0.0) 2019-06-11 20:30:18 +03:00
Slavi Pantaleev 7d3adc4512 Automatically force-pull :latest images
We do use some `:latest` images by default for the following services:
- matrix-dimension
- Goofys (in the matrix-synapse role)
- matrix-bridge-appservice-irc
- matrix-bridge-appservice-discord
- matrix-bridge-mautrix-facebook
- matrix-bridge-mautrix-whatsapp

It's terribly unfortunate that those software projects don't release
anything other than `:latest`, but that's how it is for now.

Updating that software requires that users manually do `docker pull`
on the server. The playbook didn't force-repull images that it already
had.

With this patch, it starts doing so. Any image tagged `:latest` will be
force re-pulled by the playbook every time it's executed.

It should be noted that even though we ask the `docker_image` module to
force-pull, it only reports "changed" when it actually pulls something
new. This is nice, because it lets people know exactly when something
gets updated, as opposed to giving the indication that it's always
updating the images (even though it isn't).
2019-06-10 14:30:28 +03:00
Slavi Pantaleev 4f87f7e43e
Explain matrix_postgres_container_postgres_bind_port a little more
Previously, it only mentioned exposing for psql-usage purposes.

Realistically, it can be used for much more. Especially given that
psql can be easily accessed via our matrix-postgres-cli script,
without exposing the container port.
2019-06-10 08:24:37 +03:00
Aaron Raimist 6fce809d10
Add config option to be able to access database outside of container 2019-06-09 20:35:35 -05:00
Aaron Raimist 79f4bcf5be
Enable sentry.io integration 2019-06-07 16:02:41 -05:00
Slavi Pantaleev 44156fe659 Fix Ansible 2.8 deprecation in Dimension role 2019-06-07 17:44:32 +03:00
Slavi Pantaleev 3567d9adba Fix typo 2019-06-07 16:07:01 +03:00
Slavi Pantaleev a9953dd641 Make Facebook/Telegram bridges not log to files
We log to journald anyway. There's no need for double-logging.

It should not that matrix-synapse logs to journald and to files,
but that's likely to change in the future as well.
Because Synapse's logs are insanely verbose right now (and may get
dropped by journald), it's more reliable to have file-logging too.

As Synapse matures and gets more stable, logging should hopefully
get less, we should be able to only use journald and stop writing to
files for it as well.
2019-06-07 15:48:13 +03:00
Slavi Pantaleev 18baeabdf2 Do not create Facebook bridge directories with recurse: true
I'm not sure what I had in mind when I added this earlier,
but I think we'd better go without it.
2019-06-07 15:18:29 +03:00
Slavi Pantaleev bf446b6e15 Fix double mv command 2019-06-07 15:06:21 +03:00
Slavi Pantaleev 172b0fa88c Separate Facebook bridge configuration and data
Using a separate directory allows easier backups
(only need to back up the Ansible playbook configuration and the
bridge's `./data` directory).

The playbook takes care of migrating an existing database file
from the base directory into the `./data` directory.

In the future, we can also mount the configuration read-only,
to ensure the bridge won't touch it.
For now, mautrix-facebook is keen on rebuilding the `config.yaml`
file on startup though, so this will have to wait.
2019-06-07 14:52:38 +03:00
Slavi Pantaleev 330648a3e0 Make Facebook bridge configuration playbook-managed
Related to #193, but for the Facebook bridge.
(other bridges can be changed to do the same later).

This patch makes the bridge configuration entirely managed by the
Ansible playbook. The bridge's `config.yaml` and `registration.yaml`
configuration files are regenerated every time the playbook runs.

This allows us to apply updates to those files and to avoid
people having to manage the configuration files manually on the server.

-------------------------------------------------------------

A deficiency of the current approach to dumping YAML configuration in
`config.yaml` is that we strip all comments from it.
Later on, when the bridge actually starts, it will load and redump
(this time with comments), which will make the `config.yaml` file
change.

Subsequent playbook runs will report "changed" for the
"Ensure mautrix-facebook config.yaml installed" task, which is a little
strange.

We might wish to improve this in the future, if possible.

Still, it's better to have a (usually) somewhat meaningless "changed"
task than to what we had -- never rebuilding the configuration.
2019-06-07 14:05:53 +03:00
Slavi Pantaleev d6d6c152a3 Delay bridge startup to ensure Synapse is up
Bridges start matrix-synapse.service as a dependency, but
Synapse is sometimes slow to start, while bridges are quick to
hit it and die (if unavailable).

They'll auto-restart later, but .. this still breaks `--tags=start`,
which doesn't wait long enough for such a restart to happen.

This attempts to slow down bridge startup enough to ensure Synapse
is up and no failures happen at all.
2019-06-07 12:15:37 +03:00
Slavi Pantaleev 328d981b05 Fix undefined variables in mxisd and Dimension configuration 2019-06-07 11:46:35 +03:00
Slavi Pantaleev 99086f90e8 Upgrade riot-web (1.2.0 -> 1.2.1) 2019-06-04 19:31:14 +03:00
kingoftheconnors c06b47af77 Fixed error message to direct users to the right debug command 2019-06-02 18:24:18 -04:00
Slavi Pantaleev 3bc8aa0a82 Upgrade Synapse (0.99.5.1 -> 0.99.5.2) 2019-05-30 20:50:09 +03:00
Slavi Pantaleev 35892286a1 Upgrade mxisd (1.4.3 -> 1.4.4) 2019-05-30 17:22:39 +03:00
Slavi Pantaleev 2982b03809 Explicitly serialize matrix_synapse_app_service_config_files
Attempt to fix #192 (Github Issue), potential regression since
70487061f4.

Serializing as JSON/YAML explicitly is much better than relying on
magic (well, Python serialization being valid YAML..).
It seems like Python may prefix strings with `u` sometimes (Python 3?),
which causes Python serialization to not be compatible with YAML.
2019-05-30 09:42:08 +03:00
Slavi Pantaleev f1c124331f Upgrade riot-web (1.1.2 -> 1.2.0) 2019-05-30 08:44:21 +03:00
Slavi Pantaleev 70487061f4 Prefer --mount instead of -v for mounting volumes
This doesn't replace all usage of `-v`, but it's a start.

People sometimes troubleshoot by deleting files (especially bridge
config files). Restarting Synapse with a missing registration.yaml file
for a given bridge, causes the `-v
/something/registration.yaml:/something/registration.yaml:ro` option
to force-create `/something/registration.yaml` as a directory.

When a path that's provided to the `-v` option is missing, Docker
auto-creates that path as a directory.
This causes more breakage and confusion later on.

We'd rather fail, instead of magically creating directories.
Using `--mount`, instead of `-v` is the solution to this.

From Docker's documentation:

> When you use --mount with type=bind, the host-path must refer to an existing path on the host.
> The path will not be created for you and the service will fail with an error if the path does not exist.
2019-05-29 09:59:50 +03:00
Slavi Pantaleev 7d8dde8a53 Add support for proxying /_synapse/admin APIs
Fixes #191 (Github Issue).
2019-05-29 08:32:24 +03:00
Slavi Pantaleev 5361d3a412 Fix Telegram bridge proxying config when matrix-nginx-proxy disabled
Related to #189 (Github Issue).

People had proxying problems if:
- they used the whole playbook (including the `matrix-nginx-proxy` role)
- and they were disabling the proxy (`matrix_nginx_proxy_enabled: false`)
- and they were proxying with their own nginx server

For them,
`matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks`
would not be modified to inject the necessary proxying configuration.
2019-05-27 10:04:52 +03:00
Slavi Pantaleev 7379968a3c Fix Telegram bridge HTTP proxying when not using matrix-nginx-proxy
From what I see, this was never implemented to begin with.

Fixes #189 (Github Issue).
2019-05-26 20:50:52 +03:00
Slavi Pantaleev 120abaf391 Upgrade Telegram bridge (0.5.1 -> 0.5.2) 2019-05-26 20:41:21 +03:00
Slavi Pantaleev ab59cc50bd Add support for more flexible container port exposing
Fixes #171 (Github Issue).
2019-05-25 07:41:08 +09:00
Slavi Pantaleev be2812bc8f Remove unnecessary variables
Continuation of 54a281a425.
Related to #188 (Github Pull Request).
2019-05-24 08:01:24 +09:00
Jason Locklin 54a281a425 Config validation not required
Validate_config was copied from the telegram bridge code, but doesn't
apply to the facebook bridge.
2019-05-23 15:02:44 -04:00
Slavi Pantaleev a8b633561d Upgrade Synapse (v0.99.4 -> v0.99.5.1) 2019-05-23 09:23:04 +09:00
Dan Arnfield 9c23d877fe Fix docker_image option for ansible < 2.8 2019-05-22 05:43:33 -05:00
Dan Arnfield fa38c84be2 Fix casting int to string warning 2019-05-21 10:37:05 -05:00
Dan Arnfield db15791819 Add source option to docker_image to fix deprecation warning 2019-05-21 10:29:12 -05:00
Dan Arnfield 3982f114af Fix CONDITIONAL_BARE_VARS deprecation warning in ansible 2.8 2019-05-21 10:25:59 -05:00
Slavi Pantaleev affb99003c Improve Synapse variable naming consistency 2019-05-21 12:09:38 +09:00
Slavi Pantaleev 5c821b581a Check fullchain.pem, not cert.pem
While using certbot means we'll have both files retrieved,
it's actually the fullchain.pem file that we use in nginx configuration.

Using that one for the check makes more sense.
2019-05-21 11:58:18 +09:00
Slavi Pantaleev 3250df6765 Make bridge uninstallation stop services
Fixes #155 (Github Issue)
2019-05-21 11:27:09 +09:00
Slavi Pantaleev a1e9818356 Update comment 2019-05-21 11:25:32 +09:00
Slavi Pantaleev 3ece9375c6
Merge pull request #179 from spantaleev/separate-bridge-roles
Move bridges into separate roles
2019-05-21 11:05:30 +09:00
Slavi Pantaleev fc7ba153b1 Make matrix-synapse role respect matrix_synapse_enabled flag 2019-05-21 10:46:49 +09:00
Slavi Pantaleev 8d654aecdd Improve file naming consistency 2019-05-21 09:57:48 +09:00
Stuart Mumford a4bcd7ce8f
Add a variable to control the stop tasks 2019-05-20 17:03:05 +01:00
Slavi Pantaleev e3b4622ac8 Split Synapse extension tasks into install/uninstall files 2019-05-18 06:36:54 +09:00
Slavi Pantaleev 663d1add92 Move matrix-appservice-discord into a separate role 2019-05-18 01:14:12 +09:00
Slavi Pantaleev 13c4e7e5b6 Merge branch 'master' into separate-bridge-roles 2019-05-16 09:45:06 +09:00
Slavi Pantaleev ae7c8d1524 Use SyslogIdentifier to improve logging
Reasoning is the same as for matrix-org/synapse#5023.

For us, the journal used to contain `docker` for all services, which
is not very helpful when looking at them all together (`journalctl -f`).
2019-05-16 09:43:46 +09:00
Slavi Pantaleev cf3117011b Upgrade Synapse (0.99.3.2 -> 0.99.4) 2019-05-16 09:20:43 +09:00
Slavi Pantaleev 6db10ed6f3 Upgrade riot-web (1.1.1 -> 1.1.2) 2019-05-16 09:09:42 +09:00
Slavi Pantaleev 3339e37ce9 Move matrix-appservice-irc into a separate role 2019-05-16 09:07:40 +09:00
Slavi Pantaleev 854cf84aa3 Upgrade riot-web (1.1.0 -> 1.1.1) 2019-05-15 09:50:25 +09:00
Slavi Pantaleev 47745254f9
Merge pull request #180 from eMPee584/fix-commented-mxisd-ldap-option
Fix case of the mxisd ldap.connection.baseDNs option in matrix_mxisd_configuration_extension_yaml comment
2019-05-15 09:49:34 +09:00
Slavi Pantaleev 43fd3cc274 Move mautrix-facebook into a separate role 2019-05-15 09:34:31 +09:00
Marcel Partap 5aa7f637d8 Fix matrix_synapse_ext_password_provider_ldap_start_tls (it's boolean) 2019-05-14 23:09:59 +02:00
Marcel Partap 25d3b315de Fix case of the mxisd ldap.connection.baseDNs option in comment 2019-05-14 22:38:21 +02:00
Slavi Pantaleev bb816df557 Move mautrix telegram and whatsapp into separate roles
The goal is to move each bridge into its own separate role.
This commit starts off the work on this with 2 bridges:
- mautrix-telegram
- mautrix-whatsapp

Each bridge's role (including these 2) is meant to:

- depend only on the matrix-base role

- integrate nicely with the matrix-synapse role (if available)

- integrate nicely with the matrix-nginx-proxy role (if available and if
required). mautrix-telegram bridge benefits from integrating with
it.

- not break if matrix-synapse or matrix-nginx-proxy are not used at all

This has been provoked by #174 (Github Issue).
2019-05-14 23:47:22 +09:00
Marcel Partap d114736014 Add a task to stop services (and remove containers) 2019-05-14 11:46:07 +02:00
Slavi Pantaleev 873c291be6 Fix appservice-discord configuration-extension merging 2019-05-14 08:24:03 +09:00
Slavi Pantaleev 953ae021ba Upgrade mxisd (1.4.2 -> 1.4.3) 2019-05-14 08:22:10 +09:00
Dan Arnfield 6163ba5bb1 Bump postgres versions 2019-05-10 08:02:32 -05:00
inthewaves 51e408bc94
Bump riot-web version (1.0.8->1.1.0) 2019-05-10 05:57:28 +00:00
Slavi Pantaleev 216cdf8c74
Merge pull request #166 from izissise/mautrix-facebook
Mautrix facebook
2019-05-09 10:05:14 +03:00
Slavi Pantaleev 565236a49c
Merge pull request #168 from danbob/nginx-proxy-3pid-registration
Add option to proxy 3pid registration endpoints
2019-05-09 09:53:47 +03:00
Dan Arnfield 171c6db41e Add option to proxy 3pid registration endpoints 2019-05-08 13:49:51 -05:00
Dan Arnfield 958ad68078 Add registrations_require_3pid synapse option 2019-05-08 12:29:18 -05:00
Hugues Morisset a82d5ed281 Add tulir mautrix-facebook (https://github.com/tulir/mautrix-facebook) 2019-05-08 17:11:07 +02:00
Slavi Pantaleev 5f2f17cb1e
Merge pull request #160 from danbob/fix-matrix-mxisd-config
Fix template indentation
2019-05-08 08:01:00 +03:00