Commit graph

270 commits

Author SHA1 Message Date
Slavi Pantaleev 5135c0cc0a Add Ansible guide and Ansible version checks
After having multiple people report issues with retrieving
SSL certificates, we've finally discovered the culprit to be
Ansible 2.5.1 (default and latest version on Ubuntu 18.04 LTS).

As silly as it is, certain distributions ("LTS" even) are 13 bugfix
versions of Ansible behind.

From now on, we try to auto-detect buggy Ansible versions and tell the
user. We also provide some tips for how to upgrade Ansible or
run it from inside a Docker container.

My testing shows that Ansible 2.4.0 and 2.4.6 are OK.
All other intermediate 2.4.x versions haven't been tested, but we
trust they're OK too.

From the 2.5.x releases, only 2.5.0 and 2.5.1 seem to be affected.
Ansible 2.5.2 corrects the problem with `include_tasks` + `with_items`.
2019-01-03 16:24:14 +02:00
Slavi Pantaleev 99af4543ac Replace include usage with include_tasks and import_tasks
The long-deprecated (since Ansible 2.4) use of include is
no more.
2019-01-03 15:24:08 +02:00
Slavi Pantaleev 76506f34e0 Make media-store restore work with server files, not local
This is a simplification and a way to make it consistent with
how we do Postgres imports (see 6d89319822), using
files coming from the server, not from the local machine.

By encouraging people NOT to use local files,
we potentially avoid problems such as #34 (Github issue),
where people would download `media_store` to their Mac's filesystem
and case-sensitivity issues will actually corrupt it.

By not encouraging local files usage, it's less likely that
people would copy (huge) directories to their local machine like that.
2019-01-01 15:57:50 +02:00
Slavi Pantaleev e604a7bd43 Fix error message inaccuracy 2019-01-01 15:25:52 +02:00
Slavi Pantaleev 4c2e1a0588 Make SQLite database import work with server files, not local
This is a simplification and a way to make it consistent with
how we do Postgres imports (see 6d89319822), using
files coming from the server, not from the local machine.
2019-01-01 15:21:52 +02:00
Slavi Pantaleev f153c70a60 Reorganize some files 2019-01-01 14:47:22 +02:00
Slavi Pantaleev 6d89319822 Add support for importing an existing Postgres database 2019-01-01 14:45:37 +02:00
Slavi Pantaleev f472c1b9e5 Ensure psql returns a failure exit code when it fails
Until now, if the .sql file contained invalid data, psql would
choke on it, but still return an exit code of 0.
This is very misleading.

We need to pass `-v ON_ERROR_STOP=1` to make it exit
with a proper error exit code when failures happen.
2019-01-01 14:05:11 +02:00
Slavi Pantaleev a7f791f8f9 Make Postgres version detection logic reusable to ease maintenance
We've had that logic in 2 places so far, leading to duplication
and a maintenance burden.

In the future, we'll also have an import-postgres feature,
which will also need Postgres version detection,
leading to more benefit from that logic being reusable.
2019-01-01 13:43:51 +02:00
Slavi Pantaleev c59a53551a Make well-known self-check not depend on Content-Type: application/json
Fixes #60 (Github issue)
2018-12-31 11:19:59 +02:00
Hardy Erlinger 2fc0f5f3cf Set MAILNAME env variable to FQDN hostname for matrix-mailer. 2018-12-30 21:50:59 +01:00
Slavi Pantaleev 87b5f0a4d4 Server non-scary page at matrix domain (take 2)
Fix for 12b65d8ccc.
2018-12-29 20:11:37 +02:00
Slavi Pantaleev f7aa362961 Make "obtain certificates" tasks have unique names
We always skip at least one of these tasks, depending on which
SSL retrieval method is enabled, so it could have been confusing why.
2018-12-24 09:39:27 +02:00
Slavi Pantaleev 4757c13a2e Do not install openssl if not necessary
Fix for d28bdb3258.

We were only supposed to install openssl when the self-signed
SSL certificate retrieval method is used, not always.
2018-12-24 09:38:00 +02:00
Slavi Pantaleev 12b65d8ccc Serve a non-scary page at the matrix domain
Fixes #18 (Github issue).

It would probably be better if we serve our own page,
as the Matrix one says:

"To use this server you'll need a Matrix client", which
is true, but we install Riot by default and it'd be better if we mention
that instead.
2018-12-23 19:45:03 +02:00
Slavi Pantaleev b9b5674b8a Lowercase host_specific_hostname_identity to prevent troubles
If uppercase is used, certain tools (like certbot) would cause trouble.
They would retrieve a certificate for the lowercased domain name,
but we'd try to use it from an uppercase-named directory, which will
fail.

Besides certbot, we may experience other trouble too.
(it hasn't been investigated how far the breakage goes).

To fix it all, we lowercase `host_specific_hostname_identity` by default,
which takes care of the general use-case (people only setting that
and relying on us to build the other domain names - `hostname_matrix`
and `hostname_riot`).

For others, who decide to override these other variables directly
(and who may work around us and introduce uppercase there directly),
we also have the sanity-check tool warn if uppercase is detected
in any of the final domains.
2018-12-23 19:25:57 +02:00
Slavi Pantaleev fe9b9773c0 Move setup sanity checks to a central place 2018-12-23 19:15:23 +02:00
Slavi Pantaleev 91567fc41d Remove debug 2018-12-23 17:26:39 +02:00
Slavi Pantaleev e033eb443b Do not set up nginx-proxy auto-restart if not using Let's Encrypt
Fix for d28bdb3258
2018-12-23 15:38:33 +02:00
Slavi Pantaleev 268e9b5b66 Escape homeserver variables to prevent rare breakage
Strings containing double-quotes would cause YAML syntax breakage.
Not anymore.

Closes #48 and #49.
2018-12-23 15:20:01 +02:00
Slavi Pantaleev 727dcda7d7 Organize Synapse setup files better 2018-12-23 14:26:40 +02:00
Slavi Pantaleev d28bdb3258 Add support for 2 more SSL certificate retrieval methods
Adds support for managing certificates manually and for
having the playbook generate self-signed certificates for you.

With this, Let's Encrypt usage is no longer required.

Fixes Github issue #50.
2018-12-23 11:00:12 +02:00
Slavi Pantaleev bfcba5256e Upgrade mxisd (1.2.1 -> 1.2.2) 2018-12-23 08:49:21 +02:00
Slavi Pantaleev 57ee350420 Raise default timeout value for matrix-corporal
This is in line with what the recommendation is for matrix-corporal.

A value higher than 30 seconds is required to satisfy Riot
(and other clients') default long-polling behavior.
2018-12-21 12:19:16 +02:00
Slavi Pantaleev cc9d3aef59 Upgrade matrix-corporal (1.2.1 -> 1.2.2) 2018-12-21 11:55:35 +02:00
Slavi Pantaleev 6124effbe1 Register shared-secret-auth password provider first
For people using multiple password providers, it makes sense
to have the fastest one (which doesn't make network requests) be first.
2018-12-21 10:16:36 +02:00
Slavi Pantaleev 40626ff8df Upgrade Synapse (0.33.9/Python 2 -> 0.33.4/Python 3) 2018-12-21 10:15:58 +02:00
Slavi Pantaleev 9f163b2bf5 Do not disable SELinux on RedHat systems
It looks like SELinux can be left running without any (so far) negative
effects on our Matrix services.

There's no need to use `:z` or `:Z` options when mounting volumes either.
This means that files we create are labeled with a default context
(which may not be ideal if we only want them used from containers),
but it's compatible and doesn't cause issues.

Relabelling files is probably something we wish to stay away from,
especially for things like the media store, which contains lots of
files and is possibly on a fuse-mounted (S3/goofys) filesystem.
2018-12-20 15:30:43 +02:00
Slavi Pantaleev b9e5ad6c66 Upgrade Docker images for various components 2018-12-20 15:10:53 +02:00
Slavi Pantaleev 97280c7cc1 Change Goofys Docker image (clodproto/goofys -> ewoutp/goofys)
The new image is built in a much better way (2-stage build)
and is 10x smaller.

In terms of Goofys version recency, it's about the same..
Both images (and others alike) seem to not use version tags,
but rather some `:latest` (master), with ewoutp/goofys being a bit
more recent than clodproto/goofys.

Not using version tags is good (in this case),
because the last Goofys release seems to be from about a year ago
and there had been a bunch of bugfixes afterwards.
2018-12-20 14:30:24 +02:00
Slavi Pantaleev bfcceb1e82 Make it safer to override matrix_synapse_media_store_path
This is described in Github issue #58.

Until now, we had the variable, but if you redefined it, you'd run
into multiple problems:

- we actually always mounted some "storage" directory to the Synapse
container. So if your media store is not there, you're out of luck

- homeserver.yaml always hardcoded the path to the media store,
as a directory called "media-store" inside the storage directory.

Relocating to outside the storage directory was out of the question.

Moreover, even if you had simply renamed the media store directory
(e.g. "media-store" -> "media_store"), it would have also caused trouble.

With this patch, we mount the media store's parent to the Synapse container.
This way, we don't care where the media store is (inside storage or
not). We also don't assume (anymore) that the final part of the path
is called "media-store" -- anything can be used.

The "storage" directory and variable (`matrix_synapse_storage_path`)
still remain for compatibility purposes. People who were previously
overriding `matrix_synapse_storage_path` can continue doing so
and their media store will be at the same place.

The playbook no longer explicitly creates the `matrix_synapse_storage_path` directory
though. It's not necessary. If the media store is specified to be within it, it will
get created when the media store directory is created by the playbook.
2018-12-20 13:39:01 +02:00
Slavi Pantaleev e693b12d28 Relocate fail check to a better place 2018-12-20 13:00:07 +02:00
Slavi Pantaleev 60c7af93fe
Do not restart matrix-nginx-proxy during media store import
Previously, it was more necessary to have it
(because we had a dependency between matrix-synapse and matrix-nginx-proxy)..
But nowadays, it can be removed without negative side effects.

Restarting matrix-nginx-proxy is especially bad when the proxy is not installed at all.
2018-12-14 16:34:23 +09:00
Slavi Pantaleev 59afa841cb Add unintentionally removed quote 2018-12-13 14:44:27 +09:00
Slavi Pantaleev bf8023057a Fix SQLite importing failure
Fixes a problem where importing would lead to this error:
    Cannot link to /matrix-postgres, as it does not belond to the default network.
2018-12-13 14:40:30 +09:00
Slavi Pantaleev b11a4c07a8 Upgrade mxisd (1.2.0 -> 1.2.1) 2018-12-13 10:18:56 +09:00
Slavi Pantaleev fcf43eeacc Fix "missing translation" warning on riot-web homepage
Small bugfix related to #55
2018-12-12 10:48:30 +09:00
anadahz 57bfb970a9 Enable support for custom HTML in riot-web homepage
* Add default template file for homepage HTML
* Add default riot-web config options for homepage
2018-12-11 13:48:54 +00:00
Slavi Pantaleev cb874da1f7
Merge pull request #52 from aaronraimist/utf8-encoding
Add utf8 encoding option to log config
2018-12-05 10:57:23 +09:00
haslersn 22523c0e42 Support configuring mxisd's identity stores (two of them)
mxisd supports several identity stores. Add support to configure two of them:

* synapseSql (storing identities directly in Synapse's database)
* LDAP

This removed the need to copy `mxisd.yaml.j2` to the inventory in case one wants
to use LDAP as identity store. Note that the previous solution (copying
`mxisd.yaml.j2` was poor because of two reasons:

* The copy remains outdated in case the original is updated in future versions
  of this repo.
* The role's configuration should be in one place (configured only through role
  variables) instead of in multiple.

Configuring more identity stores through role variables can be supported in the
future.
2018-12-04 17:19:49 +01:00
Aaron Raimist 9028e3714e
Add utf8 encoding option to log config 2018-12-04 09:34:32 -06:00
Hugues Morisset f8987cb775 Set default mautrix-telegram image to 0.4.0 2018-11-30 19:32:29 +01:00
Slavi Pantaleev 9dad4c7c2d Fix /.well-known/matrix/client for CORS
This is provoked by Github issue #46.

No client had made use of the well-known mechanism
so far, so the set up performed by this playbook was not tested
and turned out to be a little deficient.

Even though /.well-known/matrix/client is usually requested with a
simple request (no preflight), it's still considered cross-origin
and [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)
applies. Thus, the file always needs to be served with the appropriate
`Access-Control-Allow-Origin` header.

Github issue #46 attempts to fix it at the "reverse-proxying" layer,
which may work, but would need to be done for every server.
It's better if it's done "upstream", so that all reverse-proxy
configurations can benefit.
2018-11-29 09:13:25 +02:00
Thomas vO bb849bd34f Merge branch 'master' of https://github.com/spantaleev/matrix-docker-ansible-deploy into new-cmds 2018-11-28 11:03:47 +01:00
Thomas vO caba16ea0d add script + doc to remove everything 2018-11-28 11:02:51 +01:00
Thomas vO 2bdc35de63 add script + doc to change a user to admin 2018-11-28 11:02:15 +01:00
Slavi Pantaleev 9f212adc1d Rename variable (matrix_synapse_ext_password_provider_ldap -> matrix_synapse_ext_password_provider_ldap_enabled)
This makes it more consistent with other usage throughout the playbook.
2018-11-28 11:19:19 +02:00
Thomas vO bbf8928831 fix template and vars for ldap auth, add setup 2018-11-28 09:04:09 +01:00
Thomas vO 9cf9a1ec54 [auth-ldap] add template + vars for ldap auth 2018-11-27 16:40:22 +01:00
Slavi Pantaleev 18e1dda4c8 Revert "Update riot-web (0.17.6 -> 0.17.7)"
As Github issue #42 says, 0.17.7 is not available.
We shouldn't have tried to use it.

This reverts commit 529b5b222d.
2018-11-27 16:40:26 +02:00
Slavi Pantaleev 529b5b222d Update riot-web (0.17.6 -> 0.17.7) 2018-11-25 09:57:30 +02:00
Slavi Pantaleev 5b70ec67a4 Add support for controlling Synapse's autocreate_auto_join_rooms 2018-11-23 11:16:40 +02:00
Slavi Pantaleev de91293d0d Update homeserver.yaml with new options and comments from upstream 2018-11-23 11:16:00 +02:00
Slavi Pantaleev a454feb5df Render trusted_third_party_id_servers more prettily 2018-11-23 11:07:22 +02:00
Slavi Pantaleev bc15db3316 Fix a few minor things for consistency
Trying to:

- stay closer to naming in Synapse (autojoin -> auto_join)

- not create new variable namespaces (`matrix_homeserver_`),
when existing ones (`matrix_synapse_`) are more suitable

- allow `null` (`~`) values for `matrix_riot_web_welcome_user_id`

- render things like `auto_join_rooms` in `homeserver.yaml` more prettily

- fix breakage in `config.json` where `matrix_riot_web_roomdir_servers`
was rendered as YAML and not as JSON

- simplify code (especially in riot-web's `config.json`), which used
`if` statements that could have been omitted

- avoid changing comments in `homeserver.yaml` which are not ours,
so that we can keep closer to the configuration file generated by upstream
2018-11-23 11:00:08 +02:00
anadahz 4febb117f4 Merge remote-tracking branch 'upstream/master' into fix/add/config 2018-11-23 00:58:32 +00:00
anadahz 792bed3f5a Fix add/config based on comments by @spantaleev 2018-11-23 00:42:54 +00:00
Aaron Raimist d13aa7316e
Remove riot.im from list of trusted identity servers
This brings the list in line with the new default as of https://github.com/matrix-org/synapse/pull/4207
2018-11-20 20:15:26 -06:00
Slavi Pantaleev 0e02dbd569 Update dependencies 2018-11-20 09:02:27 +02:00
anadahz 3cb3f17a90 Add more configured options for riot-web and homeserver. 2018-11-18 02:00:08 +00:00
Slavi Pantaleev d3c9be2cdf
Merge pull request #37 from aaronraimist/fix-well-known-self-check
Fix well known self check
2018-11-17 10:04:14 +01:00
Aaron Raimist 5a2069fa63
Fix well known self check 2018-11-16 16:12:13 -06:00
Aaron Raimist ddec99b899
Allow Synapse log levels to be configured (#23) 2018-11-14 13:39:52 -06:00
Slavi Pantaleev 12c4591a58 Update mxisd 2018-11-12 08:32:13 +02:00
Slavi Pantaleev 19257677c5 Update dependencies 2018-11-10 10:41:10 +02:00
Slavi Pantaleev f88b0ca33f
Merge branch 'master' into riot-web-config-json 2018-11-03 12:00:48 +02:00
Aaron Raimist ef2e330d22
Allow a few parts of Riot config.json to be configured (#24) 2018-11-02 20:14:03 -05:00
Aaron Raimist ebab95c9ec
Add new variable matrix_nginx_proxy_ssl_protocols 2018-11-02 18:20:05 -05:00
Aaron Raimist 3254a4d161
Disable TLS 1.0 and enable TLS 1.3 2018-11-01 22:14:09 -05:00
Slavi Pantaleev 9e8f216b9b Upgrade Synapse (v0.33.7 -> v0.33.8) 2018-11-01 12:50:33 +02:00
Slavi Pantaleev 4c0970d31e Make postgres_start_wait_time (affecting upgrades) configurable
This also raises the default from 5 seconds to 15 seconds.
2018-11-01 10:11:08 +02:00
Slavi Pantaleev e417ac4922 Add support for Postgres 11 2018-11-01 09:48:56 +02:00
Slavi Pantaleev a0a4ee526e Fix bug in Postgres dump/import causing upgrades to fail
Regression since 3fd6fd647f
2018-11-01 09:48:56 +02:00
Slavi Pantaleev 009bb1b776 Add support for configuring Postgres auto-upgrade-backup path 2018-11-01 09:48:56 +02:00
Slavi Pantaleev ac8d5a839d
Merge pull request #21 from Cadair/flags
Allow specifying which parts of the role run from playbooks
2018-11-01 08:58:42 +02:00
Slavi Pantaleev fb5115a544 Rename playbook variables so they are consistently prefixed
Pretty much all variables live in their own `matrix_<whatever>`
prefix now and are grouped closer together in the default
variables file (`roles/matrix-server/defaults/main.yml`).
2018-11-01 08:46:47 +02:00
Aaron Raimist 1955aac4bd
Update riot-web (0.17.0 -> 0.17.3) 2018-10-31 11:47:17 -05:00
Stuart Mumford 67e2bf285d
A simple idea to allow playbooks to control which bit of the role to run 2018-10-30 20:49:39 +00:00
Slavi Pantaleev 2186031be2 Remove unnecessary code causing troubles on Debian-based systems
It should be `/bin/mkdir` and `/bin/chown` on Ubuntu 18.04 for example.
Still, it doesn't seem like we need to create and chown these
directories at all, since the playbook takes care of creating them
and setting appropriate permission by itself.
2018-10-29 20:47:56 +02:00
Stuart Mumford 8498c4c5de
comment out all the things 2018-10-28 15:16:21 +00:00
Slavi Pantaleev cf0a5b3d2e
Merge pull request #20 from izissise/mautrix-whatsapp
Mautrix whatsapp
2018-10-26 19:39:14 +03:00
Slavi Pantaleev 95a6519876 Fix yaml syntax breakage
Regression since 67a445a74a
2018-10-25 18:15:56 +03:00
Slavi Pantaleev 67a445a74a Add support for controlling Matrix federation 2018-10-25 18:02:04 +03:00
Hugues Morisset b3fcc641c4 Add documentation and fix templates 2018-10-24 18:23:39 +02:00
Hugues Morisset 7a94fc0e24 Add mautrix-whatsapp 2018-10-24 17:09:23 +02:00
Slavi Pantaleev 3ecb16bbef Use disable_guests=true for Riot 2018-10-24 13:59:06 +03:00
Hugues Morisset 83a17f8439 Expose mautrix-telegram public endpoint through nginx
It used to allow user to logging with their own account to the bot
see: https://github.com/tulir/mautrix-telegram/wiki/Authentication#replacing-telegram-accounts-matrix-puppet-with-matrix-account
for more informations
2018-10-21 23:20:37 +02:00
Slavi Pantaleev c7188e06f9 Relocate some playbook task files to make it easier to navigate 2018-10-21 13:14:47 +03:00
Slavi Pantaleev d0c2ef10e4 Add self-check command 2018-10-21 12:58:25 +03:00
anadahz 2313907faa Add missing tag to task
The setup-all should include the setup_well_known task as well
2018-10-19 15:07:31 +01:00
Slavi Pantaleev a0320346e1 Upgrade Synapse (v0.33.7rc2 -> 0.33.7) 2018-10-18 18:15:07 +03:00
Slavi Pantaleev 9b3c882483 Update riot-web (0.16.4 -> 0.17.0) 2018-10-18 15:13:13 +03:00
Slavi Pantaleev 6cc528ba5a Upgrade Synapse (v0.33.5.1 -> v0.33.7rc2)
We skipped v0.33.6 because of matrix-org/synapse#4014,
but v0.33.7rc2 fixed the problem.
2018-10-18 14:49:21 +03:00
Slavi Pantaleev 36658addcd Work around buggy docker_network sometimes failing to work
If a network like `matrix-whatever` already exists for some reason,
the `docker_network` module would not create our `matrix` network.
Working around it by avoiding `docker_network` and doing it manually.

Fixes Github issue #12
2018-10-15 07:49:44 +03:00
Slavi Pantaleev 17ea05683b Fix yum/apt module invocation
The old way is deprecated and would stop working
after Ansible 2.11.
2018-10-12 09:00:53 +03:00
Slavi Pantaleev a1c1ec9b7f Update dependencies 2018-10-08 08:24:20 +03:00
Slavi Pantaleev 2185177957 Remove lt-cred-mech Coturn option
Coturn reports it as an option that conflicts with `use-auth-secret`.
Some reasoning is here: fa523e8d09
2018-10-08 08:22:40 +03:00
Slavi Pantaleev cc3e34b128 Fix to_yaml silliness 2018-10-05 10:59:02 +03:00
Slavi Pantaleev b49f4531e8 Make user presence-status tracking configurable 2018-10-05 10:35:16 +03:00
Slavi Pantaleev 7350842d9b Pull in homeserver.yaml template updates 2018-09-27 10:43:31 +03:00