Commit graph

415 commits

Author SHA1 Message Date
Cody Neiman c15d5a58a9
Make mx-puppet-groupme tokens unique 2021-02-14 13:37:12 -05:00
Slavi Pantaleev 5cfeae806b Merge branch 'master' into synapse-workers 2021-02-14 13:00:57 +02:00
Slavi Pantaleev 894679750e
Merge pull request #862 from s-thom/nginx-additional
Add additional domains for Let's Encrypt certificates to be obtained
2021-02-14 11:05:25 +02:00
Cody Neiman b900a4a3ba
Add groupme postgres 2021-02-13 00:50:00 -05:00
Cody Neiman 5a70a56ff0
Initial implementation 2021-02-12 23:13:30 -05:00
Slavi Pantaleev 2b47258c6c Do not auto-expose metrics on matrix.DOMAIN/_synapse/metrics
.. and other documentation improvements.
2021-02-12 13:47:53 +02:00
Slavi Pantaleev c8ab200cb1 Break dependency between matrix-prometheus and (matrix-prometheus-node-exporter, matrix-synapse) 2021-02-12 11:59:24 +02:00
Slavi Pantaleev 18e31526a8 Rename some variables 2021-02-12 11:59:24 +02:00
Peetz0r 989100b1c1 Grafana nginx proxy config 2021-02-10 22:54:14 +01:00
Peetz0r eb5aa93e8a Grafana
Also includes the dashboards for Synapse and for Node Exporter.

Again has only been tested on debian amd64 so far, but the grafana docker image is available for arm64 and arm32. Nice.
2021-02-10 22:54:14 +01:00
Peetz0r e525970b39 Prometheus Node Exporter
Basic system stats, to show stuff the synapse metrics
can't show such as resource usage by bridges, etc

Seems to work fine as well.

This too has only been tested on debian amd64 so far
2021-02-10 22:54:14 +01:00
Peetz0r 13ef9e85cf Prometheus
Initial attempt. Seems to work fine.

Only tested on debian amd64 so far
2021-02-10 22:54:14 +01:00
Stuart Thomson 064b2e533c Add variable for extra domains to get LE certs for
I felt that adding another variable was probably going to be the easiest way to do this. I may end up adding another variable to enable this feature, for consistency with some of the other things.
2021-02-06 20:02:39 +13:00
Béla Becker 38bf1eda70 Etherpad Jitsi integration 2021-01-26 05:04:47 +01:00
Béla Becker 4b451ff782 Etherpad role 2021-01-26 05:04:47 +01:00
Slavi Pantaleev d3ecc6f017 Fix bridges failing to upload media when Synapse workers are enabled 2021-01-25 13:55:08 +02:00
Slavi Pantaleev 4d62a75f6f Get matrix-corporal to play nicely with a Synapse worker setup
We do this by creating one more layer of indirection.

First we reach some generic vhost handling matrix.DOMAIN.
A bunch of override rules are added there (capturing traffic to send to
ma1sd, etc). nginx-status and similar generic things also live there.

We then proxy to the homeserver on some other vhost (only Synapse being
available right now, but repointing this to Dendrite or other will be
possible in the future).
Then that homeserver-specific vhost does its thing to proxy to the
homeserver. It may or may not use workers, etc.

Without matrix-corporal, the flow is now:
1. matrix.DOMAIN (matrix-nginx-proxy/matrix-domain.conf)
2. matrix-nginx-proxy/matrix-synapse.conf
3. matrix-synapse

With matrix-corporal enabled, it becomes:
1. matrix.DOMAIN (matrix-nginx-proxy/matrix-domain.conf)
2. matrix-corporal
3. matrix-nginx-proxy/matrix-synapse.conf
4. matrix-synapse

(matrix-corporal gets injected at step 2).
2021-01-25 09:46:41 +02:00
Slavi Pantaleev 63301b0ef1 Improvements around Synapse worker/metrics ports exposure
There was a `matrix_nginx_proxy_enabled|default(False)` check, but:
- it didn't seem to work reliably for some reason (hmm)
- referring to a `matrix_nginx_proxy_*` variable from within the
  `matrix-synapse` role is not ideal
- exposing always happened on `127.0.0.1`, which may not be good enough
  for some rarer setups (where the own webserver is external to the host)
2021-01-25 08:25:43 +02:00
Slavi Pantaleev 5ca68210cd Do not handle /_matrix/federation on client-server port, nor /_matrix/client stuff on federation port
I guess it didn't hurt to do it until now, but it's not great serving
federation APIs on the client-server API port, etc.

matrix-corporal doesn't work yet (still something to be solved in the
future), but its firewalling operations will also be sabotaged
by Client-Server APIs being served on the federation port (it's a way to get around its firewalling).
2021-01-24 22:22:57 +02:00
Marcel Partap 183adec3d8 Merge remote-tracking branch 'origin/master' into synapse-workers 2021-01-23 15:04:11 +01:00
Panagiotis Georgiadis f10e3fef0d
Merge branch 'master' into irc 2021-01-22 20:30:24 +00:00
Panagiotis Georgiadis e502ee33da
Selfbuild appservice-irc bridge 2021-01-22 21:28:53 +01:00
Slavi Pantaleev f9c1d62435 Fix Postgres database (-alpine) failing to start on ARM32 2021-01-22 13:52:55 +02:00
Slavi Pantaleev 95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
Slavi Pantaleev 024a23ed17 Upgrade mautrix-facebook to the new Postgres-only version
I had intentionally held it back in 39ea3496a4
until:
- it received more testing (there were a few bugs during the
migration, but now it seems OK)
- this migration guide was written
2021-01-20 10:12:51 +02:00
Slavi Pantaleev 28d86e3aaa Initial work on support for matrix-corporal v2 2021-01-16 23:47:14 +02:00
Panagiotis Georgiadis a66a604e53
Selfbuild appservice-slack bridge 2021-01-14 01:29:11 +01:00
Slavi Pantaleev 48b6487d41 Use ready-made image for mautrix-telegram on arm64
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/793
2021-01-13 22:40:47 +02:00
Slavi Pantaleev 105354a052
Remove useless comment 2021-01-13 17:56:10 +02:00
Panagiotis Georgiadis fd540d8592
Enable Telegram self-building 2021-01-13 16:51:45 +01:00
Marcel Partap cd8100544b Merge remote-tracking branch 'origin/master' into synapse-workers
Sync with upstream
2021-01-08 20:58:50 +01:00
teutat3s a2f6adbeaf
Fix appservice-slack default db: nedb 2021-01-06 22:00:49 +01:00
Slavi Pantaleev 6b1e25d843 Do not create matrix_bridge_sms Postgres database
That bridge only supports its own file-based database,
so preparing a Postgres database for it is pointless.
2021-01-03 10:21:59 +02:00
Slavi Pantaleev df8d9cfd34 Remove some TODOs
The answer to these is: it's good to have them in both places.
The role defines the obvious things it depends on (not knowing
what setup it will find itself into), and then
`group_vars/matrix_servers` "extends" it based on everything else it
knows (the homeserver being Synapse, whether or not the internal
Postgres server is being used, etc.)
2021-01-03 07:46:55 +02:00
Sabine Laszakovits ffb837d4bc made the bridge use the default postgres db 2021-01-02 00:39:11 +01:00
Sabine Laszakovits a06c58c753 Merge branch 'master' into signal 2021-01-01 21:05:00 +01:00
David Gnedt befffa926b Fix concatenation of additional databases
Otherwise the postgres upgrade fails with the following error:

Unexpected templating type error occurred on ({{
  [matrix_postgres_connection_username]
  +
  matrix_postgres_additional_databases|map(attribute='username')
}}
): can only concatenate list (not "generator") to list
2020-12-23 19:24:45 +01:00
Slavi Pantaleev ea804f2f9f Do not use underscore in salts passed to sha512
Some Ansible installations choke on it, it seems.
Similar to 9f00970c90
2020-12-23 12:59:48 +02:00
Slavi Pantaleev 9f00970c90 Do not use salts longer than 16 characters
We've hit this problem before as well. Certain Ansible installations
choke on it.
2020-12-23 12:31:52 +02:00
Slavi Pantaleev f19b29846d
Merge pull request #740 from jdreichmann/postgres-per-default
postgres: create databases for all services
2020-12-23 11:00:41 +02:00
Slavi Pantaleev ad1425eee4 Add pgloader self-building support (for ARM) 2020-12-23 09:08:54 +02:00
Slavi Pantaleev 715bdf2c64 Add support for automatic (nedb -> Postgres) migration to mx-appservice-irc 2020-12-22 19:32:43 +02:00
Slavi Pantaleev 815a80c290 Fix alignment issues 2020-12-22 17:16:56 +02:00
Slavi Pantaleev 15f4cc924d Rename variables (_database_db_name -> _database_name) 2020-12-22 17:10:02 +02:00
Slavi Pantaleev 69cc2145d2 Add support for automatic (Postgres -> SQLite) migration to mx-puppet-steam 2020-12-22 16:51:59 +02:00
Slavi Pantaleev ed159cc742 Move matrix_architecture to matrix-base
We were only defining this in `group_vars/matrix_servers`, which is
inconsistent with how we normally do things.
2020-12-17 11:33:18 +02:00
Slavi Pantaleev a197968b7f Make matrix-registration use Postgres by default
Now that 0.7.2 is out, the Docker image supports Postgres
and we can do the (SQLite -> Postgres) migration.

I've also found out that we needed to fix up the `tokens.ex_date` column
data type a bit to prevent matrix-registration from raising exceptions
when comparing `datetime.now()` with `ex_date` coming from the database.

Example:

> File "/usr/local/lib/python3.8/site-packages/matrix_registration/tokens.py", line 58, in valid
> expired = self.ex_date < datetime.now()
> TypeError: can't compare offset-naive and offset-aware datetimes
2020-12-15 23:19:56 +02:00
Slavi Pantaleev dd797ba6a7 Fix Postgres database importing/upgrading conflicts
We were running into conflicts, because having initialized
the roles (users) and databases, trying to import leads to
errors (role XXX already exists, etc.).

We were previously ignoring the Synapse database (`homeserver`)
when upgrading/importing, because that one gets created by default
whenever the container starts.

For our additional databases, it's a similar situation now.
It's not created by default as soon as Postgres starts with an empty
database, but rather we create it as part of running the playbook.

So we either need to skip those role/database creation statements
while upgrading/importing, or to avoid creating the additional database
and rely on the import for that. I've gone for the former, because
it's already similar to what we were doing and it's simpler
(it lets `setup_postgres.yml` be the same in all scenarios).
2020-12-14 22:28:20 +02:00
Slavi Pantaleev 2a502db239 Add (SQLite + Postgres) support and automatic migration to matrix-dimension 2020-12-14 21:01:47 +02:00
Slavi Pantaleev 8d74593878 Prepare matrix-registation for (SQLite + Postgres) support
Auto-migration and everything seems to work. It's just that
matrix-registration cannot load the Python modules required
for talking to a Postgres database.

Tracked here: https://github.com/ZerataX/matrix-registration/issues/44

Until this gets fixed, we'll continue default to 'sqlite'.
2020-12-14 18:58:37 +02:00
Slavi Pantaleev 3ab7dd1abe Potentially add matrix-postgres.service as required for bridges, etc. 2020-12-14 17:22:31 +02:00
Slavi Pantaleev af3ea67bba Add (SQLite + Postgres) support and automatic migration to matrix-ma1sd 2020-12-14 17:16:25 +02:00
Slavi Pantaleev e3a0c9adda Add (Postgres + SQLite) support to matrix-reminder-bot
This has been tested and appears to work.
2020-12-14 15:02:11 +02:00
Slavi Pantaleev b9a04a7f95 Rename some remaining matrix_*_postgres_* vars back to matrix_*_database_*
Looks like there are some that I missed in 087dbe4ddc
2020-12-14 14:42:18 +02:00
Slavi Pantaleev 087dbe4ddc Rename matrix_*_postgres_* back to matrix_*_database_*
I was thinking that it makes sense to be more specific,
and using `_postgres_` also separated these variables
from the `_database_` variables that ended up in bridge configuration.

However, @jdreichmann makes a good point
(https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/740#discussion_r542281102)
that we don't need to be so specific and can allow for other engines (like MySQL) to use these variables.
2020-12-14 13:02:47 +02:00
Slavi Pantaleev 2848322461 Remove some remaining _database_password usage in favor of _postgres_pasword
The only one that remains is `matrix_synapse_database_password`, but
that's something old and should be dealt with separately in the future
(unless it remains as it is).
2020-12-14 12:19:35 +02:00
Slavi Pantaleev 6a2dd8088c Fix typo (matrix_telegram_facebook_postgres_hostname -> matrix_mautrix_telegram_postgres_hostname)
Related to 9bf8ce878e.
2020-12-14 12:13:10 +02:00
Slavi Pantaleev 05dd091133 Rename _database_hostname to _postgres_hostname
Should have been done during 2a99e84b5b, but I've missed it.

Because of this, 9bf8ce878e was also incomplete and would have caused
an error.
2020-12-14 12:11:41 +02:00
Slavi Pantaleev 2a99e84b5b Rename Postgres-related variables (matrix*database_* -> matrix*postgres_*)
`matrix_SERVICE_database_engine` remains as it is - the only one with a
`_database_` infix. Postgres-related ones move to `_postgres_`.
2020-12-14 11:19:18 +02:00
Slavi Pantaleev c0edacbefd Fix matrix_postgres_additional_databases syntax problems 2020-12-14 11:17:12 +02:00
Slavi Pantaleev d91aa5a060 Do not introduce sub-variables exposing implementation details 2020-12-14 10:52:07 +02:00
transcaffeine b217a2ed69
postgres: set default in playbook to postgres but fallback to sqlite 2020-12-14 07:54:24 +01:00
Slavi Pantaleev dd994995bc Simplify password for additional Postgres databases
Using the result of `password_hash` works for creating them,
but authentication seems to be failing with some tools like pgloader.

It's possible that we're not escaping things properly somewhere.
Ideally, it'd be nice to solve that. But the easier (and still
relatively safe/good) solution is to just turn that password hash
into a UUID that's safe for passing around without worrying about
escaping.
2020-12-14 01:22:23 +02:00
Slavi Pantaleev a374d309c8 Make appservice-discord support both SQLite and Postgres
People can toggle between them now. The playbook also defaults
to using SQLite if an external Postgres server is used.

Ideally, we'd be able to create databases/users in external Postgres
servers as well, but our initialization logic (and `docker run` command,
etc.) hardcode too many things right now.
2020-12-14 00:52:25 +02:00
Slavi Pantaleev 46a4034d3e Use "password" for additional Postgres databases, not "pass"
Being more explicit sounds better.
2020-12-14 00:43:03 +02:00
Slavi Pantaleev d251764c16 Fix syntax issues in matrix_postgres_additional_databases
Quotes are necessary around dictionary field names.

There was a missing `}` as well.
2020-12-13 22:37:04 +02:00
Slavi Pantaleev 0641106370 Allow username of additional Postgres databases to be different
We'll most likely use one that matches the database name, but
it's better to have it configurable.
2020-12-13 22:37:04 +02:00
transcaffeine d9f4914e0d
WIP: postgres: create databases for all services
If a service is enabled, a database for it is created in postgres with a uniqque password. The service can then use this database for data storage instead of relying on sqlite.
2020-12-10 18:26:22 +01:00
Slavi Pantaleev c07c927d9f Automatically enable openid listeners when ma1sd enabled
ma1sd requires the openid endpoints for certain functionality.
Example: 90b2b5301c/src/main/java/io/kamax/mxisd/auth/AccountManager.java (L67-L99)

If federation is disabled, we still need to expose these openid APIs on the
federation port.

Previously, we were doing similar magic for Dimension.
As per its documentation, when running unfederated, one is to enable
the openid listener as well. As per their recommendation, people
are advised to do enable it on the Client-Server API port
and use the `federationUrl` variable to override where the federation
port is (making federation requests go to the Client-Server API).

Because ma1sd always uses the federation port (unless you do some
DNS overwriting magic using its configuration -- which we'd rather not
do), it's better if we just default to putting the `openid` listener
where it belongs - on the federation port.

With this commit, we retain the "automatically enable openid APIs" thing
we've been doing for Dimension, but move it to the federation port instead.
We also now do the same thing when ma1sd is enabled.
2020-12-08 16:59:20 +02:00
Slavi Pantaleev a5ae7e9ef0 Add self-building support to matrix-corporal 2020-12-04 01:48:08 +02:00
Marcel Partap b73ac965ac Merge remote-tracking branch 'origin/master' into synapse-workers 2020-12-01 21:24:26 +01:00
Slavi Pantaleev 04da1bddf7 Update matrix-mautrix-facebook config a bit
This also disables presence if it's disabled for Synapse.
2020-12-01 11:55:18 +02:00
Slavi Pantaleev b354155d7c Make JVB websockets reverse-proxying work 2020-11-27 17:57:15 +02:00
Slavi Pantaleev d235335507 Do not mount /matrix/ssl into matrix-coturn when matrix_ssl_retrieval_method is 'none'
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/734

Similar to 12867e9f18, but for `matrix-coturn` (not
`matrix-nginx-proxy`).
2020-11-26 10:08:54 +02:00
anonym d4ef751052
Update matrix_servers 2020-11-26 01:30:08 +01:00
anonym 732f57cbd9
Note about the max length of the postgres password 2020-11-26 01:24:43 +01:00
Marcel Partap 4678c5d7bd Merge remote-tracking branch 'origin/master' into synapse-workers
Also, replace vague FIXME by a proper NOTE on the complete
story of the user_dir endpoints..
2020-11-11 21:26:08 +01:00
Slavi Pantaleev b5435db338 Fix syntax error in group_vars/matrix_servers 2020-11-11 00:22:09 +02:00
Slavi Pantaleev 4d12a6f8e9
Merge pull request #681 from scottcrossen/slc/ddclient
Dynamic DNS
2020-11-10 23:54:21 +02:00
Slavi Pantaleev de0987e806 Remove unnecessary variable definition
Related to #681 (Github Pull Request)
2020-11-10 23:51:38 +02:00
Slavi Pantaleev 1427286cec Integrate matrix-dynamic-dns with matrix-nginx-proxy without causing a dependency
We'd like the roles to be self-contained (as much as possible).

Thus, the `matrix-nginx-proxy` shouldn't reference any variables from
other roles. Instead, we rely on injection via
`group_vars/matrix_servers`.

Related to #681 (Github Pull Request)
2020-11-10 23:49:36 +02:00
Slavi Pantaleev 73d5faa1f9 Fix variable name typo 2020-11-10 23:30:38 +02:00
Scott Crossen 59bb6b2971 responded to reviewer comments 2020-11-09 13:32:58 -08:00
Scott Crossen e894befd87 Updates to reviewer comments 2020-11-07 17:53:13 -08:00
Slavi Pantaleev 56ebeb1ae2 Revert "Restore self-building on ARM for Synapse for now"
This reverts commit 7d211b71e2.

The multi-arch images are now available, as discussed in #699 (Github Issue).
2020-10-29 07:35:53 +02:00
Marcel Partap cce90b187a synapse workers: fix undefined variable cases when removing workers 2020-10-28 23:09:21 +01:00
Slavi Pantaleev 9a46647010 Make https://matrix.DOMAIN/ redirect to https://element.DOMAIN/
Fixes #696 (Github Issue)
2020-10-28 10:39:12 +02:00
Marcel Partap 2d1b9f2dbf synapse workers: reworkings + get endpoints from upstream docs via awk
(yes, a bit awkward and brittle… xD)
2020-10-28 07:13:19 +01:00
Slavi Pantaleev 5a7d265997 Fix Synapse Admin not working due to unexposed Synapse Admin APIs
Regression since 63a49bb2dc.

Related to #685 (Github Issue).

We now automatically expose the APIs when Synapse Admin is enabled.
2020-10-27 20:33:37 +02:00
Slavi Pantaleev 7d211b71e2 Restore self-building on ARM for Synapse for now
While v1.22.0 supposedly has multi-arch Docker images
(thanks to https://github.com/matrix-org/synapse/pull/7921),
I can't them on Docker Hub yet, so I'm backing out of this change
for now and letting people fall back to self-building there.
2020-10-27 14:32:32 +02:00
Slavi Pantaleev 70f0b97a0a Upgrade Synapse (v1.21.2 -> v1.22.0) 2020-10-27 14:24:02 +02:00
Sabine Laszakovits c36e1355cf cleanup 2020-10-25 20:46:57 +01:00
Sabine Laszakovits 8ebc39dea0 fixed dependencies of bridge service (not ideal, but correct) 2020-10-25 20:46:32 +01:00
Sabine Laszakovits 9b890e9ced moved some settings from role to group_vars 2020-10-25 20:45:24 +01:00
Sabine 7a54e11090 started moving settings to group_vars 2020-10-24 12:15:03 +02:00
Marcel Partap 93a8ea7e4a Merge remote-tracking branch 'master' into feature/add-worker-support 2020-10-11 20:59:05 +02:00
Slavi Pantaleev 263727095d
Merge pull request #657 from cnvandijk/feature-client-well-known
Client well known compatibility
2020-09-28 09:19:25 +03:00
Slavi Pantaleev 172bf24df7 Enable self-building of matrix-synapse-admin on non-amd64
Related to #658 (Github Issue).
2020-09-28 09:06:39 +03:00
Chris van Dijk f6b0f0a477 Rename matrix_riot_jitsi_preferredDomain and matrix_riot_e2ee_default to Element 2020-09-26 16:24:09 +00:00
Max Klenk 9a3d84b931
Merge branch 'master' into feature/add-worker-support 2020-09-10 13:57:11 +02:00
Max Klenk a25a429a52
add redis support 2020-09-10 13:39:00 +02:00
Slavi Pantaleev da38a7869f Add matrix-registration support 2020-09-01 13:46:05 +03:00
Slavi Pantaleev b117dc0cb7 Fix self-check certificate validation when not self-signed 2020-09-01 11:44:44 +03:00
Slavi Pantaleev 9952ec6c16 Upgrade Synapse (v1.18.0 -> v1.19.0) 2020-08-17 17:02:40 +03:00
Slavi Pantaleev 296bd6e211
Fix variable name typo 2020-07-22 12:04:49 +03:00
Slavi Pantaleev b2ae669566 Add synapse-admin support
Fixes #562 (Github Issue)
2020-07-22 08:10:26 +03:00
Slavi Pantaleev 78b1ef9a5f Add support for matrix-reminder-bot 2020-07-20 14:13:08 +03:00
Slavi Pantaleev c6ab1c6a90 Riot is now Element
Fixes #586 (Github Issue)
2020-07-17 11:31:20 +03:00
Hugues Morisset 57f498217a Add mx-puppet-steam 2020-07-01 21:01:00 +02:00
Hugues Morisset 42e7f5e9bc Add mx-puppet-discord 2020-07-01 13:31:31 +02:00
Slavi Pantaleev 9686a0fc5b Expose mx-puppet-twitter port when matrix-nginx-proxy disabled 2020-06-30 17:39:19 +03:00
Slavi Pantaleev 744667b270 Merge branch 'master' into mx-puppet-twitter 2020-06-30 17:37:19 +03:00
Slavi Pantaleev 31f9e7bbea
Merge pull request #551 from jdreichmann/feat-mx-puppet-instagram
Add mx-puppet-instagram
2020-06-30 17:33:37 +03:00
Tulir Asokan 13186a1ddc Add mx-puppet-twitter
Signed-off-by: Tulir Asokan <tulir@maunium.net>
2020-06-29 15:28:43 +03:00
Slavi Pantaleev 635f385971 Use pre-built arm64 image for ma1sd
ma1sd 2.4.0 announced experimental support for arm64.
We're making use of those arm64 images instead of self-building.
2020-06-29 14:53:23 +03:00
teutat3s 0162fe31d1
Re-enable ma1sd user directory search 2020-06-28 21:47:39 +02:00
jdreichmann 2073cd6cae
mx-puppet-instagram: change tokens appservice<>homeserver in order to not collide with mx-puppet-slack 2020-06-28 10:40:14 +02:00
Slavi Pantaleev 19b9a1b16c Expose mautrix-hangouts port if matrix-nginx-proxy is disabled 2020-06-28 09:01:48 +03:00
jdreichmann 0fea35cdd2
mx-puppet-instagram: add role 2020-06-27 15:23:21 +02:00
Slavi Pantaleev 10b3ceff72 Make Matrix federation port configurable
Fixes #523 (Github Issue).
2020-06-09 08:29:03 +03:00
Slavi Pantaleev 831c3f4e64 Fix certbot/cerbot image pulling for ARM32 and ARM64
Related to #529 (Github Issue).
2020-06-09 07:54:42 +03:00
Slavi Pantaleev ab32f6adf6 Add self-building support to matrix-mailer (exim-relay) 2020-06-08 09:52:34 +03:00
benkuly 1ffa4a326c matrix-sms-bridge:
added docs, added seperate service for database, pr comments
2020-05-23 13:14:24 +02:00
benkuly 2d75fca257 added matrix-sms-bridge 2020-05-22 13:50:59 +02:00
Slavi Pantaleev 07ff116266 Do not send User Directory search requests to ma1sd for now
We can undo this once https://github.com/ma1uta/ma1sd/issues/44 gets
resolved.
2020-05-19 12:57:19 +03:00
Slavi Pantaleev 308fc9c60a Fix webhooks service address when matrix-nginx-proxy disabled
Fixes #494 (Github Issue).
2020-05-10 09:52:55 +03:00
Slavi Pantaleev 8cdf3a24f9 Fix variable name typo 2020-04-23 22:13:50 +03:00
Rodrigo Belem 4149e03b69 Initial mx-puppet-slack bridge role 2020-04-23 09:06:08 -04:00
Slavi Pantaleev 272888d917
Merge pull request #462 from cnvandijk/feature-dimension-unfederated
Add support for running dimension in an unfederated environment
2020-04-22 22:47:14 +03:00
Chris van Dijk 3f4bc9b881 Move config supprt for unfederated dimension into group_vars 2020-04-22 19:23:56 +02:00
Horvath Gergely b5844d2395 shortening line based on @spantaleev 's comment 2020-04-19 13:31:36 +02:00
Horvath Gergely bd61598faf add experimental(?) architecture support for arm32 and arm64
the changes are necessary because certbot images are tagged, so docker does not recognize the necessary architecture
2020-04-18 22:55:19 +02:00
Slavi Pantaleev 515160f0d7 Fix STUN server URL for jitsi/web (turns -> stun)
Discussed in #446 (Github Pull Request).
2020-04-09 19:09:04 +03:00
Rodrigo Belem 64de103dff Add global var to control skype image self build 2020-04-09 11:03:06 -04:00
Slavi Pantaleev 2f601a4aee
Merge pull request #446 from skoal/master
Jitsi related firewall and port changes
2020-04-09 16:14:06 +03:00
Slavi Pantaleev e6ebfbf8cc Prevent conflict between Whatsapp and Skype bridge tokens
Fixes an error like this:

> synapse.config._base.ConfigError: Cannot reuse as_token across application services
2020-04-09 16:07:36 +03:00
skoal 0713570e2f
Update matrix_servers
turn tls port update
2020-04-09 14:40:32 +02:00
Slavi Pantaleev 7035af87d8 Add support for Jitsi discovery for Riot via /.well-known/matrix/client
This will not work yet, as no version of Riot currently supports it.
It's expected to land in riot-web v1.5.16 via matrix-org/matrix-react-sdk#4348.
2020-04-09 09:58:35 +03:00
Slavi Pantaleev 59d05ad72c
Merge pull request #441 from Jozian/skype
Initial mx-puppet-skype bridge role
2020-04-09 09:24:11 +03:00
Slavi Pantaleev 67a5ef97ba Fix missing "stun:" prefix for jitsi/web STUN server URLs
Hopefully fixes an error like this (which I haven't been able to
reproduce, but..):

> [modules/xmpp/strophe.util.js] <Object.i.Strophe.log>:  Strophe: Error: Failed to construct 'RTCPeerConnection': 'matrix.DOMAIN' is not one of the supported URL schemes 'stun', 'turn' or 'turns'.
2020-04-09 09:16:10 +03:00
Rodrigo Belem 1107207147 Initial mx-puppet-skype bridge role 2020-04-08 19:25:21 -04:00
Slavi Pantaleev 61fffa169c Use Jitsi STUN servers instead of Google ones
These are some new defaults that we switch to, to follow upstream
changes:

- https://github.com/jitsi/docker-jitsi-meet/commit/264df04
- 3121494d4b
2020-04-08 19:08:15 +03:00
Slavi Pantaleev 845f5f007b Make Synapse use ma1sd (if enabled) for threepid registration 2020-04-03 10:08:37 +03:00
Marcel Partap 874e2e1fc0 Rename variables (s/mxisd/ma1sd/) and adapt roles 2020-04-02 11:31:38 +02:00
Slavi Pantaleev e06ac41db1 Do not try to obtain jitsi.DOMAIN certificate if Jitsi is disabled 2020-03-24 16:21:26 +02:00
Slavi Pantaleev d605b219a2 Manage Jitsi configuration by ourselves for most components
We do this for 2 reasons:

- so we can control things which are not controllable using environment
variables (for example `stunServers` in jitsi/web, since we don't wish
to use the hardcoded Google STUN servers if our own Coturn is enabled)

- so playbook variable changes will properly rebuild the configuration.
When using Jitsi environment variables, the configuration is only built
once (the first time) and never rebuilt again. This is not the
consistent with the rest of the playbook and with how Ansible operates.
We're not perfect at it (yet), because we still let the Jitsi containers
generate some files on their own, but we are closer and it should be
good enough for most things.

Related to #415 (Github Pull Request).
2020-03-24 09:35:21 +02:00
Slavi Pantaleev cdd9ee1962 Add Jitsi support 2020-03-23 17:19:15 +02:00
Slavi Pantaleev 2b85fde103 Rename some variables for consistency 2020-03-15 10:15:27 +02:00
Slavi Pantaleev 8fe97abe7d Wire matrix_container_images_self_build to self_build variables via group_vars/matrix_servers
This keeps the roles cleaner and more independent of matrix-base,
which may be important for people building their own playbook
out of the individual roles and not using the matrix-base role.
2020-03-15 10:10:41 +02:00
Horvath Gergely 310aa685f9 refactor based on Slavi's requests 2020-03-08 00:24:00 +01:00
Horvath Gergely d53d63ab07 Merge branch 'master' of https://github.com/spantaleev/matrix-docker-ansible-deploy into raspberry-pi 2020-03-07 13:57:44 +01:00
Slavi Pantaleev 4a06e52660 Fix matrix-appservice-slack configuration when matrix-nginx-proxy is disabled 2020-03-03 17:48:51 +02:00
Slavi Pantaleev be6c048abe Remove useless matrix_appservice_webhooks_webhooks_port
In relation to #392 (Github Issue)
2020-03-03 12:38:26 +02:00
Horvath Gergely a096eafb45 add possibility to install synapse on raspberry pi 2020-02-17 21:48:48 +01:00
teutates 2c4eef72c3
fix slack bridge: salt too large, sha512_crypt requires <= 16 chars 2020-02-10 02:47:26 +01:00
joao-p-marques a0b313b3d1 fix invalid password salt on slack bridge 2020-02-06 00:25:26 +00:00
Zach Mertes e0defd4d4c
Fix invalid password salts for slack webhooks 2020-01-24 05:06:36 -05:00
Slavi Pantaleev 2c04384e8e Synchronize config with the one from Synapse 1.9.0
Related to #355.
2020-01-23 15:47:53 +02:00
Björn Marten 50bf8c8dd7 Add matrix-appservice-webhooks role. 2020-01-13 17:20:49 +01:00
Slavi Pantaleev bd38861179 Add support for automatic Double Puppeting for all Mautrix bridges 2020-01-12 20:28:36 +02:00
Aaron Raimist 2ea507e2ea
Don't make it Dimension specific 2019-12-09 22:23:56 -06:00
Aaron Raimist 79d1576648
Allow Synapse manhole to be enabled
Can you double check that the way I have this set only exposes it locally? It is important that the manhole is not available to the outside world since it is quite powerful and the password is hard coded.
2019-12-05 00:07:15 -06:00
Slavi Pantaleev 65da600426
Merge branch 'master' into master 2019-08-21 07:34:20 +03:00
Slavi Pantaleev c8a4d59a81
Merge pull request #251 from Munfred/master
Add mautrix-hangouts bridge role and documentation
2019-08-21 07:15:14 +03:00
Slavi Pantaleev e4b67fdb6b
Fix typo 2019-08-20 09:07:54 +03:00
microchipster 0585889d5a add hangouts bridge by copying facebook bridge and find-replacing 2019-08-06 05:27:40 +00:00
Slavi Pantaleev 4be35822dd Add Email2Matrix support 2019-08-05 13:09:49 +03:00
kingoftheconnors 177ec295b4 Fixed matrix-appservice-slack docker command problems 2019-07-27 14:25:13 -04:00
kingoftheconnors 49766c5dac Added Slack role 2019-07-26 21:37:21 -04:00
Slavi Pantaleev 8529efcd1c Make Discord bridge configuration playbook-managed
Well, `config.yaml` has been playbook-managed for a long time.
It's now extended to match the default sample config of the Discord
bridge.

With this patch, we also make `registration.yaml` playbook-managed,
which leads us to consistency with all other bridges.

Along with that, we introduce `./config` and `./data` separation,
like we do for the other bridges.
2019-06-26 10:35:00 +03:00
Slavi Pantaleev 782356d421 Use password_hash salts that obey passlib requirements
According to
https://passlib.readthedocs.io/en/stable/lib/passlib.hash.sha512_crypt.html:

> salt (str) – Optional salt string. If not specified, one will be autogenerated (this is recommended).
> If specified, it must be 0-16 characters, drawn from the regexp range [./0-9A-Za-z].

Until now, we were using invalid characters (like `-`). We were also
going over the requested length limit of 16 characters.

This is most likely what was causing `ValueError` exceptions for some people,
as reported in #209 (Github Issue).
Ansible's source code (`lib/ansible/utils/encrypt.py`) shows that Ansible tries
to use passlib if available and falls back to Python's `crypt` module if not.
For Mac, `crypt.crypt` doesn't seem to work, so Ansible always requires passlib.

Looks like crypt is forgiving when length or character requirements are
not obeyed. It would auto-trim a salt string to make it work, which means
that we could end up with the same hash if we call it with salts which aer only
different after their 16th character.

For these reasons (crypt autotriming and passlib downright complaining),
we're now using shorter and more diverse salts.
2019-06-26 09:37:02 +03:00
Thomas Kuehne 4797469383 Make WhatsApp bridge configuration playbook-managed
- following spantaleev transition of the telegram brigde
- adding a validate_config task
2019-06-24 00:16:04 +02:00
Slavi Pantaleev 174a6fcd1b Make IRC bridge configuration entirely managed by the playbook 2019-06-19 12:29:44 +03:00
Slavi Pantaleev deeb5a96d5 Disable IRC bridge presence if Synapse presence is disabled 2019-06-19 09:31:09 +03:00
Slavi Pantaleev 4e8543ce21 Make Telegram bridge configuration playbook-managed 2019-06-15 09:43:43 +03:00
Slavi Pantaleev 2902b53267 Minor fixes for consistency 2019-06-15 09:42:40 +03:00
Slavi Pantaleev 3956b300ed Disable riot-web's welcome bot
I've not found this welcome bot to work at all in my previous attempts.
It would simply not reply, even though federation works.

It seems like this is also a potential privacy issue, as per
https://gist.github.com/maxidorius/5736fd09c9194b7a6dc03b6b8d7220d0
2019-06-14 07:49:46 +03:00
Slavi Pantaleev 330648a3e0 Make Facebook bridge configuration playbook-managed
Related to #193, but for the Facebook bridge.
(other bridges can be changed to do the same later).

This patch makes the bridge configuration entirely managed by the
Ansible playbook. The bridge's `config.yaml` and `registration.yaml`
configuration files are regenerated every time the playbook runs.

This allows us to apply updates to those files and to avoid
people having to manage the configuration files manually on the server.

-------------------------------------------------------------

A deficiency of the current approach to dumping YAML configuration in
`config.yaml` is that we strip all comments from it.
Later on, when the bridge actually starts, it will load and redump
(this time with comments), which will make the `config.yaml` file
change.

Subsequent playbook runs will report "changed" for the
"Ensure mautrix-facebook config.yaml installed" task, which is a little
strange.

We might wish to improve this in the future, if possible.

Still, it's better to have a (usually) somewhat meaningless "changed"
task than to what we had -- never rebuilding the configuration.
2019-06-07 14:05:53 +03:00
Slavi Pantaleev 7379968a3c Fix Telegram bridge HTTP proxying when not using matrix-nginx-proxy
From what I see, this was never implemented to begin with.

Fixes #189 (Github Issue).
2019-05-26 20:50:52 +03:00
Slavi Pantaleev ab59cc50bd Add support for more flexible container port exposing
Fixes #171 (Github Issue).
2019-05-25 07:41:08 +09:00
Dan Arnfield 093859d926 Fix TRANSFORM_INVALID_GROUP_CHARS deprecation warning 2019-05-21 10:39:33 -05:00
Slavi Pantaleev c78b02cbe7 Make bridges not depend on matrix-synapse.service if not enabled 2019-05-21 11:13:52 +09:00
Slavi Pantaleev 3339e37ce9 Move matrix-appservice-irc into a separate role 2019-05-16 09:07:40 +09:00
Slavi Pantaleev 43fd3cc274 Move mautrix-facebook into a separate role 2019-05-15 09:34:31 +09:00
Slavi Pantaleev bb816df557 Move mautrix telegram and whatsapp into separate roles
The goal is to move each bridge into its own separate role.
This commit starts off the work on this with 2 bridges:
- mautrix-telegram
- mautrix-whatsapp

Each bridge's role (including these 2) is meant to:

- depend only on the matrix-base role

- integrate nicely with the matrix-synapse role (if available)

- integrate nicely with the matrix-nginx-proxy role (if available and if
required). mautrix-telegram bridge benefits from integrating with
it.

- not break if matrix-synapse or matrix-nginx-proxy are not used at all

This has been provoked by #174 (Github Issue).
2019-05-14 23:47:22 +09:00
Stuart Mumford e1d0667ead
url encode username as well 2019-05-03 22:23:03 +01:00
Stuart Mumford 4e998f52c5
urlencode mxisd password 2019-05-03 17:39:43 +01:00
Aaron Raimist e42fe4b18c
Include Slavi's improvements to keep roles independent 2019-04-27 17:09:21 -05:00
Daniel Hoffend ca15d219b9 make welcome.html customizable 2019-04-25 01:05:28 +02:00
Slavi Pantaleev 73af8f7bbb Make self-check not validate self-signed certificates
By default, `--tags=self-check` no longer validates certificates
when `matrix_ssl_retrieval_method` is set to `self-signed`.

Besides this default, people can also enable/disable validation using the
individual role variables manually.

Fixes #124 (Github Issue)
2019-03-22 09:41:08 +02:00
Slavi Pantaleev 59e37105e8 Add TLS support to Coturn 2019-03-19 10:24:39 +02:00
Slavi Pantaleev 24cf27c60c Isolate Coturn from services in the default Docker network
Most (all?) of our Matrix services are running in the `matrix` network,
so they were safe -- not accessible from Coturn to begin with.

Isolating Coturn into its own network is a security improvement
for people who were starting other services in the default
Docker network. Those services were potentially reachable over the
private Docker network from Coturn.

Discussed in #120 (Github Pull Request)
2019-03-18 17:41:14 +02:00
Slavi Pantaleev c6858d2a08 Define matrix_coturn_turn_external_ip_address in the playbook group vars
This is more explicit than hiding it in the role defaults.

People who reuse the roles in their own playbook (and not only) may
incorrectly define `ansible_host` to be a hostname or some local address.

Making it more explicit is more likely to prevent such mistakes.
2019-03-18 17:04:40 +02:00
Slavi Pantaleev c545d3eb85 Add support for serving base domain via matrix-nginx-proxy 2019-03-12 23:01:16 +02:00
Edgars Voroboks 30738d064e Fix errors 2019-03-08 19:14:15 +02:00
Edgars Voroboks 3d7dc15a54 Use Dimension if enabled, otherwise fall back to Scalar 2019-03-08 15:24:59 +02:00
Edgars Voroboks 1eb78ca93e Add additional changes for Dimension to work 2019-03-08 15:00:53 +02:00
Edgars Voroboks 9735a2f600 Implement self-hosted Dimension server 2019-03-07 07:22:08 +02:00
Slavi Pantaleev 22d72f5e6f Fix incorrectly located file
Related to #105 (Github Pull Request).
2019-03-05 11:31:20 +02:00
Lionstiger e54f388966 removed /group_vars/matrix-servers as it not needed 2019-03-04 14:16:30 +01:00
Lionstiger 835c349275 Add matrix-appservice-discord bridge
Bridge is setup to work on the matrix side with this, but the discord invite link is not automatically generated.
2019-03-03 18:22:52 +01:00
Slavi Pantaleev a43bcd81fe Rename some variables 2019-02-28 11:51:09 +02:00
Slavi Pantaleev eb08e20418 Upgrade Synapse (0.99.0 -> 0.99.1) and sync config
`matrix_synapse_no_tls` is now implicit, so we've gotten rid of it.

The `homeserver.yaml.j2` template has been synchronized with the
configuration generated by Synapse v0.99.1 (some new options
are present, etc.)
2019-02-14 18:40:55 +02:00
Slavi Pantaleev 40f3793af7 Upgrade Synapse to v0.99 and simplify dummy TLS cert logic 2019-02-06 09:17:55 +02:00
Slavi Pantaleev f6ebd4ce62 Initial work on Synapse 0.99/1.0 preparation 2019-02-05 12:09:46 +02:00
Aaron Raimist 1f0cc92b33
Use IPv4 localhost everywhere (or almost everywhere) 2019-02-04 09:49:45 -06:00
dhose 87e3deebfd Enable exposure of Prometheus metrics. 2019-02-01 20:02:11 +01:00
Plailect 5e1d96c727
Add matrix_appservice_irc_container_expose_client_server_api_port 2019-01-31 11:20:45 -05:00
Slavi Pantaleev 0be7b25c64 Make (most) containers run with a read-only filesystem 2019-01-29 18:52:02 +02:00
Slavi Pantaleev 299a8c4c7c Make (most) containers start as non-root
This makes all containers (except mautrix-telegram and
mautrix-whatsapp), start as a non-root user.

We do this, because we don't trust some of the images.
In any case, we'd rather not trust ALL images and avoid giving
`root` access at all. We can't be sure they would drop privileges
or what they might do before they do it.

Because Postfix doesn't support running as non-root,
it had to be replaced by an Exim mail server.

The matrix-nginx-proxy nginx container image is patched up
(by replacing its main configuration) so that it can work as non-root.
It seems like there's no other good image that we can use and that is up-to-date
(https://hub.docker.com/r/nginxinc/nginx-unprivileged is outdated).

Likewise for riot-web (https://hub.docker.com/r/bubuntux/riot-web/),
we patch it up ourselves when starting (replacing the main nginx
configuration).
Ideally, it would be fixed upstream so we can simplify.
2019-01-27 20:25:13 +02:00
Slavi Pantaleev ba75ab496d Send Host/X-Forwarded-For to mxisd
It worked without it too, but doing this is more consistent with the
mxisd recommendations.
2019-01-17 16:22:49 +02:00
Slavi Pantaleev cb11548eec Use mxisd for user directory searches
Implements #77 (Github issue).
2019-01-17 15:55:23 +02:00
Slavi Pantaleev df0d465482 Fix typos in some variables (matrix_mxid -> matrix_mxisd) 2019-01-17 14:47:37 +02:00
Slavi Pantaleev f4f06ae068 Make matrix-nginx-proxy role independent of others
The matrix-nginx-proxy role can now be used independently.
This makes it consistent with all other roles, with
the `matrix-base` role remaining as their only dependency.

Separating matrix-nginx-proxy was relatively straightforward, with
the exception of the Mautrix Telegram reverse-proxying configuration.
Mautrix Telegram, being an extension/bridge, does not feel important enough
to justify its own special handling in matrix-nginx-proxy.

Thus, we've introduced the concept of "additional configuration blocks"
(`matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks`),
where any module can register its own custom nginx server blocks.

For such dynamic registration to work, the order of role execution
becomes important. To make it possible for each module participating
in dynamic registration to verify that the order of execution is
correct, we've also introduced a `matrix_nginx_proxy_role_executed`
variable.

It should be noted that this doesn't make the matrix-synapse role
dependent on matrix-nginx-proxy. It's optional runtime detection
and registration, and it only happens in the matrix-synapse role
when `matrix_mautrix_telegram_enabled: true`.
2019-01-17 13:32:46 +02:00
Slavi Pantaleev c10182e5a6 Make roles more independent of one another
With this change, the following roles are now only dependent
on the minimal `matrix-base` role:
- `matrix-corporal`
- `matrix-coturn`
- `matrix-mailer`
- `matrix-mxisd`
- `matrix-postgres`
- `matrix-riot-web`
- `matrix-synapse`

The `matrix-nginx-proxy` role still does too much and remains
dependent on the others.

Wiring up the various (now-independent) roles happens
via a glue variables file (`group_vars/matrix-servers`).
It's triggered for all hosts in the `matrix-servers` group.

According to Ansible's rules of priority, we have the following
chain of inclusion/overriding now:
- role defaults (mostly empty or good for independent usage)
- playbook glue variables (`group_vars/matrix-servers`)
- inventory host variables (`inventory/host_vars/matrix.<your-domain>`)

All roles default to enabling their main component
(e.g. `matrix_mxisd_enabled: true`, `matrix_riot_web_enabled: true`).
Reasoning: if a role is included in a playbook (especially separately,
in another playbook), it should "work" by default.

Our playbook disables some of those if they are not generally useful
(e.g. `matrix_corporal_enabled: false`).
2019-01-16 18:05:48 +02:00