Merge branch 'master' into pub.solar

.github/workflows/ansible-lint.yml

@@ -0,0 +1,22 @@
+name: Ansible Lint
+
+on: [push, pull_request]
+
+jobs:
+  build:
+  
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Lint Ansible Playbook
+      uses: ansible/ansible-lint-action@c37fb7b4bda2c8cb18f4942716bae9f11b0dc9bc
+      with:
+        # Paths to ansible files (i.e., playbooks, tasks, handlers etc..)
+        targets: "./"
+
+        override-deps: |
+          ansible-lint==5.3.1
+          
+        args: "-x metadata, formatting"

CHANGELOG.md

@@ -1,3 +1,10 @@
+# 2021-12-22
+
+## Twitter bridging support via mautrix-twitter
+
+Thanks to [Matthew Cengia](https://github.com/mattcen) and [Shreyas Ajjarapu](https://github.com/shreyasajj), besides [mx-puppet-twitter](docs/configuring-playbook-bridge-mx-puppet-twitter.md), bridging to [Twitter](https://twitter.com/) can now also happen with [mautrix-twitter](docs/configuring-playbook-bridge-mautrix-twitter.md).
+
+
 # 2021-12-14
 
 ## (Security) Users of the Signal bridge may wish to upgrade it to work around log4j vulnerability

README.md

@@ -51,6 +51,8 @@ Using this playbook, you can get the following services configured on your serve
 
 - (optional) the [mautrix-facebook](https://github.com/mautrix/facebook) bridge for bridging your Matrix server to [Facebook](https://facebook.com/)
 
+- (optional) the [mautrix-twitter](https://github.com/mautrix/twitter) bridge for bridging your Matrix server to [Twitter](https://twitter.com/)
+
 - (optional) the [mautrix-hangouts](https://github.com/mautrix/hangouts) bridge for bridging your Matrix server to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts)
 
 - (optional) the [mautrix-googlechat](https://github.com/mautrix/googlechat) bridge for bridging your Matrix server to [Google Chat](https://en.wikipedia.org/wiki/Google_Chat)

docs/configuring-playbook-bridge-mautrix-twitter.md

@@ -0,0 +1,37 @@
+# Setting up Mautrix Twitter (optional)
+
+**Note**: bridging to [Twitter](https://twitter.com/) can also happen via the [mx-puppet-twitter](configuring-playbook-bridge-mx-puppet-twitter.md) bridge supported by the playbook.
+
+The playbook can install and configure [mautrix-twitter](https://github.com/tulir/mautrix-twitter) for you.
+
+See the project's [documentation](https://github.com/tulir/mautrix-twitter/wiki#usage) to learn what it does and why it might be useful to you.
+
+```yaml
+matrix_mautrix_twitter_enabled: true
+```
+
+
+## Set up Double Puppeting
+
+If you'd like to use [Double Puppeting](https://github.com/tulir/mautrix-twitter/wiki/Authentication#double-puppeting) (hint: you most likely do), you have 2 ways of going about it.
+
+### Method 1: automatically, by enabling Shared Secret Auth
+
+The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
+
+This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
+
+### Method 2: manually, by asking each user to provide a working access token
+
+This method is currently not available for the Mautrix-Twitter bridge, but is on the [roadmap](https://github.com/tulir/mautrix-twitter/blob/master/ROADMAP.md) under Misc/Manual login with `login-matrix`
+
+## Usage
+
+1. You then need to start a chat with `@twitterbot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
+2. Send login-cookie to start the login. The bot should respond with instructions on how to proceed.
+
+You can learn more here about authentication from the bridge's [official documentation on Authentication](https://docs.mau.fi/bridges/python/twitter/authentication.html).
+
+If you run into trouble, check the [Troubleshooting](#troubleshooting) section below.
+
+After successfully enabling bridging, you may wish to [set up Double Puppeting](#set-up-double-puppeting), if you haven't already done so.

docs/configuring-playbook-bridge-mx-puppet-twitter.md

@@ -1,5 +1,7 @@
 # Setting up MX Puppet Twitter (optional)
 
+**Note**: bridging to [Twitter](https://twitter.com/) can also happen via the [mautrix-twitter](configuring-playbook-bridge-mautrix-twitter.md) bridge supported by the playbook.
+
 The playbook can install and configure
 [mx-puppet-twitter](https://github.com/Sorunome/mx-puppet-twitter) for you.
 

docs/configuring-playbook.md

@@ -102,6 +102,8 @@ When you're done with all the configuration you'd like to do, continue with [Ins
 
 - [Setting up Mautrix Instagram bridging](configuring-playbook-bridge-mautrix-instagram.md) (optional)
 
+- [Setting up Mautrix Twitter bridging](configuring-playbook-bridge-mautrix-twitter.md) (optional)
+
 - [Setting up Mautrix Signal bridging](configuring-playbook-bridge-mautrix-signal.md) (optional)
 
 - [Setting up Appservice IRC bridging](configuring-playbook-bridge-appservice-irc.md) (optional)

docs/container-images.md

@@ -46,6 +46,8 @@ These services are not part of our default installation, but can be enabled by [
 
 - [mautrix/facebook](https://mau.dev/mautrix/facebook/container_registry) - the [mautrix-facebook](https://github.com/mautrix/facebook) bridge to [Facebook](https://facebook.com/) (optional)
 
+- [tulir/mautrix-twitter](https://mau.dev/mautrix/twitter/container_registry) - the [mautrix-twitter](https://github.com/tulir/mautrix-twitter) bridge to [Twitter](https://twitter.com/) (optional)
+
 - [mautrix/hangouts](https://mau.dev/mautrix/hangouts/container_registry) - the [mautrix-hangouts](https://github.com/mautrix/hangouts) bridge to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts) (optional)
 
 - [mautrix/googlechat](https://mau.dev/mautrix/googlechat/container_registry) - the [mautrix-googlechat](https://github.com/mautrix/googlechat) bridge to [Google Chat](https://en.wikipedia.org/wiki/Google_Chat) (optional)

examples/caddy2/Caddyfile

@@ -27,6 +27,10 @@ matrix.DOMAIN.tld {
         not path /matrix/static-files/*
   }
 
+  @wellknown {
+        path /.well-known/matrix/*
+  }
+
   header {
         # Enable HTTP Strict Transport Security (HSTS) to force clients to always connect via HTTPS
         Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
@@ -69,6 +73,15 @@ matrix.DOMAIN.tld {
         }
   }
 
+  handle @wellknown {
+        encode zstd gzip
+        root * /matrix/static-files
+	header Cache-Control max-age=14400
+        header Content-Type application/json
+        header Access-Control-Allow-Origin *
+        file_server
+  }
+
   handle {
         encode zstd gzip
 

group_vars/matrix_servers

@@ -511,6 +511,45 @@ matrix_mautrix_telegram_database_password: "{{ '%s' | format(matrix_synapse_maca
 #
 ######################################################################
 
+######################################################################
+#
+# matrix-bridge-mautrix-twitter
+#
+######################################################################
+
+# We don't enable bridges by default.
+matrix_mautrix_twitter_enabled: false
+
+matrix_mautrix_twitter_container_image_self_build: "{{ matrix_architecture not in ['amd64', 'arm64'] }}"
+
+matrix_mautrix_twitter_systemd_required_services_list: |
+  {{
+    ['docker.service']
+    +
+    (['matrix-synapse.service'] if matrix_synapse_enabled else [])
+    +
+    (['matrix-postgres.service'] if matrix_postgres_enabled else [])
+    +
+    (['matrix-nginx-proxy.service'] if matrix_nginx_proxy_enabled else [])
+  }}
+
+matrix_mautrix_twitter_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'twt.as.token') | to_uuid }}"
+
+matrix_mautrix_twitter_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'twt.hs.token') | to_uuid }}"
+
+matrix_mautrix_twitter_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}"
+
+# We'd like to force-set people with external Postgres to SQLite, so the bridge role can complain
+# and point them to a migration path.
+matrix_mautrix_twitter_database_engine: "{{ 'postgres' if matrix_postgres_enabled else '' }}"
+matrix_mautrix_twitter_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.twt.db') | to_uuid }}"
+
+######################################################################
+#
+# /matrix-bridge-mautrix-twitter
+#
+######################################################################
+
 ######################################################################
 #
 # matrix-bridge-mautrix-whatsapp
@@ -1511,6 +1550,12 @@ matrix_postgres_additional_databases: |
       'password': matrix_mautrix_telegram_database_password,
     }] if (matrix_mautrix_telegram_enabled and matrix_mautrix_telegram_database_engine == 'postgres' and matrix_mautrix_telegram_database_hostname == 'matrix-postgres') else [])
     +
+    ([{
+      'name': matrix_mautrix_twitter_database_name,
+      'username': matrix_mautrix_twitter_database_username,
+      'password': matrix_mautrix_twitter_database_password,
+    }] if (matrix_mautrix_twitter_enabled and matrix_mautrix_twitter_database_engine == 'postgres' and matrix_mautrix_twitter_database_hostname == 'matrix-postgres') else [])
+    +
     ([{
       'name': matrix_mautrix_whatsapp_database_name,
       'username': matrix_mautrix_whatsapp_database_username,

roles/matrix-bridge-mautrix-instagram/templates/config.yaml.j2

@@ -43,7 +43,7 @@ appservice:
   bot_username: {{ matrix_mautrix_instagram_appservice_bot_username|to_json }}
   # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty
   # to leave display name/avatar as-is.
-  bot_displayname: instagram bridge bot
+  bot_displayname: Instagram bridge bot
   bot_avatar: mxc://maunium.net/JxjlbZUlCPULEeHZSwleUXQv
 
   # Community ID for bridged users (changes registration file) and rooms.

roles/matrix-bridge-mautrix-twitter/defaults/main.yml

@@ -0,0 +1,103 @@
+# mautrix-twitter is a Matrix <-> Twitter bridge
+# See: https://github.com/tulir/mautrix-twitter
+
+matrix_mautrix_twitter_enabled: true
+
+matrix_mautrix_twitter_container_image_self_build: false
+matrix_mautrix_twitter_container_image_self_build_repo: "https://github.com/tulir/mautrix-twitter.git"
+
+matrix_mautrix_twitter_version: latest
+# See: https://mau.dev/tulir/mautrix-twitter/container_registry
+matrix_mautrix_twitter_docker_image: "{{ matrix_mautrix_twitter_docker_image_name_prefix }}mautrix/twitter:{{ matrix_mautrix_twitter_version }}"
+matrix_mautrix_twitter_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_twitter_container_image_self_build else 'dock.mau.dev/' }}"
+matrix_mautrix_twitter_docker_image_force_pull: "{{ matrix_mautrix_twitter_docker_image.endswith(':latest') }}"
+
+matrix_mautrix_twitter_base_path: "{{ matrix_base_data_path }}/mautrix-twitter"
+matrix_mautrix_twitter_config_path: "{{ matrix_mautrix_twitter_base_path }}/config"
+matrix_mautrix_twitter_data_path: "{{ matrix_mautrix_twitter_base_path }}/data"
+matrix_mautrix_twitter_docker_src_files_path: "{{ matrix_mautrix_twitter_base_path }}/docker-src"
+
+matrix_mautrix_twitter_homeserver_address: "{{ matrix_homeserver_container_url }}"
+matrix_mautrix_twitter_homeserver_domain: '{{ matrix_domain }}'
+matrix_mautrix_twitter_appservice_address: 'http://matrix-mautrix-twitter:29327'
+
+# A list of extra arguments to pass to the container
+matrix_mautrix_twitter_container_extra_arguments: []
+
+# List of systemd services that matrix-mautrix-twitter.service depends on.
+matrix_mautrix_twitter_systemd_required_services_list: ['docker.service']
+
+# List of systemd services that matrix-mautrix-twitter.service wants
+matrix_mautrix_twitter_systemd_wanted_services_list: []
+
+matrix_mautrix_twitter_appservice_token: ''
+matrix_mautrix_twitter_homeserver_token: ''
+
+
+# Database-related configuration fields.
+#
+# To use Postgres:
+# - adjust your database credentials via the `matrix_mautrix_twitter_postgres_*` variables
+matrix_mautrix_twitter_database_engine: 'postgres'
+
+matrix_mautrix_twitter_database_username: 'matrix_mautrix_twitter'
+matrix_mautrix_twitter_database_password: 'some-password'
+matrix_mautrix_twitter_database_hostname: 'matrix-postgres'
+matrix_mautrix_twitter_database_port: 5432
+matrix_mautrix_twitter_database_name: 'matrix_mautrix_twitter'
+
+matrix_mautrix_twitter_database_connection_string: 'postgres://{{ matrix_mautrix_twitter_database_username }}:{{ matrix_mautrix_twitter_database_password }}@{{ matrix_mautrix_twitter_database_hostname }}:{{ matrix_mautrix_twitter_database_port }}/{{ matrix_mautrix_twitter_database_name }}'
+
+matrix_mautrix_twitter_appservice_database: "{{
+	{
+		'postgres': matrix_mautrix_twitter_database_connection_string,
+	}[matrix_mautrix_twitter_database_engine]
+}}"
+
+
+# Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth).
+matrix_mautrix_twitter_login_shared_secret: ''
+
+matrix_mautrix_twitter_bridge_login_shared_secret_map: "{{ {matrix_mautrix_twitter_homeserver_domain: matrix_mautrix_twitter_login_shared_secret} if matrix_mautrix_twitter_login_shared_secret else {} }}"
+
+matrix_mautrix_twitter_appservice_bot_username: twitterbot
+
+# Default configuration template which covers the generic use case.
+# You can customize it by controlling the various variables inside it.
+#
+# For a more advanced customization, you can extend the default (see `matrix_mautrix_twitter_configuration_extension_yaml`)
+# or completely replace this variable with your own template.
+matrix_mautrix_twitter_configuration_yaml: "{{ lookup('template', 'templates/config.yaml.j2') }}"
+
+matrix_mautrix_twitter_configuration_extension_yaml: |
+  # Your custom YAML configuration goes here.
+  # This configuration extends the default starting configuration (`matrix_mautrix_twitter_configuration_yaml`).
+  #
+  # You can override individual variables from the default configuration, or introduce new ones.
+  #
+  # If you need something more special, you can take full control by
+  # completely redefining `matrix_mautrix_twitter_configuration_yaml`.
+
+matrix_mautrix_twitter_configuration_extension: "{{ matrix_mautrix_twitter_configuration_extension_yaml|from_yaml if matrix_mautrix_twitter_configuration_extension_yaml|from_yaml is mapping else {} }}"
+
+# Holds the final configuration (a combination of the default and its extension).
+# You most likely don't need to touch this variable. Instead, see `matrix_mautrix_twitter_configuration_yaml`.
+matrix_mautrix_twitter_configuration: "{{ matrix_mautrix_twitter_configuration_yaml|from_yaml|combine(matrix_mautrix_twitter_configuration_extension, recursive=True) }}"
+
+matrix_mautrix_twitter_registration_yaml: |
+  id: twitter
+  as_token: "{{ matrix_mautrix_twitter_appservice_token }}"
+  hs_token: "{{ matrix_mautrix_twitter_homeserver_token }}"
+  namespaces:
+    users:
+    - exclusive: true
+      regex: '^@twitter_.+:{{ matrix_mautrix_twitter_homeserver_domain|regex_escape }}$'
+    - exclusive: true
+      regex: '^@{{ matrix_mautrix_twitter_appservice_bot_username|regex_escape }}:{{ matrix_mautrix_twitter_homeserver_domain|regex_escape }}$'
+  url: {{ matrix_mautrix_twitter_appservice_address }}
+  # See https://github.com/tulir/mautrix-signal/issues/43
+  sender_localpart: _bot_{{ matrix_mautrix_twitter_appservice_bot_username }}
+  rate_limited: false
+  de.sorunome.msc2409.push_ephemeral: true
+
+matrix_mautrix_twitter_registration: "{{ matrix_mautrix_twitter_registration_yaml|from_yaml }}"

roles/matrix-bridge-mautrix-twitter/tasks/init.yml

@@ -0,0 +1,23 @@
+- set_fact:
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-twitter.service'] }}"
+  when: matrix_mautrix_twitter_enabled|bool
+
+# If the matrix-synapse role is not used, these variables may not exist.
+- set_fact:
+    matrix_synapse_container_extra_arguments: >
+      {{ matrix_synapse_container_extra_arguments|default([]) }}
+      +
+      ["--mount type=bind,src={{ matrix_mautrix_twitter_config_path }}/registration.yaml,dst=/matrix-mautrix-twitter-registration.yaml,ro"]
+
+    matrix_synapse_app_service_config_files: >
+      {{ matrix_synapse_app_service_config_files|default([]) }}
+      +
+      {{ ["/matrix-mautrix-twitter-registration.yaml"] }}
+  when: matrix_mautrix_twitter_enabled|bool
+
+# ansible lower than 2.8, does not support docker_image build parameters
+# for self buildig it is explicitly needed, so we rather fail here
+- name: Fail if running on Ansible lower than 2.8 and trying self building
+  fail:
+    msg: "To self build Mautrix Twitter image, you should usa ansible 2.8 or higher. E.g. pip contains such packages."
+  when: "ansible_version.major == 2 and ansible_version.minor < 8 and matrix_mautrix_twitter_container_image_self_build"

roles/matrix-bridge-mautrix-twitter/tasks/main.yml

@@ -0,0 +1,21 @@
+- import_tasks: "{{ role_path }}/tasks/init.yml"
+  tags:
+    - always
+
+- import_tasks: "{{ role_path }}/tasks/validate_config.yml"
+  when: "run_setup|bool and matrix_mautrix_twitter_enabled|bool"
+  tags:
+    - setup-all
+    - setup-mautrix-twitter
+
+- import_tasks: "{{ role_path }}/tasks/setup_install.yml"
+  when: "run_setup|bool and matrix_mautrix_twitter_enabled|bool"
+  tags:
+    - setup-all
+    - setup-mautrix-twitter
+
+- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
+  when: "run_setup|bool and not matrix_mautrix_twitter_enabled|bool"
+  tags:
+    - setup-all
+    - setup-mautrix-twitter

roles/matrix-bridge-mautrix-twitter/tasks/setup_install.yml

@@ -0,0 +1,88 @@
+---
+
+# If the matrix-synapse role is not used, `matrix_synapse_role_executed` won't exist.
+# We don't want to fail in such cases.
+- name: Fail if matrix-synapse role already executed
+  fail:
+    msg: >-
+      The matrix-bridge-mautrix-twitter role needs to execute before the matrix-synapse role.
+  when: "matrix_synapse_role_executed|default(False)"
+
+- set_fact:
+    matrix_mautrix_twitter_requires_restart: false
+
+- name: Ensure Mautrix Twitter image is pulled
+  docker_image:
+    name: "{{ matrix_mautrix_twitter_docker_image }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_mautrix_twitter_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_twitter_docker_image_force_pull }}"
+  when: matrix_mautrix_twitter_enabled|bool and not matrix_mautrix_twitter_container_image_self_build
+
+- name: Ensure Mautrix Twitter paths exist
+  file:
+    path: "{{ item.path }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - { path: "{{ matrix_mautrix_twitter_base_path }}", when: true }
+    - { path: "{{ matrix_mautrix_twitter_config_path }}", when: true }
+    - { path: "{{ matrix_mautrix_twitter_data_path }}", when: true }
+    - { path: "{{ matrix_mautrix_twitter_docker_src_files_path }}", when: "{{ matrix_mautrix_twitter_container_image_self_build }}" }
+  when: item.when|bool
+
+- name: Ensure Mautrix Twitter repository is present on self-build
+  git:
+    repo: "{{ matrix_mautrix_twitter_container_image_self_build_repo }}"
+    dest: "{{ matrix_mautrix_twitter_docker_src_files_path }}"
+#    version: "{{ matrix_coturn_docker_image.split(':')[1] }}"
+    force: "yes"
+  register: matrix_mautrix_twitter_git_pull_results
+  when: "matrix_mautrix_twitter_enabled|bool and matrix_mautrix_twitter_container_image_self_build"
+
+- name: Ensure Mautrix Twitter Docker image is built
+  docker_image:
+    name: "{{ matrix_mautrix_twitter_docker_image }}"
+    source: build
+    force_source: "{{ matrix_mautrix_twitter_git_pull_results.changed }}"
+    build:
+      dockerfile: Dockerfile
+      path: "{{ matrix_mautrix_twitter_docker_src_files_path }}"
+      pull: yes
+  when: "matrix_mautrix_twitter_enabled|bool and matrix_mautrix_twitter_container_image_self_build|bool"
+
+- name: Ensure mautrix-twitter config.yaml installed
+  copy:
+    content: "{{ matrix_mautrix_twitter_configuration|to_nice_yaml }}"
+    dest: "{{ matrix_mautrix_twitter_config_path }}/config.yaml"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+
+- name: Ensure mautrix-twitter registration.yaml installed
+  copy:
+    content: "{{ matrix_mautrix_twitter_registration|to_nice_yaml }}"
+    dest: "{{ matrix_mautrix_twitter_config_path }}/registration.yaml"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+
+- name: Ensure matrix-mautrix-twitter.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-mautrix-twitter.service.j2"
+    dest: "{{ matrix_systemd_path }}/matrix-mautrix-twitter.service"
+    mode: 0644
+  register: matrix_mautrix_twitter_systemd_service_result
+
+- name: Ensure systemd reloaded after matrix-mautrix-twitter.service installation
+  service:
+    daemon_reload: yes
+  when: "matrix_mautrix_twitter_systemd_service_result.changed"
+
+- name: Ensure matrix-mautrix-twitter.service restarted, if necessary
+  service:
+    name: "matrix-mautrix-twitter.service"
+    state: restarted
+  when: "matrix_mautrix_twitter_requires_restart|bool"

roles/matrix-bridge-mautrix-twitter/tasks/setup_uninstall.yml

@@ -0,0 +1,24 @@
+---
+
+- name: Check existence of matrix-mautrix-twitter service
+  stat:
+    path: "{{ matrix_systemd_path }}/matrix-mautrix-twitter.service"
+  register: matrix_mautrix_twitter_service_stat
+
+- name: Ensure matrix-mautrix-twitter is stopped
+  service:
+    name: matrix-mautrix-twitter
+    state: stopped
+    daemon_reload: yes
+  when: "matrix_mautrix_twitter_service_stat.stat.exists"
+
+- name: Ensure matrix-mautrix-twitter.service doesn't exist
+  file:
+    path: "{{ matrix_systemd_path }}/matrix-mautrix-twitter.service"
+    state: absent
+  when: "matrix_mautrix_twitter_service_stat.stat.exists"
+
+- name: Ensure systemd reloaded after matrix-mautrix-twitter.service removal
+  service:
+    daemon_reload: yes
+  when: "matrix_mautrix_twitter_service_stat.stat.exists"

roles/matrix-bridge-mautrix-twitter/tasks/validate_config.yml

@@ -0,0 +1,18 @@
+---
+
+- name: Fail if required settings not defined
+  fail:
+    msg: >-
+      You need to define a required configuration setting (`{{ item }}`).
+  when: "vars[item] == ''"
+  with_items:
+    - "matrix_mautrix_twitter_appservice_token"
+    - "matrix_mautrix_twitter_homeserver_token"
+
+- name: Fail if database is not defined
+  fail:
+    msg: >-
+      You need to define a need to set `matrix_mautrix_twitter_database_engine: postgres` and redefine the other `matrix_mautrix_twitter_database_*` variables
+  when: "vars[item] == ''"
+  with_items:
+    - "matrix_mautrix_twitter_database_engine"

roles/matrix-bridge-mautrix-twitter/templates/config.yaml.j2

@@ -0,0 +1,209 @@
+#jinja2: lstrip_blocks: "True"
+# Homeserver details
+homeserver:
+    # The address that this appservice can use to connect to the homeserver.
+    address: {{ matrix_mautrix_twitter_homeserver_address }}
+    # The domain of the homeserver (for MXIDs, etc).
+    domain: {{ matrix_mautrix_twitter_homeserver_domain }}
+    # Whether or not to verify the SSL certificate of the homeserver.
+    # Only applies if address starts with https://
+    verify_ssl: true
+    asmux: false
+
+# Application service host/registration related details
+# Changing these values requires regeneration of the registration.
+appservice:
+    # The address that the homeserver can use to connect to this appservice.
+    address: {{ matrix_mautrix_twitter_appservice_address }}
+    # When using https:// the TLS certificate and key files for the address.
+    tls_cert: false
+    tls_key: false
+
+    # The hostname and port where this appservice should listen.
+    hostname: 0.0.0.0
+    port: 29327
+    # The maximum body size of appservice API requests (from the homeserver) in mebibytes
+    # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s
+    max_body_size: 1
+
+    # The full URI to the database. Only Postgres is currently supported.
+    database: {{ matrix_mautrix_twitter_appservice_database|to_json }}
+    # Additional arguments for asyncpg.create_pool()
+    # https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.pool.create_pool
+    database_opts:
+        min_size: 5
+        max_size: 10
+
+    # Provisioning API part of the web server for automated portal creation and fetching information.
+    # Used by things like mautrix-manager (https://github.com/tulir/mautrix-manager).
+    provisioning:
+        # Whether or not the provisioning API should be enabled.
+        enabled: true
+        # The prefix to use in the provisioning API endpoints.
+        prefix: /_matrix/provision/v1
+        # The shared secret to authorize users of the API.
+        # Set to "generate" to generate and save a new token.
+        shared_secret: generate
+
+    # The unique ID of this appservice.
+    id: twitter
+    # Username of the appservice bot.
+    bot_username: {{ matrix_mautrix_twitter_appservice_bot_username|to_json }}
+    # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty
+    # to leave display name/avatar as-is.
+    bot_displayname: Twitter bridge bot
+    bot_avatar: mxc://maunium.net/HVHcnusJkQcpVcsVGZRELLCn
+
+    # Community ID for bridged users (changes registration file) and rooms.
+    # Must be created manually.
+    #
+    # Example: "+twitter:example.com". Set to false to disable.
+    community_id: false
+
+    # Whether or not to receive ephemeral events via appservice transactions.
+    # Requires MSC2409 support (i.e. Synapse 1.22+).
+    # You should disable bridge -> sync_with_custom_puppets when this is enabled.
+    ephemeral_events: false
+
+    # Authentication tokens for AS <-> HS communication. Autogenerated; do not modify.
+    as_token: "{{ matrix_mautrix_twitter_appservice_token }}"
+    hs_token: "{{ matrix_mautrix_twitter_homeserver_token }}"
+
+# Prometheus telemetry config. Requires prometheus-client to be installed.
+metrics:
+    enabled: false
+    listen_port: 8000
+
+# Bridge config
+bridge:
+    # Localpart template of MXIDs for Twitter users.
+    # {userid} is replaced with the user ID of the Twitter user.
+    username_template: "twitter_{userid}"
+    # Displayname template for Twitter users.
+    # {displayname} is replaced with the display name of the Twitter user.
+    # {username} is replaced with the username of the Twitter user.
+    displayname_template: "{displayname} (Twitter)"
+
+    # Maximum length of displayname
+    displayname_max_length: 100
+
+    # Number of conversations to sync (and create portals for) on login.
+    # Set 0 to disable automatic syncing.
+    initial_conversation_sync: 10
+    # Whether or not to use /sync to get read receipts and typing notifications
+    # when double puppeting is enabled
+    sync_with_custom_puppets: true
+    # Whether or not to update the m.direct account data event when double puppeting is enabled.
+    # Note that updating the m.direct event is not atomic (except with mautrix-asmux)
+    # and is therefore prone to race conditions.
+    sync_direct_chat_list: false
+    # Allow using double puppeting from any server with a valid client .well-known file.
+    double_puppet_allow_discovery: false
+    # Servers to allow double puppeting from, even if double_puppet_allow_discovery is false.
+    double_puppet_server_map: {}
+    # Shared secret for https://github.com/devture/matrix-synapse-shared-secret-auth
+    #
+    # If set, custom puppets will be enabled automatically for local users
+    # instead of users having to find an access token and run `login-matrix`
+    # manually.
+    # If using this for other servers than the bridge's server,
+    # you must also set the URL in the double_puppet_server_map.
+    login_shared_secret_map: {{ matrix_mautrix_twitter_bridge_login_shared_secret_map|to_json }}
+    # Whether or not created rooms should have federation enabled.
+    # If false, created portal rooms will never be federated.
+    federate_rooms: true
+    # Settings for backfilling messages from Twitter.
+    #
+    # Missed message backfilling is currently based on receiving them from the Twitter polling API,
+    # rather than manually asking for messages in each conversation. Due to this, there's no way to
+    # set a limit for missed message backfilling.
+    backfill:
+        # Whether or not the Twitter users of logged in Matrix users should be
+        # invited to private chats when backfilling history from Twitter. This is
+        # usually needed to prevent rate limits and to allow timestamp massaging.
+        invite_own_puppet: true
+        # Maximum number of messages to backfill initially.
+        # Set to 0 to disable backfilling when creating portal.
+        initial_limit: 0
+        # If using double puppeting, should notifications be disabled
+        # while the initial backfill is in progress?
+        disable_notifications: true
+    # End-to-bridge encryption support options. You must install the e2be optional dependency for
+    # this to work. See https://github.com/tulir/mautrix-telegram/wiki/End‐to‐bridge-encryption
+    encryption:
+        # Allow encryption, work in group chat rooms with e2ee enabled
+        allow: false
+        # Default to encryption, force-enable encryption in all portals the bridge creates
+        # This will cause the bridge bot to be in private chats for the encryption to work properly.
+        default: false
+        # Options for automatic key sharing.
+        key_sharing:
+            # Enable key sharing? If enabled, key requests for rooms where users are in will be fulfilled.
+            # You must use a client that supports requesting keys from other users to use this feature.
+            allow: false
+            # Require the requesting device to have a valid cross-signing signature?
+            # This doesn't require that the bridge has verified the device, only that the user has verified it.
+            # Not yet implemented.
+            require_cross_signing: false
+            # Require devices to be verified by the bridge?
+            # Verification by the bridge is not yet implemented.
+            require_verification: true
+    # Whether or not to explicitly set the avatar and room name for private
+    # chat portal rooms. This will be implicitly enabled if encryption.default is true.
+    private_chat_portal_meta: false
+    # Whether or not the bridge should send a read receipt from the bridge bot when a message has
+    # been sent to Twitter.
+    delivery_receipts: false
+    # Whether or not delivery errors should be reported as messages in the Matrix room.
+    delivery_error_reports: false
+    # Whether or not non-fatal polling errors should send notices to the notice room.
+    temporary_disconnect_notices: true
+    # Number of seconds to sleep more than the previous error when a polling error occurs.
+    # Growth is capped at 15 minutes.
+    error_sleep: 5
+    # Maximum number of polling errors before giving up. Set to -1 to retry forever.
+    max_poll_errors: 12
+    # Set this to true to tell the bridge to re-send m.bridge events to all rooms on the next run.
+    # This field will automatically be changed back to false after it,
+    # except if the config file is not writable.
+    resend_bridge_info: false
+
+    # The prefix for commands. Only required in non-management rooms.
+    command_prefix: "!tw"
+
+    # Permissions for using the bridge.
+    # Permitted values:
+    #       user - Use the bridge with puppeting.
+    #      admin - Use and administrate the bridge.
+    # Permitted keys:
+    #        * - All Matrix users
+    #   domain - All users on that homeserver
+    #     mxid - Specific user
+    permissions:
+      '{{ matrix_mautrix_twitter_homeserver_domain }}': user
+
+
+# Python logging configuration.
+#
+# See section 16.7.2 of the Python documentation for more info:
+# https://docs.python.org/3.6/library/logging.config.html#configuration-dictionary-schema
+logging:
+    version: 1
+    formatters:
+        colored:
+            (): mautrix_twitter.util.ColorFormatter
+            format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s"
+        normal:
+            format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s"
+    handlers:
+        console:
+            class: logging.StreamHandler
+            formatter: colored
+    loggers:
+        mau:
+            level: DEBUG
+        aiohttp:
+            level: INFO
+    root:
+        level: DEBUG
+        handlers: [console]

roles/matrix-bridge-mautrix-twitter/templates/systemd/matrix-mautrix-twitter.service.j2

@@ -0,0 +1,42 @@
+#jinja2: lstrip_blocks: "True"
+[Unit]
+Description=Matrix Mautrix Twitter bridge
+{% for service in matrix_mautrix_twitter_systemd_required_services_list %}
+Requires={{ service }}
+After={{ service }}
+{% endfor %}
+{% for service in matrix_mautrix_twitter_systemd_wanted_services_list %}
+Wants={{ service }}
+{% endfor %}
+DefaultDependencies=no
+
+[Service]
+Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
+ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill matrix-mautrix-twitter 2>/dev/null'
+ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-mautrix-twitter 2>/dev/null'
+
+# Intentional delay, so that the homeserver (we likely depend on) can manage to start.
+ExecStartPre={{ matrix_host_command_sleep }} 5
+
+ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-twitter \
+			--log-driver=none \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--network={{ matrix_docker_network }} \
+			-v {{ matrix_mautrix_twitter_config_path }}:/config:z \
+			-v {{ matrix_mautrix_twitter_data_path }}:/data:z \
+			{% for arg in matrix_mautrix_twitter_container_extra_arguments %}
+			{{ arg }} \
+			{% endfor %}
+			{{ matrix_mautrix_twitter_docker_image }} \
+			python3 -m mautrix_twitter -c /config/config.yaml --no-update
+
+ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} kill matrix-mautrix-twitter 2>/dev/null'
+ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-mautrix-twitter 2>/dev/null'
+Restart=always
+RestartSec=30
+SyslogIdentifier=matrix-mautrix-twitter
+
+[Install]
+WantedBy=multi-user.target

roles/matrix-client-element/defaults/main.yml

@@ -7,7 +7,7 @@ matrix_client_element_container_image_self_build_repo: "https://github.com/vecto
 # - https://github.com/vector-im/element-web/issues/19544
 matrix_client_element_container_image_self_build_low_memory_system_patch_enabled: "{{ ansible_memtotal_mb < 4096 }}"
 
-matrix_client_element_version: v1.9.7
+matrix_client_element_version: v1.9.8
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"

roles/matrix-nginx-proxy/defaults/main.yml

@@ -442,7 +442,14 @@ matrix_ssl_domains_to_obtain_certificates_for: "{{ matrix_ssl_additional_domains
 matrix_ssl_additional_domains_to_obtain_certificates_for: []
 
 # Controls whether to obtain production or staging certificates from Let's Encrypt.
+# If you'd like to use another ACME Certificate Authority server (not Let's Encrypt), use `matrix_ssl_lets_encrypt_server`
 matrix_ssl_lets_encrypt_staging: false
+
+# Controls from which Certificate Authority server to retrieve the SSL certificates (passed as a `--server` flag to Certbot).
+# By default, we use the Let's Encrypt production environment (use `matrix_ssl_lets_encrypt_staging` for using the staging environment).
+# Learn more here: https://eff-certbot.readthedocs.io/en/stable/using.html#changing-the-acme-server
+matrix_ssl_lets_encrypt_server: ''
+
 matrix_ssl_lets_encrypt_certbot_docker_image: "{{ matrix_container_global_registry_prefix }}certbot/certbot:{{ matrix_ssl_architecture }}-v1.21.0"
 matrix_ssl_lets_encrypt_certbot_docker_image_force_pull: "{{ matrix_ssl_lets_encrypt_certbot_docker_image.endswith(':latest') }}"
 matrix_ssl_lets_encrypt_certbot_standalone_http_port: 2402

roles/matrix-nginx-proxy/tasks/ssl/setup_ssl_lets_encrypt_obtain_for_domain.yml

@@ -42,6 +42,7 @@
     --non-interactive
     --work-dir=/tmp
     --http-01-port 8080
+    {% if matrix_ssl_lets_encrypt_server %}--server={{ matrix_ssl_lets_encrypt_server|quote }}{% endif %}
     {% if matrix_ssl_lets_encrypt_staging %}--staging{% endif %}
     --standalone
     --preferred-challenges http
@@ -70,6 +71,7 @@
     --non-interactive
     --work-dir=/tmp
     --http-01-port 8080
+    {% if matrix_ssl_lets_encrypt_server %}--server={{ matrix_ssl_lets_encrypt_server|quote }}{% endif %}
     {% if matrix_ssl_lets_encrypt_staging %}--staging{% endif %}
     --standalone
     --preferred-challenges http

roles/matrix-synapse-admin/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_synapse_admin_container_self_build_repo: "https://github.com/Awesome-Tech
 
 matrix_synapse_admin_docker_src_files_path: "{{ matrix_base_data_path }}/synapse-admin/docker-src"
 
-matrix_synapse_admin_version: 0.8.1
+matrix_synapse_admin_version: 0.8.4
 matrix_synapse_admin_docker_image: "{{ matrix_synapse_admin_docker_image_name_prefix }}awesometechnologies/synapse-admin:{{ matrix_synapse_admin_version }}"
 matrix_synapse_admin_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_admin_container_self_build else matrix_container_global_registry_prefix }}"
 matrix_synapse_admin_docker_image_force_pull: "{{ matrix_synapse_admin_docker_image.endswith(':latest') }}"

roles/matrix-synapse/defaults/main.yml

@@ -15,8 +15,8 @@ matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_cont
 # amd64 gets released first.
 # arm32 relies on self-building, so the same version can be built immediately.
 # arm64 users need to wait for a prebuilt image to become available.
-matrix_synapse_version: v1.49.0
+matrix_synapse_version: v1.49.2
-matrix_synapse_version_arm64: v1.49.0
+matrix_synapse_version_arm64: v1.49.2
 matrix_synapse_docker_image_tag: "{{ matrix_synapse_version if matrix_architecture in ['arm32', 'amd64'] else matrix_synapse_version_arm64 }}"
 matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}"
 
@@ -321,6 +321,9 @@ matrix_synapse_push_include_content: true
 # URLs shared by users.
 matrix_synapse_url_preview_enabled: true
 
+# A list of values for the Accept-Language HTTP header used when downloading webpages during URL preview generation
+matrix_url_preview_accept_language: ['en-US', 'en']
+
 # Enable exposure of metrics to Prometheus
 # See https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md
 matrix_synapse_metrics_enabled: false

roles/matrix-synapse/tasks/synapse/workers/setup_uninstall.yml

@@ -8,6 +8,7 @@
     name: "{{ item.key }}"
     state: stopped
   with_dict: "{{ ansible_facts.services|default({})|dict2items|selectattr('key', 'match', 'matrix-synapse-worker-.+\\.service')|list|items2dict }}"
+  when: "item.value['status'] != 'not-found'" # see https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1461
 
 - name: Find worker configs to be cleaned
   find:

roles/matrix-synapse/templates/synapse/homeserver.yaml.j2

@@ -1149,8 +1149,7 @@ max_spider_size: 10M
 #   - fr;q=0.8
 #   - *;q=0.7
 #
-url_preview_accept_language:
+url_preview_accept_language: {{ matrix_url_preview_accept_language|to_json }}
-#   - en
 
 
 ## Captcha ##

setup.yml

@@ -20,6 +20,7 @@
     - matrix-bridge-appservice-irc
     - matrix-bridge-beeper-linkedin
     - matrix-bridge-mautrix-facebook
+    - matrix-bridge-mautrix-twitter
     - matrix-bridge-mautrix-hangouts
     - matrix-bridge-mautrix-googlechat
     - matrix-bridge-mautrix-instagram