Merge branch 'master' into pub.solar

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,57 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+<!--
+NOTE: This Ansible playbook installs tens of separate services. If you're having a problem with a specific service, it is likely that the problem is not with our deployment method, but with the service itself. You may wish to report that problem at the source, upstream, and not to us
+-->
+
+**To Reproduce**
+My `vars.yml` file looks like this:
+
+```yaml
+Paste your vars.yml file here.
+Make sure to remove any secret values before posting your vars.yml file publicly.
+```
+
+<!-- Below this line, tell us what you're doing to reproduce the problem. -->
+
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Matrix Server:**
+ - OS: [e.g. Ubuntu 21.04]
+ - Architecture [e.g. amd64, arm32, arm64]
+
+**Ansible:**
+If your problem appears to be with Ansible, tell us:
+- where you run Ansible -- e.g. on the Matrix server itself; on another computer (which OS? distro? standard installation or containerized Ansible?)
+- what version of Ansible you're running (see `ansible --version`)
+
+<!--
+The above is only applicable if you're hitting a problem with Ansible itself.
+We don't need this information in most cases. Delete this section if not applicable.
+-->
+
+**Client:**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+<!--
+The above is only applicable if you're hitting a problem with a specific device, but not with others.
+We don't need this information in most cases. Delete this section if not applicable.
+-->
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,28 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+<!--
+NOTE: When submitting feature requests, be aware that:
+
+- This Ansible playbook installs tens of separate services. If you're having a problem with a specific service or you'd like some functionality added to it, it is likely that the problem is not with our deployment method, but with the service itself. You may wish to report that problem at the source, upstream, and not to us.
+
+- This is a community project with no financial backing. The easiest way to get a feature into this project is to just develop it yourself.
+-->
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/i-need-help.md

@@ -0,0 +1,49 @@
+---
+name: I need help
+about: Get support from our community
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!--
+NOTE: you can usually get more timely support and from more people by joining our Matrix room (also bridged to IRC). See the [Support section of our README](https://github.com/spantaleev/matrix-docker-ansible-deploy#support)
+-->
+
+**Playbook Configuration**:
+
+My `vars.yml` file looks like this:
+
+```yaml
+Paste your vars.yml file here.
+Make sure to remove any secret values before posting your vars.yml file publicly.
+```
+
+**Matrix Server:**
+ - OS: [e.g. Ubuntu 21.04]
+ - Architecture [e.g. amd64, arm32, arm64]
+
+**Ansible:**
+If your problem appears to be with Ansible, tell us:
+- where you run Ansible -- e.g. on the Matrix server itself; on another computer (which OS? distro? standard installation or containerized Ansible?)
+- what version of Ansible you're running (see `ansible --version`)
+
+**Problem description**:
+
+Describe what you're doing, what you expect to happen and what happens instead here.
+Tell us what you've tried and what you're aiming to achieve.
+
+**Client (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+<!--
+The above is only applicable if you're hitting a problem with a specific device, but not with others.
+We don't need this information in most cases. Delete this section if not applicable.
+-->
+
+**Additional context**
+Add any other context about the problem here.

README.md

@@ -152,5 +152,3 @@ When updating the playbook, refer to [the changelog](CHANGELOG.md) to catch up w
 ## Services by the community
 
 - [etke.cc](https://etke.cc) - matrix-docker-ansible-deploy and system stuff "as a service". That service will create your matrix homeserver on your domain and server (doesn't matter if it's cloud provider or on an old laptop in the corner of your room), (optional) maintains it (server's system updates, cleanup, security adjustments, tuning, etc.; matrix homeserver updates & maintenance) and (optional) provide full-featured email service for your domain
-
-- [GoMatrixHosting](https://gomatrixhosting.com) - matrix-docker-ansible-deploy "as a service" with [Ansible AWX](https://github.com/ansible/awx). Members can be assigned a server from DigitalOcean, or they can connect their on-premises server. This AWX system can manage the updates, configuration, import and export, backups, and monitoring on its own. For more information [see our GitLab group](https://gitlab.com/GoMatrixHosting) or come [visit us on Matrix](https://matrix.to/#/#general:gomatrixhosting.com).

docs/configuring-awx-system.md

@@ -10,6 +10,7 @@ The AWX system is arranged into 'members' each with their own 'subscriptions'. A
 
 This system can manage the updates, configuration, import and export, backups and monitoring on its own. It is an extension of the popular deploy script [spantaleev/matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy).
 
+Warning: This system is about to undergo heavy revision, **we do not recommend using it at this time.**
 
 ## Other Required Playbooks
 

roles/matrix-awx/surveys/configure_mjolnir.json.j2

@@ -0,0 +1,29 @@
+{
+  "name": "Configure Mjolnir",
+  "description": "Configure Mjolnir settings, Mjolnir is a moderation bot for Matrix.",
+  "spec": [
+    {
+      "question_name": "Enable Mjolnir",
+      "question_description": "Set if Mjolnir is enabled or not. Mjolnir is a moderation bot for Matrix.",
+      "required": true,
+      "min": null,
+      "max": null,
+      "default": "{{ matrix_bot_mjolnir_enabled | string | lower }}",
+      "choices": "true\nfalse",
+      "new_question": true,
+      "variable": "matrix_bot_mjolnir_enabled",
+      "type": "multiplechoice"
+    },
+    {
+      "question_name": "Mjolnir Management Room",
+      "question_description": "Sets the internal ID of the management room for Mjolnir. Example: '!wAeZaPCKvaCHcSqxAW:matrix.org'",
+      "required": true,
+      "min": null,
+      "max": null,
+      "default": "{{ matrix_bot_mjolnir_management_room }}",
+      "new_question": true,
+      "variable": "matrix_bot_mjolnir_management_room",
+      "type": "text"
+    }
+  ]
+}

roles/matrix-awx/tasks/main.yml

@@ -170,6 +170,15 @@
   tags:
     - setup-ma1sd
 
+# Additional playbook to set the variable file during Mjolnir Bot configuration
+- include_tasks:
+    file: "set_variables_mjolnir.yml"
+    apply:
+      tags: setup-bot-mjolnir
+  when: run_setup|bool and matrix_awx_enabled|bool
+  tags:
+    - setup-bot-mjolnir
+
 # Additional playbook to set the variable file during Corporal configuration
 - include_tasks:
     file: "set_variables_corporal.yml"

roles/matrix-awx/tasks/set_variables_dimension.yml

@@ -14,7 +14,7 @@
 
 - name: Collect access token of @admin-dimension user
   shell: |
-    curl -X POST --header 'Content-Type: application/json' -d '{"identifier": {"type": "m.id.user","user": "admin-dimension"}, "password": "{{ awx_dimension_user_password }}", "type": "m.login.password"}' 'https://matrix.{{ matrix_domain }}/_matrix/client/r0/login' | jq -c '. | {access_token}' | sed 's/.*\":\"//' | sed 's/\"}//'
+    curl -X POST --header 'Content-Type: application/json' -d '{"identifier": {"type": "m.id.user","user": "admin-dimension"}, "password": "{{ awx_dimension_user_password }}", "type": "m.login.password"}' 'https://matrix.{{ matrix_domain }}/_matrix/client/r0/login' | jq '.access_token'
   register: awx_dimension_user_access_token
 
 - name: Record Synapse variables locally on AWX
@@ -26,7 +26,7 @@
     insertafter: '# Dimension Settings Start'
   with_dict:
     'matrix_dimension_enabled': '{{ matrix_dimension_enabled }}'
-    'matrix_dimension_access_token': '"{{ awx_dimension_user_access_token.stdout }}"'
+    'matrix_dimension_access_token': '"{{ awx_dimension_user_access_token.stdout[1:-1] }}"'
 
 - name: Set final users list if users are defined
   set_fact:

roles/matrix-awx/tasks/set_variables_ma1sd.yml

@@ -38,22 +38,22 @@
   replace:
     path: '{{ awx_cached_matrix_vars }}'
     regexp: '^.*\n'
-    after: '# Start ma1sd Extension'
-    before: '# End ma1sd Extension'
+    after: '# ma1sd Extension Start'
+    before: '# ma1sd Extension End'
 
 - name: Replace conjoined ma1sd configuration extension limiters
   delegate_to: 127.0.0.1
   replace:
     path: '{{ awx_cached_matrix_vars }}'
-    regexp: '^# Start ma1sd Extension# End ma1sd Extension'
-    replace: '# Start ma1sd Extension\n# End ma1sd Extension'
+    regexp: '^# ma1sd Extension Start# ma1sd Extension End'
+    replace: '# ma1sd Extension Start\n# ma1sd Extension End'
 
 - name: Insert/Update ma1sd configuration extension variables
   delegate_to: 127.0.0.1
   blockinfile:
     path: '{{ awx_cached_matrix_vars }}'
     marker: "# {mark} ma1sd ANSIBLE MANAGED BLOCK"
-    insertafter: '# Start ma1sd Extension'
+    insertafter: '# ma1sd Extension Start'
     block: '{{ awx_matrix_ma1sd_configuration_extension_yaml }}'
 
 - name: Record ma1sd Custom variables locally on AWX

roles/matrix-awx/tasks/set_variables_mjolnir.yml

@@ -0,0 +1,68 @@
+---
+
+- name: Include vars in matrix_vars.yml
+  include_vars:
+    file: '{{ awx_cached_matrix_vars }}'
+  no_log: true
+
+- name: Collect the internal IP of the matrix-synapse container
+  shell: |
+    /usr/bin/docker inspect --format '{''{range.NetworkSettings.Networks}''}{''{.IPAddress}''}{''{end}''}' matrix-synapse
+  register: matrix_synapse_ip
+
+- name: Collect access token of @admin-mjolnir user
+  shell: |
+    curl -X POST --header 'Content-Type: application/json' -d '{"identifier": {"type": "m.id.user","user": "admin-mjolnir"}, "password": "{{ awx_mjolnir_user_password }}", "type": "m.login.password"}' 'http://{{ matrix_synapse_ip.stdout }}:8008/_matrix/client/r0/login' | jq '.access_token'
+  register: awx_mjolnir_user_access_token
+  no_log: true
+
+- name: Record Mjolnir Bot variables locally on AWX
+  delegate_to: 127.0.0.1
+  lineinfile:
+    path: '{{ awx_cached_matrix_vars }}'
+    regexp: "^#? *{{ item.key | regex_escape() }}:"
+    line: "{{ item.key }}: {{ item.value }}"
+    insertafter: '# Mjolnir Settings Start'
+  with_dict:
+    'matrix_bot_mjolnir_enabled': '{{ matrix_bot_mjolnir_enabled }}'
+    'matrix_bot_mjolnir_access_token': '{{ awx_mjolnir_user_access_token.stdout[1:-1] }}'
+    'matrix_bot_mjolnir_management_room': '"{{ matrix_bot_mjolnir_management_room }}"'
+  no_log: true
+
+- name: Remove Synapse rate-limiting for admin-mjolnir user
+  shell: |
+    /usr/local/bin/matrix-postgres-cli-non-interactive --dbname=synapse --command="INSERT INTO ratelimit_override VALUES ('@admin-mjolnir:{{ matrix_domain }}', 0, 0);"
+  ignore_errors: true
+
+- name: Save new 'Configure Mjolnir' survey.json to the AWX tower, template
+  delegate_to: 127.0.0.1
+  template:
+    src: 'roles/matrix-awx/surveys/configure_mjolnir.json.j2'
+    dest: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/configure_mjolnir.json'
+
+- name: Copy new 'Configure Mjolnir' survey.json to target machine
+  copy:
+    src: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/configure_mjolnir.json'
+    dest: '/matrix/awx/configure_mjolnir.json'
+    mode: '0660'
+
+- name: Recreate 'Configure Mjolnir Bot' job template
+  delegate_to: 127.0.0.1
+  awx.awx.tower_job_template:
+    name: "{{ matrix_domain }} - 1 - Configure Mjolnir Bot"
+    description: "Configure Mjolnir settings, Mjolnir is a moderation bot for Matrix."
+    extra_vars: "{{ lookup('file', '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/extra_vars.json') }}"
+    job_type: run
+    job_tags: "start,setup-bot-mjolnir"
+    inventory: "{{ member_id }}"
+    project: "{{ member_id }} - Matrix Docker Ansible Deploy"
+    playbook: setup.yml
+    credential: "{{ member_id }} - AWX SSH Key"
+    survey_enabled: true
+    survey_spec: "{{ lookup('file', '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/configure_mjolnir.json') }}"
+    become_enabled: true
+    state: present
+    verbosity: 1
+    tower_host: "https://{{ awx_host }}"
+    tower_oauthtoken: "{{ awx_session_token.ansible_facts.tower_token.token }}"
+    validate_certs: true

roles/matrix-awx/tasks/update_variables.yml

@@ -7,15 +7,21 @@
     regexp: 'matrix_synapse_use_presence'
     replace: 'matrix_synapse_presence_enabled'
 
-- name: Generate matrix_homeserver_generic_secret_key variable
+- name: Search for matrix_homeserver_generic_secret_key variable in matrix_vars.yml
+  delegate_to: 127.0.0.1
+  register: presence
+  shell: "grep -i 'matrix_homeserver_generic_secret_key' /var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml"
+  no_log: true
+
+- name: Generate matrix_homeserver_generic_secret_key variable if not present
   delegate_to: 127.0.0.1
   command: |
       openssl rand -hex 16
   register: generic_secret
   no_log: true
-  when: ( matrix_homeserver_generic_secret_key is undefined ) or ( matrix_homeserver_generic_secret_key | length == 0 )
+  when: presence is not changed
 
-- name: Add new matrix_homeserver_generic_secret_key variable
+- name: Add new matrix_homeserver_generic_secret_key variable if not present
   delegate_to: 127.0.0.1
   lineinfile:
     path: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
@@ -23,4 +29,4 @@
     insertbefore: '# Basic Settings End'
     mode: '0600'
     state: present
-  when: ( matrix_homeserver_generic_secret_key is undefined ) or ( matrix_homeserver_generic_secret_key | length == 0 )
+  when: presence is not changed

roles/matrix-bot-honoroit/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_bot_honoroit_container_image_self_build: false
 matrix_bot_honoroit_docker_repo: "https://gitlab.com/etke.cc/honoroit.git"
 matrix_bot_honoroit_docker_src_files_path: "{{ matrix_base_data_path }}/honoroit/docker-src"
 
-matrix_bot_honoroit_version: v0.9.4
+matrix_bot_honoroit_version: v0.9.5
 matrix_bot_honoroit_docker_image: "{{ matrix_bot_honoroit_docker_image_name_prefix }}honoroit:{{ matrix_bot_honoroit_version }}"
 matrix_bot_honoroit_docker_image_name_prefix: "{{ 'localhost/' if matrix_bot_honoroit_container_image_self_build else 'registry.gitlab.com/etke.cc/' }}"
 matrix_bot_honoroit_docker_image_force_pull: "{{ matrix_bot_honoroit_docker_image.endswith(':latest') }}"
@@ -84,6 +84,9 @@ matrix_bot_honoroit_sentry: ''
 # Log level
 matrix_bot_honoroit_loglevel: ''
 
+# Max items in cache
+matrix_bot_honoroit_cachesize: ''
+
 # Text prefix: open
 matrix_bot_honoroit_text_prefix_open: ''
 

roles/matrix-bot-honoroit/templates/env.j2

@@ -7,6 +7,7 @@ HONOROIT_DB_DIALECT={{ matrix_bot_honoroit_database_dialect }}
 HONOROIT_PREFIX={{ matrix_bot_honoroit_prefix }}
 HONOROIT_SENTRY={{ matrix_bot_honoroit_sentry }}
 HONOROIT_LOGLEVEL={{ matrix_bot_honoroit_loglevel }}
+HONOROIT_CACHESIZE={{ matrix_bot_honoroit_cachesize }}
 HONOROIT_TEXT_PREFIX_OPEN={{ matrix_bot_honoroit_text_prefix_open }}
 HONOROIT_TEXT_PREFIX_DONE={{ matrix_bot_honoroit_text_prefix_done }}
 HONOROIT_TEXT_GREETINGS={{ matrix_bot_honoroit_text_greetings }}

roles/matrix-client-element/defaults/main.yml

@@ -9,7 +9,7 @@ matrix_client_element_container_image_self_build_repo: "https://github.com/vecto
 # - https://github.com/vector-im/element-web/issues/19544
 matrix_client_element_container_image_self_build_low_memory_system_patch_enabled: "{{ ansible_memtotal_mb < 4096 }}"
 
-matrix_client_element_version: v1.10.4
+matrix_client_element_version: v1.10.6
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"

roles/matrix-jitsi/defaults/main.yml

@@ -70,7 +70,7 @@ matrix_jitsi_jibri_recorder_password: ''
 
 matrix_jitsi_enable_lobby: false
 
-matrix_jitsi_version: stable-6726-2
+matrix_jitsi_version: stable-6865
 matrix_jitsi_container_image_tag: "{{ matrix_jitsi_version }}"  # for backward-compatibility
 
 matrix_jitsi_web_docker_image: "{{ matrix_container_global_registry_prefix }}jitsi/web:{{ matrix_jitsi_container_image_tag }}"

roles/matrix-postgres/tasks/setup_postgres.yml

@@ -85,6 +85,13 @@
     mode: 0755
   when: matrix_postgres_enabled|bool
 
+- name: Ensure matrix-postgres-cli-non-interactive script created
+  template:
+    src: "{{ role_path }}/templates/usr-local-bin/matrix-postgres-cli-non-interactive.j2"
+    dest: "{{ matrix_local_bin_path }}/matrix-postgres-cli-non-interactive"
+    mode: 0755
+  when: matrix_postgres_enabled|bool
+
 - name: Ensure matrix-change-user-admin-status script created
   template:
     src: "{{ role_path }}/templates/usr-local-bin/matrix-change-user-admin-status.j2"

roles/matrix-postgres/templates/usr-local-bin/matrix-postgres-cli-non-interactive.j2

@@ -0,0 +1,12 @@
+#jinja2: lstrip_blocks: "True"
+#!/bin/bash
+
+docker run \
+	--rm \
+	--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+	--cap-drop=ALL \
+	--env-file={{ matrix_postgres_base_path }}/env-postgres-psql \
+	--network {{ matrix_docker_network }} \
+	{{ matrix_postgres_docker_image_to_use }} \
+	psql -h {{ matrix_postgres_connection_hostname }} \
+	"$@"