Add matrix-user-creator role - automatic user account creation support

We no longer ask users to create Matrix user accounts for these bots:

- Postmoogle
- Honoroit
- Reminder Bot

Other bots and services (matrix-registration-bot, maubot, mjolnir,
Dimension, etc.) require an Access Token to run (not a password),
so this new role doesn't help for them.

It does help for the above bots though, and for defining your own
"initial user accounts" in the `matrix_user_creator_users_additional`
variable.

docs/configuring-playbook-bot-honoroit.md

@@ -7,21 +7,6 @@ It's a bot you can use to setup **your own helpdesk on matrix**
 See the project's [documentation](https://gitlab.com/etke.cc/honoroit#how-it-looks-like) to learn what it does with screenshots and why it might be useful to you.
 
 
-## Registering the bot user
-
-By default, the playbook will set up the bot with a username like this: `@honoroit:DOMAIN`.
-
-(to use a different username, adjust the `matrix_bot_honoroit_login` variable).
-
-You **need to register the bot user manually** before setting up the bot. You can use the playbook to [register a new user](registering-users.md):
-
-```
-ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=honoroit password=PASSWORD_FOR_THE_BOT admin=no' --tags=register-user
-```
-
-Choose a strong password for the bot. You can generate a good password with a command like this: `pwgen -s 64 1`.
-
-
 ## Adjusting the playbook configuration
 
 Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file:
@@ -29,7 +14,10 @@ Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.
 ```yaml
 matrix_bot_honoroit_enabled: true
 
-# Adjust this to whatever password you chose when registering the bot user
+# Uncomment and adjust this part if you'd like to use a username different than the default
+# matrix_bot_honoroit_login: honoroit
+
+# Generate a strong password here. Consider generating it with `pwgen -s 64 1`
 matrix_bot_honoroit_password: PASSWORD_FOR_THE_BOT
 
 # Adjust this to your room ID
@@ -41,9 +29,15 @@ matrix_bot_honoroit_roomid: "!yourRoomID:DOMAIN"
 
 After configuring the playbook, run the [installation](installing.md) command again:
 
+```sh
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
 ```
-ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+
-```
+**Notes**:
+
+- the `ensure-matrix-users-created` playbook tag makes the playbook automatically create the bot's user account
+
+- if you change the bot password (`matrix_bot_honoroit_password` in your `vars.yml` file) subsequently, the bot user's credentials on the homeserver won't be updated automatically. If you'd like to change the bot user's password, use a tool like [synapse-admin](configuring-playbook-synapse-admin.md) to change it, and then update `matrix_bot_honoroit_password` to let the bot know its new password
 
 
 ## Usage

docs/configuring-playbook-bot-matrix-registration-bot.md

@@ -16,7 +16,7 @@ By default, the playbook will set use the bot with a username like this: `@bot.m
 
 (to use a different username, adjust the `matrix_bot_matrix_registration_bot_matrix_user_id_localpart` variable).
 
-You **need to register the bot user manually** before setting up the bot. You can use the playbook to [register a new user](registering-users.md):
+For [other bots supported by the playbook](configuring-playbook.md#bots), Matrix bot user accounts are created and put to use automatically. For `matrix-registration-bot`, however, this is not the case - you **need to register the bot user manually** before setting up the bot. You can use the playbook to [register a new user](registering-users.md):
 
 ```
 ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=bot.matrix-registration-bot password=PASSWORD_FOR_THE_BOT admin=yes' --tags=register-user

docs/configuring-playbook-bot-matrix-reminder-bot.md

@@ -7,21 +7,6 @@ It's a bot you can use to **schedule one-off & recurring reminders and alarms**.
 See the project's [documentation](https://github.com/anoadragon453/matrix-reminder-bot#usage) to learn what it does and why it might be useful to you.
 
 
-## Registering the bot user
-
-By default, the playbook will set up the bot with a username like this: `@bot.matrix-reminder-bot:DOMAIN`.
-
-(to use a different username, adjust the `matrix_bot_matrix_reminder_bot_matrix_user_id_localpart` variable).
-
-You **need to register the bot user manually** before setting up the bot. You can use the playbook to [register a new user](registering-users.md):
-
-```
-ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=bot.matrix-reminder-bot password=PASSWORD_FOR_THE_BOT admin=no' --tags=register-user
-```
-
-Choose a strong password for the bot. You can generate a good password with a command like this: `pwgen -s 64 1`.
-
-
 ## Adjusting the playbook configuration
 
 Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file:
@@ -29,7 +14,10 @@ Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.
 ```yaml
 matrix_bot_matrix_reminder_bot_enabled: true
 
-# Adjust this to whatever password you chose when registering the bot user
+# Uncomment and adjust this part if you'd like to use a username different than the default
+# matrix_bot_matrix_reminder_bot_matrix_user_id_localpart: bot.matrix-reminder-bot
+
+# Generate a strong password here. Consider generating it with `pwgen -s 64 1`
 matrix_bot_matrix_reminder_bot_matrix_user_password: PASSWORD_FOR_THE_BOT
 
 # Adjust this to your timezone
@@ -41,9 +29,15 @@ matrix_bot_matrix_reminder_bot_reminders_timezone: Europe/London
 
 After configuring the playbook, run the [installation](installing.md) command again:
 
+```sh
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
 ```
-ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+
-```
+**Notes**:
+
+- the `ensure-matrix-users-created` playbook tag makes the playbook automatically create the bot's user account
+
+- if you change the bot password (`matrix_bot_matrix_reminder_bot_matrix_user_password` in your `vars.yml` file) subsequently, the bot user's credentials on the homeserver won't be updated automatically. If you'd like to change the bot user's password, use a tool like [synapse-admin](configuring-playbook-synapse-admin.md) to change it, and then update `matrix_bot_matrix_reminder_bot_matrix_user_password` to let the bot know its new password
 
 
 ## Usage

docs/configuring-playbook-bot-postmoogle.md

@@ -9,21 +9,6 @@ It's a bot/bridge you can use to forward emails to Matrix rooms
 See the project's [documentation](https://gitlab.com/etke.cc/postmoogle) to learn what it does and why it might be useful to you.
 
 
-## Registering the bot user
-
-By default, the playbook will set up the bot with a username like this: `@postmoogle:DOMAIN`.
-
-(to use a different username, adjust the `matrix_bot_postmoogle_login` variable).
-
-You **need to register the bot user manually** before setting up the bot. You can use the playbook to [register a new user](registering-users.md):
-
-```
-ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=postmoogle password=PASSWORD_FOR_THE_BOT admin=no' --tags=register-user
-```
-
-Choose a strong password for the bot. You can generate a good password with a command like this: `pwgen -s 64 1`.
-
-
 ## Adjusting the playbook configuration
 
 Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file:
@@ -31,7 +16,10 @@ Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.
 ```yaml
 matrix_bot_postmoogle_enabled: true
 
-# Adjust this to whatever password you chose when registering the bot user
+# Uncomment and adjust this part if you'd like to use a username different than the default
+# matrix_bot_postmoogle_login: postmoogle
+
+# Generate a strong password here. Consider generating it with `pwgen -s 64 1`
 matrix_bot_postmoogle_password: PASSWORD_FOR_THE_BOT
 ```
 
@@ -43,9 +31,15 @@ See [Configuring DNS](configuring-dns.md).
 
 After configuring the playbook, run the [installation](installing.md) command again:
 
+```sh
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
 ```
-ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+
-```
+**Notes**:
+
+- the `ensure-matrix-users-created` playbook tag makes the playbook automatically create the bot's user account
+
+- if you change the bot password (`matrix_bot_postmoogle_password` in your `vars.yml` file) subsequently, the bot user's credentials on the homeserver won't be updated automatically. If you'd like to change the bot user's password, use a tool like [synapse-admin](configuring-playbook-synapse-admin.md) to change it, and then update `matrix_bot_postmoogle_password` to let the bot know its new password
 
 
 ## Usage

docs/installing.md

@@ -1,25 +1,64 @@
 # Installing
 
-## 1. Installing the Matrix services
-
 If you've [configured your DNS](configuring-dns.md) and have [configured the playbook](configuring-playbook.md), you can start the installation procedure.
 
-Run this command to install the Matrix services:
+## Playbook tags introduction
 
-```bash
+The Ansible playbook's tasks are tagged, so that certain parts of the Ansible playbook can be run without running all other tasks.
+
+The general command syntax is: `ansible-playbook -i inventory/hosts setup.yml --tags=COMMA_SEPARATED_TAGS_GO_HERE`
+
+Here are some playbook tags that you should be familiar with:
+
+- `setup-all` - runs all setup tasks for all components, but does not start/restart services
+
+- `setup-SERVICE` (e.g. `setup-bot-postmoogle`) - runs the setup tasks only for a given role, but does not start/restart services. You can discover these additional tags in each role (`roles/*/main.yml`). Running per-component setup tasks is **not recommended**, as components sometimes depend on each other and running just the setup tasks for a given component may not be enough. For example, setting up the [mautrix-telegram bridge](configuring-playbook-bridge-mautrix-telegram.md), in addition to the `setup-mautrix-telegram` tag, requires database changes (the `setup-postgres` tag) as well as reverse-proxy changes (the `setup-nginx-proxy` tag).
+
+- `start` - starts all systemd services and makes them start automatically in the future
+
+- `stop` - stops all systemd services
+
+- `ensure-matrix-users-created` - a special tag which ensures that all special users needed by the playbook (for bots, etc.) are created
+
+`setup-*` tags **do not start services** automatically, because you may wish to do things before starting services, such as importing a database dump, restoring data from another server, etc.
+
+
+## 1. Installing Matrix
+
+If you **don't** use SSH keys for authentication, but rather a regular password, you may need to add `--ask-pass` to the all Ansible commands
+
+If you **do** use SSH keys for authentication, **and** use a non-root user to *become* root (sudo), you may need to add `-K` (`--ask-become-pass`) to all Ansible commands
+
+There 2 ways to start the installation process - depending on whether you're [Installing a brand new server (without importing data)](#installing-a-brand-new-server-without-importing-data) or [Installing a server into which you'll import old data](#installing-a-server-into-which-youll-import-old-data).
+
+
+### Installing a brand new server (without importing data)
+
+If this is **a brand new** Matrix server and you **won't be importing old data into it**, run all these tags:
+
+```sh
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
+```
+
+This will do a full installation and start all Matrix services.
+
+Proceed to [Maintaining your setup in the future](#2-maintaining-your-setup-in-the-future) and [Finalize the installation](#3-finalize-the-installation)
+
+
+### Installing a server into which you'll import old data
+
+If you will be importing data into your newly created Matrix server, install it, but **do not** start its services just yet.
+Starting its services or messing with its database now will affect your data import later on.
+
+To do the installation **without** starting services, run only the `setup-all` tag:
+
+```sh
 ansible-playbook -i inventory/hosts setup.yml --tags=setup-all
 ```
 
-The above command **doesn't start any services just yet** (another step does this later - below). Feel free to **re-run this setup command any time** you think something is off with the server configuration.
+When this command completes, services won't be running yet.
 
-**Notes**:
+You can now:
-- if you **don't** use SSH keys for authentication, but rather a regular password, you may need to add `--ask-pass` to the above (and all other) Ansible commands.
-- if you **do** use SSH keys for authentication, **and** use a non-root user to *become* root (sudo), you may need to add `-K` (`--ask-become-pass`) to the above (and all other) Ansible commands.
-
-
-## 2. Things you might want to do after installing
-
-**Before starting the services**, you may want to do additional things like:
 
 - [Importing an existing SQLite database (from another Synapse installation)](importing-synapse-sqlite.md) (optional)
 
@@ -27,21 +66,26 @@ The above command **doesn't start any services just yet** (another step does thi
 
 - [Importing `media_store` data files from an existing Synapse installation](importing-synapse-media-store.md) (optional)
 
+.. and then proceed to starting all services:
 
-## 3. Starting the services
+```sh
-
-When you're ready to start the Matrix services (and set them up to auto-start in the future), run this command:
-
-```bash
 ansible-playbook -i inventory/hosts setup.yml --tags=start
 ```
 
-## 4. Finalize the installation
+Proceed to [Maintaining your setup in the future](#2-maintaining-your-setup-in-the-future) and [Finalize the installation](#3-finalize-the-installation)
+
+
+## 2. Maintaining your setup in the future
+
+Feel free to **re-run the setup command any time** you think something is off with the server configuration. Ansible will take your configuration and update your server to match.
+
+
+## 3. Finalize the installation
 
 Now that services are running, you need to **finalize the installation process** (required for federation to work!) by [Configuring Service Discovery via .well-known](configuring-well-known.md).
 
 
-## 5. Things to do next
+## 4. Things to do next
 
 After you have started the services and **finalized the installation process** (required for federation to work!) by [Configuring Service Discovery via .well-known](configuring-well-known.md), you can:
 

group_vars/matrix_servers

@@ -2677,3 +2677,37 @@ matrix_conduit_systemd_required_services_list: |
 # /matrix-conduit
 #
 ######################################################################
+
+
+######################################################################
+#
+# matrix-user-creator
+#
+######################################################################
+
+matrix_user_creator_users_auto: |
+  {{
+    [{
+      'username': matrix_bot_matrix_reminder_bot_matrix_user_id_localpart,
+      'initial_password': matrix_bot_matrix_reminder_bot_matrix_user_password,
+      'initial_type': 'bot',
+    }] if matrix_bot_matrix_reminder_bot_enabled else []
+    +
+    [{
+      'username': matrix_bot_honoroit_login,
+      'initial_password': matrix_bot_honoroit_password,
+      'initial_type': 'bot',
+    }] if matrix_bot_honoroit_enabled else []
+    +
+    [{
+      'username': matrix_bot_postmoogle_login,
+      'initial_password': matrix_bot_postmoogle_password,
+      'initial_type': 'bot',
+    }] if matrix_bot_postmoogle_enabled else []
+  }}
+
+######################################################################
+#
+# /matrix-user-creator
+#
+######################################################################

roles/matrix-user-creator/defaults/main.yml

@@ -0,0 +1,22 @@
+---
+
+# matrix-user-creator is a role that aims to automate initial Matrix user account creation.
+#
+# This role only supports initial user account creation and will not manage subsequent user-type changes
+# or password changes.
+#
+# The playbook registers various bot user accounts automatically using this role by injecting
+# user creation definitions into the `matrix_user_creator_users_auto` variable.
+#
+# To get started creating your own Matrix user accounts, use the `matrix_user_creator_users_additional` variable.
+
+# matrix_user_creator_users_auto holds a list of users that should be created on the Matrix homeserver.
+# The playbook adds some user definitions here to have them end up in `matrix_user_creator_users` (see `vars/main.yml`) and get automatically created.
+# This value is influenced by the playbook and will be overwritten elsewhere (`group_vars/`, etc.)
+# To add your own user definitions, use the `matrix_user_creator_users_additional` variable.
+matrix_user_creator_users_auto: []
+
+# matrix_user_creator_users_additional holds a list of users that should be created on the Matrix homeserver.
+# Add your own users here to have them end up in `matrix_user_creator_users` (see `vars/main.yml`) and get automatically created.
+# For example syntax for this variable, see the documentation for `matrix_user_creator_users` in `vars/main.yml`.
+matrix_user_creator_users_additional: []

roles/matrix-user-creator/tasks/main.yml

@@ -0,0 +1,9 @@
+---
+
+- when: matrix_user_creator_users | length > 0
+  ansible.builtin.import_tasks: "{{ role_path }}/tasks/setup.yml"
+  tags:
+    # This role intentionally doesn't do work on a `setup-all` tag.
+    # If it did, the initial installation (`--tags=setup-all`) would also potentially polute the database with data,
+    # which would make importing a database dump problematic.
+    - ensure-matrix-users-created

roles/matrix-user-creator/tasks/setup.yml

@@ -0,0 +1,33 @@
+---
+
+- name: Validate Matrix users to create
+  ansible.builtin.include_tasks: "{{ role_path }}/tasks/util/validate_user.yml"
+  with_items: "{{ matrix_user_creator_users }}"
+  loop_control:
+    loop_var: user
+  # Suppress logging to avoid dumping the credentials to the shell
+  no_log: true
+
+- name: Ensure systemd is reloaded before starting the homeserver
+  ansible.builtin.service:
+    daemon_reload: true
+
+- name: Ensure homeserver is started before creating Matrix users
+  ansible.builtin.service:
+    name: "matrix-{{ matrix_homeserver_implementation }}.service"
+    state: started
+    daemon_reload: true
+  register: matrix_user_registrator_homeserver_start_result
+
+- name: Wait a while, so that the homeserver can manage to start before creating Matrix users
+  ansible.builtin.pause:
+    seconds: 7
+  when: matrix_user_registrator_homeserver_start_result.changed | bool
+
+- name: Ensure Matrix users are created
+  ansible.builtin.include_tasks: "{{ role_path }}/tasks/util/ensure_user_registered_{{ matrix_homeserver_implementation }}.yml"
+  with_items: "{{ matrix_user_creator_users }}"
+  loop_control:
+    loop_var: user
+  # Suppress logging to avoid dumping the credentials to the shell
+  no_log: true

roles/matrix-user-creator/tasks/util/ensure_user_registered_conduit.yml

@@ -0,0 +1,5 @@
+---
+
+- name: Ensure Conduit user registered - {{ user.username | quote }}
+  ansible.builtin.debug:
+    msg: "Not registering user. To register Conduit users, message the Conduit bot"

roles/matrix-user-creator/tasks/util/ensure_user_registered_dendrite.yml

@@ -0,0 +1,17 @@
+---
+
+- name: Ensure Dendrite user registered - {{ user.username | quote }}
+  ansible.builtin.command:
+    cmd: |-
+      {{ matrix_host_command_docker }} exec matrix-dendrite
+      create-account
+      -config /data/dendrite.yaml
+      -username {{ user.username | quote }}
+      -password {{ user.initial_password | quote }}
+      {% if user.initial_type == 'admin' %}
+        -admin
+      {% endif %}
+      -url http://localhost:{{ matrix_dendrite_http_bind_port }}
+  register: matrix_dendrite_register_user_result
+  changed_when: matrix_dendrite_register_user_result.rc == 0 and 'Desired user ID is already taken' not in matrix_dendrite_register_user_result.stderr
+  failed_when: matrix_dendrite_register_user_result.rc != 0 and 'Desired user ID is already taken' not in matrix_dendrite_register_user_result.stderr

roles/matrix-user-creator/tasks/util/ensure_user_registered_synapse.yml

@@ -0,0 +1,22 @@
+---
+
+- name: Ensure Synapse user registered - {{ user.username | quote }}
+  ansible.builtin.command:
+    cmd: |-
+      {{ matrix_host_command_docker }} exec matrix-synapse
+      register_new_matrix_user
+      -u  {{ user.username | quote }}
+      -p {{ user.initial_password | quote }}
+      -c /data/homeserver.yaml
+      {% if user.initial_type == 'admin' %}
+        --admin
+      {% else %}
+        --no-admin
+        {% if user.initial_type != 'user' %}
+          --user_type={{ user.initial_type | quote }}
+        {% endif %}
+      {% endif %}
+      http://localhost:{{ matrix_synapse_container_client_api_port }}
+  register: matrix_synapse_register_user_result
+  changed_when: matrix_synapse_register_user_result.rc == 0 and 'User ID already taken' not in matrix_synapse_register_user_result.stdout
+  failed_when: matrix_synapse_register_user_result.rc != 0 and 'User ID already taken' not in matrix_synapse_register_user_result.stdout

roles/matrix-user-creator/tasks/util/validate_user.yml

@@ -0,0 +1,16 @@
+---
+
+- name: Fail if invalid username
+  ansible.builtin.fail:
+    msg: "Empty usernames values are not allowed ({{ user }})"
+  when: not (user.username | default(''))
+
+- name: Fail if invalid initial_password for user - {{ user.username }}
+  ansible.builtin.fail:
+    msg: "Empty initial_password values are not allowed"
+  when: not (user.initial_password | default(''))
+
+- name: Fail if invalid initial_type for user - {{ user.username }}
+  ansible.builtin.fail:
+    msg: "User initial_type `{{ user.initial_type | default('undefined') }}` is not supported"
+  when: user.initial_type | default('undefined') not in ['admin', 'user', 'bot', 'support']

roles/matrix-user-creator/vars/main.yml

@@ -0,0 +1,34 @@
+---
+
+# matrix_user_creator_users holds a list of users that should be created on the Matrix homeserver.
+#
+# Removing a user from this list will not automatically delete/disable the Matrix user on the homeserver.
+#
+# As the `initial_password` / `initial_type` field names indicate, these are just initial values.
+# Changing the password or type values subsequently will not update the already existing user's details.
+#
+# The known user types are: 'admin', 'user', 'bot', 'support'.
+# These are inspired by Synapse's user types.
+# 'admin' and 'user' types are generally recognized across homeservers.
+# Other homeservers may not support 'bot' and 'support'. Such homeservers will fall back to whatever types they do support.
+#
+# Example:
+# matrix_user_creator_users:
+#   - username: root
+#     initial_password: some-password
+#     initial_type: admin
+#
+#   - username: john
+#     initial_password: some-password
+#     initial_type: user
+#
+#   - username: bot.matrix-reminder-bot
+#     initial_password: some-password
+#     initial_type: bot
+#
+#   - username: bot.matrix-reminder-bot
+#     initial_password: some-password
+#     initial_type: support
+#
+# To create you own users, use the `matrix_user_creator_users_additional` variable.
+matrix_user_creator_users: "{{ matrix_user_creator_users_auto + matrix_user_creator_users_additional }}"

setup.yml

@@ -72,4 +72,5 @@
     - matrix-aux
     - matrix-postgres-backup
     - matrix-backup-borg
+    - matrix-user-creator
     - matrix-common-after