Commit graph

6760 commits

Author SHA1 Message Date
Oliver Schmidt e362fe9c6d security/acme: limit concurrent certificate generations
fixes #232505

Implements the new option `security.acme.maxConcurrentRenewals` to limit
the number of certificate generation (or renewal) jobs that can run in
parallel. This avoids overloading the system resources with many
certificates or running into acme registry rate limits and network
timeouts.

Architecture considerations:
- simplicity, lightweight: Concerns have been voiced about making this
  already rather complex module even more convoluted. Additionally,
  locking solutions shall not significantly increase performance and
  footprint of individual job runs.
  To accomodate these concerns, this solution is implemented purely in
  Nix, bash, and using the light-weight `flock` util. To reduce
  complexity, jobs are already assigned their lockfile slot at system
  build time instead of dynamic locking and retrying. This comes at the
  cost of not always maxing out the permitted concurrency at runtime.
- no stale locks: Limiting concurrency via locking mechanism is usually
  approached with semaphores. Unfortunately, both SysV as well as
  POSIX-Semaphores are *not* released when the process currently locking
  them is SIGKILLed. This poses the danger of stale locks staying around
  and certificate renewal being blocked from running altogether.
  `flock` locks though are released when the process holding the file
  descriptor of the lock file is KILLed or terminated.
- lockfile generation: Lock files could either be created at build time
  in the Nix store or at script runtime in a idempotent manner.
  While the latter would be simpler to achieve, we might exceed the number
  of permitted concurrent runs during a system switch: Already running
  jobs are still locked on the existing lock files, while jobs started
  after the system switch will acquire locks on freshly created files,
  not being blocked by the still running services.
  For this reason, locks are generated and managed at runtime in the
  shared state directory `/var/lib/locks/`.

nixos/security/acme: move locks to /run

also, move over permission and directory management to systemd-tmpfiles

nixos/security/acme: fix some linter remarks in my code

there are some remarks left for existing code, not touching that

nixos/security/acme: redesign script locking flow

- get rid of subshell
- provide function for wrapping scripts in a locked environment

nixos/acme: improve visibility of blocking on locks

nixos/acme: add smoke test for concurrency limitation

heavily inspired by m1cr0man

nixos/acme: release notes entry on new concurrency limits

nixos/acme: cleanup, clarifications
2023-09-09 20:13:18 +02:00
ajs124 0a3aa06b53
Merge pull request #253739 from mweinelt/firefox-102-removal
firefox-esr-102-unwrapped: remove
2023-09-09 19:25:42 +02:00
Janik eda85eb31d
Merge pull request #251062 from ajs124/restic-wrapper-script 2023-09-09 19:11:33 +02:00
Pierre Bourdon 69defb96b5
nixosTests.sudo: use same maintainers as the package 2023-09-08 14:47:13 +02:00
Kerstin Humm 36ff7d5d5d mobilizon: init at 3.1.3
Co-Authored-By: Minijackson <minijackson@riseup.net>
Co-Authored-By: summersamara <summersamara@proton.me>
2023-09-07 08:59:40 +00:00
Martin Weinelt ba881ef613
firefox-esr-102-unwrapped: remove
The Firefox ESR 102.0 series has reached its end of life.

Removes package and test and references to them.
2023-09-07 01:17:27 +02:00
Weijia Wang 67160d51fe
Merge pull request #251987 from illdefined/akkoma
akkoma: 3.9.3 → 3.10.4
2023-09-06 16:03:52 +02:00
Weijia Wang cfb61436d7
Merge pull request #252978 from oluceps/dae-upup
dae,nixos/dae: 0.2.4 -> 0.3.0
2023-09-05 23:57:14 +02:00
Michele Guerini Rocco 704c791a90
Merge pull request #253146 from rnhmjoj/pr-anbox
anbox: unbreak
2023-09-05 23:46:54 +02:00
Adam Stephens c811cf643f nixos/tests/lxd: disable virtual-machine test on aarch64 2023-09-05 15:28:00 +00:00
Adam Stephens a90385c62b nixos/lxd: add preseed option 2023-09-05 15:28:00 +00:00
Mikael Voss b56f8149b2 nixos/tests/akkoma: Disable retrieving timeline 2023-09-05 12:26:33 +02:00
Matt Votava b3bb611ce6
nixos/tests/anbox: init 2023-09-05 10:09:45 +02:00
rnhmjoj 640c86c256
nixos/tests/jool: update for module changes 2023-09-04 18:38:25 +02:00
Martin Weinelt 390d535562
Merge pull request #253259 from mweinelt/custom-ca-firefox-memlimit
nixosTests.custom-ca: resolve out of memory situations
2023-09-04 15:05:54 +02:00
Martin Weinelt 311ce2ad11
nixosTests.custom-ca: resolve out of memory situations
They are easily observable on hydra. E.g. on the latest eval

https://hydra.nixos.org/build/233893887
https://hydra.nixos.org/build/233900101
2023-09-04 14:34:21 +02:00
Mario Rodas 4f2c276497
Merge pull request #244093 from adamcstephens/lxd/vm
lxd: Add VM image and server support for QEMU VMs
2023-09-03 22:02:54 -05:00
pacien c6808723b0 nixos/stalwart-mail: add vm test 2023-09-03 22:18:50 -04:00
Adam Stephens efd1605be6
nixos/lxd: add virtual-machine support, image and module 2023-09-03 20:06:44 -04:00
oluceps 17e386205b
nixos/dae: add basic test 2023-09-03 03:06:52 +08:00
Florian Klink 90204f058c
Merge pull request #251684 from jmbaur/user-activation-tmpfiles
nixos/systemd-user: call systemd-tmpfiles during activation
2023-09-01 21:52:19 +03:00
Martin Weinelt d17a82f986
Merge pull request #252041 from mweinelt/unbound-exporter
prometheus-unbound-exporter: replace at 0.4.4
2023-08-29 21:26:42 +02:00
Jared Baur e39c9f13bc
nixos/systemd-user: call systemd-tmpfiles during activation
Just like with system-wide tmpfiles, call `systemd-tmpfiles --create
--remove` for users during activation. This fixes an issue where new
entries in a user's tmpfiles are not reflected after activation, only at
boot when the user service systemd-tmpfiles-setup.service runs or only
after running systemd-tmpfiles manually.
2023-08-29 10:46:44 -07:00
Martin Weinelt 73c5a5a778
nixos/prometheus/unbound-exporter: update for new package
The command line interface changed slightly, but still supports the TCP
as well as the UDS control interface.
2023-08-28 23:21:52 +02:00
Jan Tojnar 434d160d7c
Merge pull request #234615 from linsui/dconf
nixos/dconf: support generating from attrs
2023-08-28 18:31:52 +02:00
ajs124 dbb69f82c6 nixos/restic: add wrapper scripts that set parameters for backup
and use in test
2023-08-28 15:17:37 +02:00
Pol Dellaiera df24943cc0
Merge pull request #249765 from NixOS/nixos/modules/honk-init
nixos/honk: init
2023-08-28 10:40:23 +02:00
Janne Heß fd454fe4b6
Merge pull request #250316 from helsinki-systems/feat/stc-improve-mount-units
nixos/switch-to-configuration: Improve mount unit handling
2023-08-28 09:18:44 +02:00
Maciej Krüger 311d2fa994
*: migrate to using nftables.tables instead of ruleset directly 2023-08-28 00:30:29 +02:00
Pol Dellaiera 48de649336
nixos/modules/honk: init 2023-08-27 23:43:40 +02:00
Theodore Ni c68a6a21ca
Merge pull request #247909 from Sigmanificient/ragnarwm
ragnarwm: init at 1.3.1
2023-08-27 13:41:04 -07:00
Maciej Krüger c3ed95307e
Merge pull request #251413 from karolinschlegel/init-odoo-v15
odoo15: init at 15.0-20230720
2023-08-27 14:10:04 +02:00
Karolin Schlegel 4eee509d25 odoo15: init at 15.0-20230720
This contribution adds `odoo15` as a fixed version. To allow reusing the
existing test for Odoo, I made the package attribute configurable. To
reference the test for `odoo15` in `passthru` I added it to
`all-tests.nix`.
2023-08-27 09:52:43 +02:00
Ryan Lahfa 72494b65bc
Merge pull request #219873 from r-ryantm/auto-update/dolibarr 2023-08-26 16:23:13 +02:00
Mario Rodas 913c460a8c
Merge pull request #251431 from adamcstephens/lxd/5.17
lxd: 5.16 -> 5.17
2023-08-26 09:19:53 -05:00
Adam Stephens 6175a329d2
lxd: fix passthru test building 2023-08-26 08:51:13 -04:00
Raito Bezarius 7207b25099 nixos/tests/dolibarr: use -X GET instead -X POST to test for redirection
Previously, we were POST-ing the homepage to test for redirection, this is wrong.

We are supposed to GET-ing it.

This is fixed.
2023-08-26 13:44:19 +02:00
Franz Pletz da37904672
Merge pull request #246772 from R-VdP/nixos-user-expiry
nixos/update-users-groups: add support for account expiry
2023-08-26 02:41:34 +02:00
Franz Pletz fe907aa37e
Merge pull request #242365 from h7x4/add-mysqld-exporter
services/prometheus/exporters: add mysqld
2023-08-25 23:56:22 +02:00
Ryan Lahfa 4fb9aeae23
Merge pull request #250843 from RaitoBezarius/listmonk 2023-08-24 10:59:58 +02:00
Pierre Bourdon 4428f3a79a
Revert "nixos/security/wrappers: simplifications and a fix for #98863" 2023-08-24 08:35:11 +02:00
Mario Rodas 8720d96225
Merge pull request #248395 from adamcstephens/lxd/tests
nixos/tests/lxd: move into subdir, use minimal init, remove sleeps
2023-08-24 00:37:04 -05:00
Nick Cao 8d524e610b
Merge pull request #249502 from oddlama/feat-influxdb-provision-full
nixos/influxdb2: add provisioning and nixos tests
2023-08-24 08:55:38 +08:00
Pierre Bourdon 0930308453
Merge pull request #199599 from robryk/suidwrapnoreal
nixos/security/wrappers: simplifications and a fix for #98863
2023-08-24 01:09:36 +02:00
Felix Bühler fb6665f4de
Merge pull request #250970 from alyssais/virtualbox-test-eval
nixosTests.virtualbox: fix eval
2023-08-23 22:02:34 +02:00
Emily 73a060e7ca
Merge pull request #244626 from Sohalt/caddy-rfc42
nixos/caddy: Add rfc42 settings option
2023-08-23 20:36:46 +02:00
Benjamin Staffin 5b198578db
Merge pull request #239801 from benley/hddfancontrol
hddfancontrol: init at 1.5.1 (plus nixos module)
2023-08-23 13:32:37 -04:00
Benjamin Staffin 12b3178bd2 nixos/hddfancontrol: initial module & test 2023-08-23 13:25:29 -04:00
Raito Bezarius ffdeabbadf listmonk: perform tests only linux supported platforms
Otherwise, Darwin will try to run them and fail.
2023-08-23 17:50:28 +02:00
Alyssa Ross 42f1b5c6db
nixosTests.os-prober: add bintools to the VM
machine # warning: error: unable to download 'https://ftpmirror.gnu.org/bash/bash-5.2.tar.gz': Couldn't resolve host name (6); retrying in 329 ms
machine # warning: error: unable to download 'https://ftpmirror.gnu.org/bash/bash-5.2.tar.gz': Couldn't resolve host name (6); retrying in 530 ms
machine # warning: error: unable to download 'https://ftpmirror.gnu.org/bash/bash-5.2.tar.gz': Couldn't resolve host name (6); retrying in 1066 ms
machine # warning: error: unable to download 'https://ftpmirror.gnu.org/bash/bash-5.2.tar.gz': Couldn't resolve host name (6); retrying in 2544 ms
machine # [ 1283.199545] GUEST-test1[1465]: [  414.478221] stage-1-init: [Wed Aug 23 13:26:29 UTC 2023] + loadkmap
machine # error:
machine #        … writing file '/nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz'
machine #
machine #        error: unable to download 'https://ftpmirror.gnu.org/bash/bash-5.2.tar.gz': Couldn't resolve host name (6)
machine # error: builder for '/nix/store/5jrd75v747s76s16zxk59384xfcjqn58-bash-5.2.tar.gz.drv' failed with exit code 1
machine # error: 1 dependencies of derivation '/nix/store/0cgj4m2h51hjhmz5h4440pd73kv5lm5v-bash-5.2-p15.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/d2x66i0dfv9w81gl1w3nbkn0nz7mawaz-bash-5.2-p15.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/lcz1v3h1nsbyz2fp7xkp113jvyjqq0sx-bash-5.2-p15.drv' failed to build
machine # building '/nix/store/20d5pi1a5i9jj041i0gvr9zcs7bjbw46-binutils-2.40.tar.bz2.drv'...
machine # error: 1 dependencies of derivation '/nix/store/zb0ykvcllgc8l9ki38fdv9n8xp3rnphb-gcc-12.3.0.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/7kxjnzmc79sickp7hiyp8v169idyw8f2-gettext-0.21.1.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/wp7hpglhgwljl3fsfyx8caaakh4a1r72-xgcc-12.3.0.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/f7glbcn7n59k22b911bx1vyy13g4bdxh-binutils-2.40.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/q7yvprjmnqprx743ikkcz4kqx2mjdas4-binutils-wrapper-2.40.drv' failed to build
machine # building '/nix/store/vks3aqqal1rjvrsbj61nl1yh7r5shhdh-builder.pl.drv'...
machine # error: 1 dependencies of derivation '/nix/store/qmdff14r0l31mzx8al7h1kp9h5pck5wr-extra-utils.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/rdlk4188b2jp4ac38w94qazdaxk6sga9-stage-1-init.sh.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/rq15acvd6hcr52a5dlmk1p7mlyzjack0-initrd-linux-6.1.46.drv' failed to build
machine # error: 1 dependencies of derivation '/nix/store/h1rch2zqjacijnn0szq2hgwmd6v1r1ld-nixos-system-nixos-23.11pre-git.drv' failed to build
2023-08-23 13:50:44 +00:00