Commit graph

106 commits

Author SHA1 Message Date
teutat3s 73333537a5
Merge pull request 'alertmanager: alert on high load only after 20m' (#255) from alerts-tweak-load into main
Reviewed-on: pub-solar/infra#255
Reviewed-by: hensoko <hensoko@noreply.git.pub.solar>
2024-11-12 14:47:53 +00:00
teutat3s ab85ba751a
alertmanager: enable e2e_dead_man_switch 2024-11-12 13:41:42 +01:00
teutat3s a9c5edfeb3
alertmanager: don't alert on high memory page faults
This alert is non actionable, we still monitor high memory usage.
2024-11-12 13:40:46 +01:00
teutat3s 43b0c8d489
matrix-appservice-irc: reduce logging level to warn 2024-11-06 21:29:27 +01:00
teutat3s afe52ca6af
alertmanager: alert on high load only after 20m 2024-11-06 21:28:28 +01:00
teutat3s 3ec5c9f343
style: fix formatting 2024-10-30 20:32:47 +01:00
b12f 041d311bb2
modules/matrix: rename used config options 2024-10-30 18:37:47 +01:00
teutat3s 9d9bcf9a15
mas: move to module, add secrets for prod 2024-10-30 18:37:46 +01:00
teutat3s 9d7d251369
style: fix formatting 2024-10-30 18:37:46 +01:00
teutat3s 7775ad332e
matrix: do not change paths for nachtigall secrets 2024-10-30 18:37:46 +01:00
teutat3s d6cc9c8164
matrix-authentication-service: init host underground
to test mas, related to #242
2024-10-30 18:37:45 +01:00
b12f 471d7650ff
modules/tt-rss: pin on revision 2024-10-30 18:35:18 +01:00
teutat3s 9758aeda5d
garage: fix wildcard DNS cert renewal with wildcard
CNAME records

By usind wildcard CNAME records, we make lego think it needs to validate
challenges using these CNAME records. We actually want regular
_acme-challenge.* records, so use a environment variable to avoid CNAME
detection. This fixes DNS cert renewal. Still curious? See:
https://letsencrypt.org/2019/10/09/onboarding-your-customers-with-lets-encrypt-and-acme/
2024-10-23 20:18:57 +02:00
teutat3s 5300f381b0
nginx: use safer request_uri variable
Fix >> Problem: [http_splitting] Possible HTTP-Splitting vulnerability.
https://github.com/yandex/gixy/blob/master/docs/en/plugins/httpsplitting.md
2024-10-17 21:15:57 +02:00
teutat3s 8a18ee452b
garage: fix s3_api root_domain 2024-10-17 21:15:57 +02:00
teutat3s 666de2c8f4
mastodon: switch files.pub.solar from storj to garage
s3 backend
2024-10-17 21:15:55 +02:00
teutat3s c39cf9c0b9
mastodon: update to version 4.3.0 from nixos-unstable
https://github.com/mastodon/mastodon/releases/tag/v4.3.0
https://github.com/NixOS/nixpkgs/pull/337545/files
2024-10-17 20:31:47 +02:00
teutat3s 092a45e3bd
mastodon: actually use opensearch via module option 2024-10-08 19:09:17 +02:00
teutat3s 8c8a757f8f
garage: update to 1.0.1
https://git.deuxfleurs.fr/Deuxfleurs/garage/releases/tag/v1.0.1
2024-10-05 13:03:40 +02:00
teutat3s 37f210c96f
security: add libolm to permittedInsecurePackages 2024-10-05 13:03:40 +02:00
b12f 4831430455
chore: run nix fmt 2024-09-10 16:02:26 +02:00
teutat3s 663ef8feb1
alerts: fix condition 2024-09-10 16:02:26 +02:00
teutat3s 63fa03e971
alerts.pub.solar: use DNS challenge for cert 2024-09-10 16:02:26 +02:00
teutat3s faa71b7797
alerts: add check for healthy garage cluster 2024-09-10 16:02:26 +02:00
teutat3s 19723f3812
monitoring: add prometheus-exporter, promtail to
delite, blue-shell

add instance labels to garage scrape jobs
2024-09-10 16:02:26 +02:00
teutat3s 47b076e0a6
loki: store logs in /var/lib/loki 2024-09-10 16:02:25 +02:00
b12f 1ec5bafa30
flora-6: remove
This commit removes the flora-6 host. All services are moved to
trinkgenossin, with the drone service being removed completely in favour
of forgejo actions.
2024-09-10 16:02:24 +02:00
teutat3s 44f708ec76
obs-portal: run backups 1h later to avoid lock conflict 2024-09-09 17:28:57 +02:00
teutat3s cd82b83427
obs-portal: fix backups, docker command does not
need a TTY
2024-08-31 22:05:11 +02:00
teutat3s 2d94ed5a0d
Merge pull request 'obs-portal: add backups' (#228) from obs-portal-backups into main
Reviewed-on: pub-solar/infra#228
Reviewed-by: b12f <b12f@noreply.git.pub.solar>
Reviewed-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-31 19:43:10 +00:00
teutat3s 2eb54a331e
backups: add storagebox to programs.ssh.knownHosts 2024-08-29 16:36:09 +02:00
teutat3s 77b642f646
garage: increase nginx client_body_size to 64m
To make bigger garage uploads work well, avoiding error
HTTP 413 Entity Too Large
2024-08-29 16:24:32 +02:00
teutat3s 2e16c77956
secrets: rename restic-repo-storagebox{,-nachtigall}
To use a restic repository per host
2024-08-29 16:22:58 +02:00
teutat3s e2ba1aacf4
mail: add backups to garage bucket + storagebox
Restic backups to garage S3 bucket metronom-backups
2024-08-29 16:19:24 +02:00
teutat3s 27dc20dd04
obs-portal: add backups to garage bucket + storagebox
Restic backups to garage S3 bucket nachtigall-backups
2024-08-29 10:09:04 +02:00
teutat3s d2389497c2
Merge pull request 'garage: initial cluster' (#222) from garage-cluster into main
Reviewed-on: pub-solar/infra#222
Reviewed-by: b12f <b12f@noreply.git.pub.solar>
Reviewed-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-28 15:55:16 +00:00
teutat3s 4626fd85c0
mediawiki: add backups to garage bucket + storagebox
Restic backups to garage S3 bucket nachtigall-backups
https://garagehq.deuxfleurs.fr/documentation/connect/backup/#restic
2024-08-28 17:13:34 +02:00
teutat3s c0a3d90d63
backups: add environmentFile option 2024-08-28 17:13:34 +02:00
teutat3s 1d92ef53ca
backups: storeName -> repoName 2024-08-28 17:13:33 +02:00
teutat3s 751d82f7e3
backups: rename pub-solar-os.backups.backups -> pub-solar-os.backups.restic 2024-08-28 17:12:22 +02:00
teutat3s 88b76beb5c
keycloak: use backups module
Co-authored-by: b12f <b12f@noreply.git.pub.solar>
Co-authored-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-27 10:09:07 +02:00
teutat3s e857c6198b
modules/backup: init
Co-authored-by: b12f <b12f@noreply.git.pub.solar>
Co-authored-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-27 10:04:10 +02:00
teutat3s 998cf4c63d
website: force HTTPS
Co-authored-by: b12f <b12f@noreply.git.pub.solar>
Co-authored-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-27 10:03:43 +02:00
teutat3s a0b52d51e5
nachtigall: make postgres wait for zfs mount
Co-authored-by: b12f <b12f@noreply.git.pub.solar>
Co-authored-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-27 10:00:42 +02:00
teutat3s 701c62dd69
tests: create keycloak test, add working test for website
Co-authored-by: b12f <b12f@noreply.git.pub.solar>
Co-authored-by: Hendrik Sokolowski <hensoko@noreply.git.pub.solar>
2024-08-27 09:55:25 +02:00
teutat3s f236962e17
garage: add monitoring, connect to grafana + loki
https://garagehq.deuxfleurs.fr/documentation/reference-manual/monitoring/
2024-08-25 00:18:09 +02:00
teutat3s 15b507904f
garage: init buckets.pub.solar, use nginx as reverse proxy
https://garagehq.deuxfleurs.fr/documentation/cookbook/reverse-proxy/
2024-08-24 21:48:48 +02:00
teutat3s 25827a97d3
modules: add unlock-luks-on-boot 2024-08-24 03:05:28 +02:00
teutat3s 4a3d3ce84b
garage: init module 2024-08-24 03:05:16 +02:00
teutat3s 7f2bfd923f
loki: move data dir to /data disk with more room 2024-08-07 10:19:53 +02:00