loki, prometheus, promtail should connect via wireguard #200

Merged
teutat3s merged 7 commits from loki-prometheus-via-wireguard into main 2024-06-05 00:04:40 +00:00
Owner

This changes the behaviour that was implemented before we had wireguard connections between hosts. Previously, HTTPS + basic auth was used to push logs to loki and scrape metrics using prometheus. With the wireguard VPN between hosts, this is no longer necessary. Instead, we bind to the wireguard interface wg-ssh (or its IPv4+IPv6 IP addresses) and use HTTP tunnelled via wireguard.

Also add new *.wg.pub.solar DNS A + AAAA records to get the wireguard IPs of our hosts.

This changes the behaviour that was implemented before we had wireguard connections between hosts. Previously, HTTPS + basic auth was used to push logs to loki and scrape metrics using prometheus. With the wireguard VPN between hosts, this is no longer necessary. Instead, we bind to the wireguard interface `wg-ssh` (or its IPv4+IPv6 IP addresses) and use HTTP tunnelled via wireguard. Also add new *.wg.pub.solar DNS `A` + `AAAA` records to get the wireguard IPs of our hosts.
teutat3s added 2 commits 2024-06-01 15:09:21 +00:00
teutat3s requested review from b12f 2024-06-01 15:09:26 +00:00
teutat3s requested review from hensoko 2024-06-01 15:09:26 +00:00
teutat3s requested review from axeman 2024-06-01 15:09:27 +00:00
b12f reviewed 2024-06-02 18:47:07 +00:00
@ -27,11 +27,6 @@
# Don't expose SSH via public interfaces
networking.firewall.interfaces.wg-ssh.allowedTCPPorts = [ 22 ];
networking.hosts = {
Owner

Is there a reason to move this into the global DNS? we can connect to other nodes in the network without DNS being available, so it might be nice to also have their local names without a DNS server being available.

Is there a reason to move this into the global DNS? we can connect to other nodes in the network [without DNS being available](https://git.pub.solar/pub-solar/infra/src/branch/main/hosts/nachtigall/wireguard.nix#L24), so it might be nice to also have their local names without a DNS server being available.
Author
Owner

As the commit message states, the goal is to avoid surprises. I find it very confusing to have the same DNS name resolve to different IP addresses depending what host you're on. It would be my preference to remove this for clarity.

As the commit message states, the goal is to avoid surprises. I find it very confusing to have the same DNS name resolve to different IP addresses depending what host you're on. It would be my preference to remove this for clarity.
Owner

But to get this straight: We have the internal "wg.pub.solar" hosts that would could additionally set statically via networking.hosts right? @teutat3s

I tend to agree with @b12f about a possible fuckup because of broken DNS but the middle ground here would be to not have context based in relation to the host your on but cached dns entries. It creates a bit of additional maintenance work as we have to update both but that could be ruled out by mentioning it in the docs and hanging people for not RTFM.

But to get this straight: We have the internal "wg.pub.solar" hosts that would could additionally set statically via `networking.hosts` right? @teutat3s I tend to agree with @b12f about a possible fuckup because of broken DNS but the middle ground here would be to not have context based in relation to the host your on but cached dns entries. It creates a bit of additional maintenance work as we have to update both but that could be ruled out by mentioning it in the docs and hanging people for not RTFM.
Author
Owner

I have no objections to setting *.wg.pub.solar in /etc/hosts as well if you're suggesting that. Not sure if I understood your comment correctly, @hensoko.

I have no objections to setting `*.wg.pub.solar` in `/etc/hosts` as well if you're suggesting that. Not sure if I understood your comment correctly, @hensoko.
Author
Owner

Better like this? 56f692740e

Better like this? https://git.pub.solar/pub-solar/infra/commit/56f692740e31c183b7624f9d81f4ebf1237b08ec
Owner

Yessss. @b12f also happy?

Yessss. @b12f also happy?
teutat3s marked this conversation as resolved
teutat3s force-pushed loki-prometheus-via-wireguard from 62add8e095 to 8f1b932fdc 2024-06-03 10:30:15 +00:00 Compare
teutat3s added 1 commit 2024-06-03 10:34:19 +00:00
networking: add internal IPv6 wireguard IPs to /etc/hosts
All checks were successful
Flake checks / Check (pull_request) Successful in 3m8s
61ea0ad7c2
hensoko approved these changes 2024-06-03 12:31:10 +00:00
hensoko approved these changes 2024-06-03 12:31:35 +00:00
b12f approved these changes 2024-06-04 07:20:31 +00:00
teutat3s added 2 commits 2024-06-05 00:00:19 +00:00
teutat3s merged commit 10ed117dfe into main 2024-06-05 00:04:40 +00:00
teutat3s deleted branch loki-prometheus-via-wireguard 2024-06-05 00:04:40 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: pub-solar/infra#200
No description provided.