From 2e13b2dbbe631d81f4c694dadfda5f38a3a421cd Mon Sep 17 00:00:00 2001 From: Earl Warren Date: Sat, 19 Oct 2024 12:29:49 +0200 Subject: [PATCH] add disaster recovery instructions --- README.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/README.md b/README.md index 95c4cd2..3e0aede 100644 --- a/README.md +++ b/README.md @@ -1066,6 +1066,28 @@ persistence: claimName: forgejo-data ``` +## Disaster recovery and maintenance + +### When a machine or disk is scheduled for replacement. + +* `kubectl drain hetzner05` # evacuate all the pods out of the node to be shutdown + +### Routing the failover IP + +When the machine to which the failover IP (failover.forgejo.org) is routed is unavailable or to be shutdown, to the [Hetzner server panel](https://robot.hetzner.com/server), to the IPs tab and change the route of the failover IP to another node. All nodes are configured with the failover IP, there is nothing else to do. + +### Manual boot operations + +#### On the machine that runs the NFS server + +* `sudo drbdadm primary r1` # Switch the DRBD to primary +* `sudo mount /precious` # DRBD volume shared via NFS +* `sudo ip addr add 10.53.101.100/24 dev enp5s0.4001` # add NFS server IP + +#### On the other machines + +* `sudo ip addr del 10.53.101.100/24 dev enp5s0.4001` # remove NFS server IP + ## Uberspace The website https://forgejo.org is hosted at