Merge pull request 'k8s: reference the scripts and add an intro' (#36) from earl-warren/documentation:wip-split into main

Reviewed-on: https://code.forgejo.org/infrastructure/documentation/pulls/36

k8s.md

@@ -1,4 +1,8 @@
-#### Imaging
+# K8S node
+
+Installing a K8S node using [scripts from the k3s-host](k3s-host) directory.
+
+## Imaging
 
 Using installimage from the rescue instance.
 
@@ -25,7 +29,7 @@ Debian user.
 - `mkdir -p /home/debian/.ssh ; cp -a .ssh/authorized_keys /home/debian/.ssh ; chown -R debian /home/debian/.ssh`
 - in `/etc/sudoers` edit `%sudo   ALL=(ALL:ALL) NOPASSWD:ALL`
 
-#### Install helpers
+## Install helpers
 
 Each node is identifed by the last digit of the hostname.
 
@@ -52,11 +56,11 @@ Variables that must be set depending on the role of the node.
 
 The other variables depend on the setup.
 
-#### Firewall
+## Firewall
 
 `./setup.sh setup_ufw`
 
-#### DRBD 
+## DRBD
 
 DRBD is [configured](https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#p-work) with:
 
@@ -71,7 +75,7 @@ sudo drbdadm new-current-uuid --clear-bitmap r1/0
 sudo mount /precious
 ```
 
-#### NFS
+## NFS
 
 `./setup.sh setup_nfs`
 
@@ -81,7 +85,7 @@ On the node that has the DRBD volume `/precious` mounted, set the IP of the NFS
 sudo ip addr add 10.53.101.100/24 dev enp5s0.4001
 ```
 
-#### K8S
+## K8S
 
 For the first node `./setup.sh setup_k8s`. For nodes joining the cluster `./setup.sh setup_k8s 6` where `hetzner06` is an existing node.
 
@@ -94,7 +98,7 @@ For the first node `./setup.sh setup_k8s`. For nodes joining the cluster `./setu
 - NFS storage class
   `./setup.sh setup_k8s_nfs`
 
-#### Forgejo
+## Forgejo
 
 [forgejo](https://code.forgejo.org/forgejo-helm/forgejo-helm) configuration in [ingress](https://code.forgejo.org/forgejo-helm/forgejo-helm#ingress) for the reverse proxy (`traefik`) to route the domain and for the ACME issuer (`cert-manager`) to obtain a certificate. And in [service](https://code.forgejo.org/forgejo-helm/forgejo-helm#service) for the `ssh` port to be bound to the desired IPs of the load balancer (`metallb`).
 
@@ -102,38 +106,38 @@ For the first node `./setup.sh setup_k8s`. For nodes joining the cluster `./setu
 ingress:
   enabled: true
   annotations:
-    # https://cert-manager.io/docs/usage/ingress/#supported-annotations
+	# https://cert-manager.io/docs/usage/ingress/#supported-annotations
-    # https://github.com/cert-manager/cert-manager/issues/2239
+	# https://github.com/cert-manager/cert-manager/issues/2239
-    cert-manager.io/cluster-issuer: letsencrypt-http
+	cert-manager.io/cluster-issuer: letsencrypt-http
-    cert-manager.io/private-key-algorithm: ECDSA
+	cert-manager.io/private-key-algorithm: ECDSA
-    cert-manager.io/private-key-size: 384
+	cert-manager.io/private-key-size: 384
-    kubernetes.io/ingress.class: traefik
+	kubernetes.io/ingress.class: traefik
-    traefik.ingress.kubernetes.io/router.entrypoints: websecure
+	traefik.ingress.kubernetes.io/router.entrypoints: websecure
   tls:
-    - hosts:
+	- hosts:
-        - t1.forgejo.org
+		- t1.forgejo.org
-      secretName: tls-forgejo-t1-ingress-http
+	  secretName: tls-forgejo-t1-ingress-http
   hosts:
-    - host: t1.forgejo.org
+	- host: t1.forgejo.org
-      paths:
+	  paths:
-        - path: /
+		- path: /
-          pathType: Prefix
+		  pathType: Prefix
 
 service:
   http:
-    type: ClusterIP
+	type: ClusterIP
-    ipFamilyPolicy: PreferDualStack
+	ipFamilyPolicy: PreferDualStack
-    port: 3000
+	port: 3000
   ssh:
-    type: LoadBalancer
+	type: LoadBalancer
-    annotations:
+	annotations:
-      metallb.universe.tf/loadBalancerIPs: 188.40.16.47,2a01:4f8:fff2:48::2
+	  metallb.universe.tf/loadBalancerIPs: 188.40.16.47,2a01:4f8:fff2:48::2
-      metallb.universe.tf/allow-shared-ip: "key-to-share-failover"
+	  metallb.universe.tf/allow-shared-ip: "key-to-share-failover"
-    ipFamilyPolicy: PreferDualStack
+	ipFamilyPolicy: PreferDualStack
-    port: 2222
+	port: 2222
 ```
 
-### K8S NFS storage creation
+# K8S NFS storage creation
 
 Define the 20GB `forgejo-data` pvc owned by user id 1000.
 
@@ -150,27 +154,26 @@ persistence:
   claimName: forgejo-data
 ```
 
-## Disaster recovery and maintenance
+ Disaster recovery and maintenance
 
-### When a machine or disk is scheduled for replacement.
+# When a machine or disk is scheduled for replacement.
 
 * `kubectl drain hetzner05` # evacuate all the pods out of the node to be shutdown
 * `kubectl taint nodes hetzner05 key1=value1:NoSchedule` # prevent any pod from being created there (metallb speaker won't be drained, for instance)
 * `kubectl delete node hetzner05` # let the cluster know it no longer exists so a new one by the same name can replace it
 
-### Routing the failover IP
+# Routing the failover IP
 
 When the machine to which the failover IP (failover.forgejo.org) is routed is unavailable or to be shutdown, to the [Hetzner server panel](https://robot.hetzner.com/server), to the IPs tab and change the route of the failover IP to another node. All nodes are configured with the failover IP, there is nothing else to do.
 
-### Manual boot operations
+# Manual boot operations
 
-#### On the machine that runs the NFS server
+## On the machine that runs the NFS server
 
 * `sudo drbdadm primary r1` # Switch the DRBD to primary
 * `sudo mount /precious` # DRBD volume shared via NFS
 * `sudo ip addr add 10.53.101.100/24 dev enp5s0.4001` # add NFS server IP
 
-#### On the other machines
+## On the other machines
 
 * `sudo ip addr del 10.53.101.100/24 dev enp5s0.4001` # remove NFS server IP
-