Merge pull request 'use and create a NFS backed PVC' (#21) from earl-warren/documentation:wip-nfs into main

Reviewed-on: https://code.forgejo.org/infrastructure/documentation/pulls/21

README.md

@@ -70,6 +70,13 @@ https://robot.hetzner.com/server to only allow incoming ssh, http,
 https requests for both IPv4 & IPv6. Each server has a "Firewall" tab
 that provides control over this firewall.
 
+The firewall applies to the VLAN too. The 10.0.0.0/8 source address
+must therefore be allowed explicitly for IPv4 and that must be the
+first rule of the firewall.
+
+A template "k8s" was defined that can be used for sharing the same
+rules between multiple k8s nodes.
+
 ## nftables
 
 ```sh
@@ -354,6 +361,10 @@ The IP address ends with the same number as the hardware (hetzner02 => .2).
 
 The vSwitch on VLAN 4000 is for DRBD exclusively
 
+#### vSwitch NFS
+
+The vSwitch on VLAN 4001 is for NFS
+
 #### vSwitch k8s
 
 The vSwitch on VLAN 4002 is for the k8s control plane
@@ -754,8 +765,11 @@ lxc-helpers.sh lxc_install_lxc_inside 10.47.3 fc11
 
 ```sh
 sudo apt install nfs-kernel-server nfs-common
-echo /precious 10.53.101.0/255.255.255.0(rw,no_root_squash,subtree_check) | sudo tee -a /etc/exports
+cat <<EOF | sudo tee -a /etc/exports
-sudo exportfs -a
+/precious 10.53.101.0/24(rw,fsid=0,no_root_squash,no_subtree_check)
+/precious/k8s 10.53.101.0/24(rw,nohide,insecure,no_subtree_check)
+EOF
+sudo exportfs -av
 sudo exportfs -s
 ```
 
@@ -957,6 +971,85 @@ service:
     port: 2222
 ```
 
+Define the nfs storage class.
+
+```sh
+$ cat nfs.yml
+apiVersion: helm.cattle.io/v1
+kind: HelmChart
+metadata:
+  name: nfs
+  namespace: default
+spec:
+  chart: nfs-subdir-external-provisioner
+  repo: https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner
+  targetNamespace: default
+  set:
+    nfs.server: 10.53.101.5
+    nfs.path: /k8s
+    storageClass.name: nfs
+$ kubectl apply --server-side=true -f nfs.yml
+$ sleep 120 ; kubectl get storageclass nfs
+```
+
+### k8s NFS storage creation
+
+Create the directory to be used, with the expected permissions (assuing `/k8s` is the directory exported via NFS).
+
+```sh
+sudo mkdir /precious/k8s/forgejo-data
+sudo chmod 1000:1000 /precious/k8s/forgejo-data
+```
+
+Define the `forgejo-data` pvc.
+
+```sh
+$ cat pv.yml
+---
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: forgejo-data
+spec:
+  capacity:
+    storage: 20Gi
+  nfs:
+    server: 10.53.101.5
+    path: /k8s/forgejo-data
+  accessModes:
+    - ReadWriteMany
+  persistentVolumeReclaimPolicy: Retain
+  storageClassName: nfs
+  mountOptions:
+    - noatime
+    - nfsvers=4.2
+  volumeMode: Filesystem
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: forgejo-data
+spec:
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: 20Gi
+  volumeName: forgejo-data
+  storageClassName: nfs
+  volumeMode: Filesystem
+$ kubectl apply --server-side=true -f pv.yml
+```
+
+[Instruct the forgejo pod](https://code.forgejo.org/forgejo-helm/forgejo-helm#persistence) to use the `forgejo-data` pvc.
+
+```yaml
+persistence:
+  enabled: true
+  create: false
+  claimName: forgejo-data
+```
+
 ## Uberspace
 
 The website https://forgejo.org is hosted at