mirror of
https://code.forgejo.org/infrastructure/documentation
synced 2024-11-22 11:21:10 +00:00
Merge pull request 'k8s: reference the scripts and add an intro' (#36) from earl-warren/documentation:wip-split into main
Reviewed-on: https://code.forgejo.org/infrastructure/documentation/pulls/36
This commit is contained in:
commit
4a54adf37f
33
k8s.md
33
k8s.md
|
@ -1,4 +1,8 @@
|
||||||
#### Imaging
|
# K8S node
|
||||||
|
|
||||||
|
Installing a K8S node using [scripts from the k3s-host](k3s-host) directory.
|
||||||
|
|
||||||
|
## Imaging
|
||||||
|
|
||||||
Using installimage from the rescue instance.
|
Using installimage from the rescue instance.
|
||||||
|
|
||||||
|
@ -25,7 +29,7 @@ Debian user.
|
||||||
- `mkdir -p /home/debian/.ssh ; cp -a .ssh/authorized_keys /home/debian/.ssh ; chown -R debian /home/debian/.ssh`
|
- `mkdir -p /home/debian/.ssh ; cp -a .ssh/authorized_keys /home/debian/.ssh ; chown -R debian /home/debian/.ssh`
|
||||||
- in `/etc/sudoers` edit `%sudo ALL=(ALL:ALL) NOPASSWD:ALL`
|
- in `/etc/sudoers` edit `%sudo ALL=(ALL:ALL) NOPASSWD:ALL`
|
||||||
|
|
||||||
#### Install helpers
|
## Install helpers
|
||||||
|
|
||||||
Each node is identifed by the last digit of the hostname.
|
Each node is identifed by the last digit of the hostname.
|
||||||
|
|
||||||
|
@ -52,11 +56,11 @@ Variables that must be set depending on the role of the node.
|
||||||
|
|
||||||
The other variables depend on the setup.
|
The other variables depend on the setup.
|
||||||
|
|
||||||
#### Firewall
|
## Firewall
|
||||||
|
|
||||||
`./setup.sh setup_ufw`
|
`./setup.sh setup_ufw`
|
||||||
|
|
||||||
#### DRBD
|
## DRBD
|
||||||
|
|
||||||
DRBD is [configured](https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#p-work) with:
|
DRBD is [configured](https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#p-work) with:
|
||||||
|
|
||||||
|
@ -71,7 +75,7 @@ sudo drbdadm new-current-uuid --clear-bitmap r1/0
|
||||||
sudo mount /precious
|
sudo mount /precious
|
||||||
```
|
```
|
||||||
|
|
||||||
#### NFS
|
## NFS
|
||||||
|
|
||||||
`./setup.sh setup_nfs`
|
`./setup.sh setup_nfs`
|
||||||
|
|
||||||
|
@ -81,7 +85,7 @@ On the node that has the DRBD volume `/precious` mounted, set the IP of the NFS
|
||||||
sudo ip addr add 10.53.101.100/24 dev enp5s0.4001
|
sudo ip addr add 10.53.101.100/24 dev enp5s0.4001
|
||||||
```
|
```
|
||||||
|
|
||||||
#### K8S
|
## K8S
|
||||||
|
|
||||||
For the first node `./setup.sh setup_k8s`. For nodes joining the cluster `./setup.sh setup_k8s 6` where `hetzner06` is an existing node.
|
For the first node `./setup.sh setup_k8s`. For nodes joining the cluster `./setup.sh setup_k8s 6` where `hetzner06` is an existing node.
|
||||||
|
|
||||||
|
@ -94,7 +98,7 @@ For the first node `./setup.sh setup_k8s`. For nodes joining the cluster `./setu
|
||||||
- NFS storage class
|
- NFS storage class
|
||||||
`./setup.sh setup_k8s_nfs`
|
`./setup.sh setup_k8s_nfs`
|
||||||
|
|
||||||
#### Forgejo
|
## Forgejo
|
||||||
|
|
||||||
[forgejo](https://code.forgejo.org/forgejo-helm/forgejo-helm) configuration in [ingress](https://code.forgejo.org/forgejo-helm/forgejo-helm#ingress) for the reverse proxy (`traefik`) to route the domain and for the ACME issuer (`cert-manager`) to obtain a certificate. And in [service](https://code.forgejo.org/forgejo-helm/forgejo-helm#service) for the `ssh` port to be bound to the desired IPs of the load balancer (`metallb`).
|
[forgejo](https://code.forgejo.org/forgejo-helm/forgejo-helm) configuration in [ingress](https://code.forgejo.org/forgejo-helm/forgejo-helm#ingress) for the reverse proxy (`traefik`) to route the domain and for the ACME issuer (`cert-manager`) to obtain a certificate. And in [service](https://code.forgejo.org/forgejo-helm/forgejo-helm#service) for the `ssh` port to be bound to the desired IPs of the load balancer (`metallb`).
|
||||||
|
|
||||||
|
@ -133,7 +137,7 @@ service:
|
||||||
port: 2222
|
port: 2222
|
||||||
```
|
```
|
||||||
|
|
||||||
### K8S NFS storage creation
|
# K8S NFS storage creation
|
||||||
|
|
||||||
Define the 20GB `forgejo-data` pvc owned by user id 1000.
|
Define the 20GB `forgejo-data` pvc owned by user id 1000.
|
||||||
|
|
||||||
|
@ -150,27 +154,26 @@ persistence:
|
||||||
claimName: forgejo-data
|
claimName: forgejo-data
|
||||||
```
|
```
|
||||||
|
|
||||||
## Disaster recovery and maintenance
|
Disaster recovery and maintenance
|
||||||
|
|
||||||
### When a machine or disk is scheduled for replacement.
|
# When a machine or disk is scheduled for replacement.
|
||||||
|
|
||||||
* `kubectl drain hetzner05` # evacuate all the pods out of the node to be shutdown
|
* `kubectl drain hetzner05` # evacuate all the pods out of the node to be shutdown
|
||||||
* `kubectl taint nodes hetzner05 key1=value1:NoSchedule` # prevent any pod from being created there (metallb speaker won't be drained, for instance)
|
* `kubectl taint nodes hetzner05 key1=value1:NoSchedule` # prevent any pod from being created there (metallb speaker won't be drained, for instance)
|
||||||
* `kubectl delete node hetzner05` # let the cluster know it no longer exists so a new one by the same name can replace it
|
* `kubectl delete node hetzner05` # let the cluster know it no longer exists so a new one by the same name can replace it
|
||||||
|
|
||||||
### Routing the failover IP
|
# Routing the failover IP
|
||||||
|
|
||||||
When the machine to which the failover IP (failover.forgejo.org) is routed is unavailable or to be shutdown, to the [Hetzner server panel](https://robot.hetzner.com/server), to the IPs tab and change the route of the failover IP to another node. All nodes are configured with the failover IP, there is nothing else to do.
|
When the machine to which the failover IP (failover.forgejo.org) is routed is unavailable or to be shutdown, to the [Hetzner server panel](https://robot.hetzner.com/server), to the IPs tab and change the route of the failover IP to another node. All nodes are configured with the failover IP, there is nothing else to do.
|
||||||
|
|
||||||
### Manual boot operations
|
# Manual boot operations
|
||||||
|
|
||||||
#### On the machine that runs the NFS server
|
## On the machine that runs the NFS server
|
||||||
|
|
||||||
* `sudo drbdadm primary r1` # Switch the DRBD to primary
|
* `sudo drbdadm primary r1` # Switch the DRBD to primary
|
||||||
* `sudo mount /precious` # DRBD volume shared via NFS
|
* `sudo mount /precious` # DRBD volume shared via NFS
|
||||||
* `sudo ip addr add 10.53.101.100/24 dev enp5s0.4001` # add NFS server IP
|
* `sudo ip addr add 10.53.101.100/24 dev enp5s0.4001` # add NFS server IP
|
||||||
|
|
||||||
#### On the other machines
|
## On the other machines
|
||||||
|
|
||||||
* `sudo ip addr del 10.53.101.100/24 dev enp5s0.4001` # remove NFS server IP
|
* `sudo ip addr del 10.53.101.100/24 dev enp5s0.4001` # remove NFS server IP
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue