forgejo/infrastructure-documentation
1
0
Fork 0
mirror of https://code.forgejo.org/infrastructure/documentation synced 2024-12-22 12:53:53 +00:00
Code Issues Activity

k8s three nodes cluster

Browse source 
Closes #18
This commit is contained in:
Earl Warren 2024-10-04 14:04:10 +02:00
parent 3b892d0830
commit fb8209f378
No known key found for this signature in database
GPG key ID: 0579CB2928A78A00
1 changed files with 57 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

70
README.md
View file

@ -775,18 +775,6 @@ iface enp5s0 inet6 static
#### k8s node
```sh
$ cat /etc/rancher/k3s/config.yaml.d/config.yaml
# https://docs.k3s.io/installation/network-options
# the ipv4-ranges are default in k3s
# the fd-range is a private ipv6-range
cluster-cidr: 10.42.0.0/16,fd01::/48
service-cidr: 10.43.0.0/16,fd02::/112
disable:
- servicelb
```
The `10.88.1.5` and `fd01::5` IPs are assigned to the interface with VLAN 4002.
```
@ -799,12 +787,68 @@ iface enp5s0.4002 inet static
up ip addr add fd01::5/48 dev enp5s0.4002
```
### k8s first server node
```sh
$ cat /etc/rancher/k3s/config.yaml.d/config.yaml
# https://docs.k3s.io/installation/network-options
# the ipv4-ranges are default in k3s
# the fd-range is a private ipv6-range
cluster-cidr: 10.42.0.0/16,fd01::/48
service-cidr: 10.43.0.0/16,fd02::/112
disable:
- servicelb
```
```sh
sudo apt-get install curl
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--write-kubeconfig-mode=644 --node-ip=10.88.1.5,fd01::5' sh -
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode=644 --cluster-init --node-ip=10.88.1.5,fd01::5" sh -
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -
```
### k8s second server node
```sh
$ cat /etc/rancher/k3s/config.yaml.d/config.yaml
# https://docs.k3s.io/installation/network-options
# the ipv4-ranges are default in k3s
# the fd-range is a private ipv6-range
cluster-cidr: 10.42.0.0/16,fd01::/48
service-cidr: 10.43.0.0/16,fd02::/112
disable:
- servicelb
```
The token is found on the first node in the `/var/lib/rancher/k3s/server/token` file.
```sh
sudo apt-get install curl
master_node_ip=10.88.1.5
second_server_ip=10.88.1.6,fd01::6
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode=644 --token $token --server https://$master_node_ip:6443 --node-ip=$second_server_ip" sh -
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -
```
### k8s dedicated etcd node
[dedicated etcd node](https://docs.k3s.io/installation/server-roles#dedicated-etcd-nodes)
The token is found on one of the master nodes in the `/var/lib/rancher/k3s/server/token` file.
```sh
master_node_ip=10.88.1.5
etcd_node_ip=10.88.1.3,fd01::3
curl -fL https://get.k3s.io | sh -s - server --token "$token" --server https://$master_node_ip:6443 --cluster-init --disable-apiserver --disable-controller-manager --disable-scheduler --write-kubeconfig-mode=644 --node-ip=$etcd_node_ip --cluster-cidr=10.42.0.0/16,fd01::/48 --service-cidr=10.43.0.0/16,fd02::/112
```
It should not be allowed to schedule pods but for some reason it is. Working around this with:
```sh
kubectl taint nodes hetzner03 key1=value1:NoSchedule
```
### k8s networking
`k3s` is configured to use [metallb](https://metallb.universe.tf) instead of the default load balancer because it does not allow for a public IP different from the `k8s` node IP.