forgejo/infrastructure-documentation
1
0
Fork 0
mirror of https://code.forgejo.org/infrastructure/documentation synced 2024-11-25 12:11:10 +00:00
Code Issues Activity

Merge pull request 'k8s three nodes cluster' (#19) from earl-warren/documentation:wip-k8s-cluster into main

Browse source 
Reviewed-on: https://code.forgejo.org/infrastructure/documentation/pulls/19
This commit is contained in:
earl-warren 2024-10-05 10:25:00 +00:00
parent be0278a40e fb8209f378
commit bed22adf2b
1 changed files with 57 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

70
README.md
View file

@ -792,18 +792,6 @@ iface enp5s0 inet6 static
#### k8s node
```sh
$ cat /etc/rancher/k3s/config.yaml.d/config.yaml
# https://docs.k3s.io/installation/network-options
# the ipv4-ranges are default in k3s
# the fd-range is a private ipv6-range
cluster-cidr: 10.42.0.0/16,fd01::/48
service-cidr: 10.43.0.0/16,fd02::/112
disable:
- servicelb
```
The `10.88.1.5` and `fd01::5` IPs are assigned to the interface with VLAN 4002.
```
@ -816,12 +804,68 @@ iface enp5s0.4002 inet static
up ip addr add fd01::5/48 dev enp5s0.4002
```
### k8s first server node
```sh
$ cat /etc/rancher/k3s/config.yaml.d/config.yaml
# https://docs.k3s.io/installation/network-options
# the ipv4-ranges are default in k3s
# the fd-range is a private ipv6-range
cluster-cidr: 10.42.0.0/16,fd01::/48
service-cidr: 10.43.0.0/16,fd02::/112
disable:
- servicelb
```
```sh
sudo apt-get install curl
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--write-kubeconfig-mode=644 --node-ip=10.88.1.5,fd01::5' sh -
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode=644 --cluster-init --node-ip=10.88.1.5,fd01::5" sh -
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -
```
### k8s second server node
```sh
$ cat /etc/rancher/k3s/config.yaml.d/config.yaml
# https://docs.k3s.io/installation/network-options
# the ipv4-ranges are default in k3s
# the fd-range is a private ipv6-range
cluster-cidr: 10.42.0.0/16,fd01::/48
service-cidr: 10.43.0.0/16,fd02::/112
disable:
- servicelb
```
The token is found on the first node in the `/var/lib/rancher/k3s/server/token` file.
```sh
sudo apt-get install curl
master_node_ip=10.88.1.5
second_server_ip=10.88.1.6,fd01::6
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode=644 --token $token --server https://$master_node_ip:6443 --node-ip=$second_server_ip" sh -
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -
```
### k8s dedicated etcd node
[dedicated etcd node](https://docs.k3s.io/installation/server-roles#dedicated-etcd-nodes)
The token is found on one of the master nodes in the `/var/lib/rancher/k3s/server/token` file.
```sh
master_node_ip=10.88.1.5
etcd_node_ip=10.88.1.3,fd01::3
curl -fL https://get.k3s.io | sh -s - server --token "$token" --server https://$master_node_ip:6443 --cluster-init --disable-apiserver --disable-controller-manager --disable-scheduler --write-kubeconfig-mode=644 --node-ip=$etcd_node_ip --cluster-cidr=10.42.0.0/16,fd01::/48 --service-cidr=10.43.0.0/16,fd02::/112
```
It should not be allowed to schedule pods but for some reason it is. Working around this with:
```sh
kubectl taint nodes hetzner03 key1=value1:NoSchedule
```
### k8s networking
`k3s` is configured to use [metallb](https://metallb.universe.tf) instead of the default load balancer because it does not allow for a public IP different from the `k8s` node IP.