Set Up a K8s Cluster with kubeadm

Environment Preparation #

Prepare three Linux machines (Ubuntu 20.04 LTS is used as the example here). The three machines must be able to communicate with each other. I’m using VPSs here, so I’m not exposing the IPs.

Below are the three Ubuntu 20.04 LTS nodes used in this article:

hostname	system	memory
k8s-master	Ubuntu 20.04 LTS	8GB
k8s-worker1	Ubuntu 20.04 LTS	8GB
k8s-worker2	Ubuntu 20.04 LTS	8GB

Install containerd, kubeadm, kubelet, kubectl##

Save the following shell script into a file, for example master.sh, and copy it to all three machines.

Then run the script on each machine with:

1
sudo sh master.sh

To change the Kubernetes version, edit the last line of the script below. The version we use here is 1.28.0. You can list available versions with apt list -a kubeadm.

1
#!/bin/bash
2

3
echo "[TASK 1] Disable and turn off SWAP"
4
sed -i '/swap/d' /etc/fstab
5
swapoff -a
6

7
echo "[TASK 2] Stop and Disable firewall"
8
systemctl disable --now ufw >/dev/null 2>&1
9

10
echo "[TASK 3] Enable and Load Kernel modules"
11
cat >>/etc/modules-load.d/containerd.conf<<EOF
12
overlay
13
br_netfilter
14
EOF
15
modprobe overlay
16
modprobe br_netfilter
17

18
echo "[TASK 4] Add Kernel settings"
19
cat >>/etc/sysctl.d/kubernetes.conf<<EOF
20
net.bridge.bridge-nf-call-ip6tables = 1
21
net.bridge.bridge-nf-call-iptables  = 1
22
net.ipv4.ip_forward                 = 1
23
EOF
24
sysctl --system >/dev/null 2>&1
25

26
echo "[TASK 5] Install containerd runtime"
27
mkdir -p /etc/apt/keyrings
28
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
29
echo   "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
30
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
31
apt -qq update >/dev/null 2>&1
32
apt install -qq -y containerd.io >/dev/null 2>&1
33
containerd config default >/etc/containerd/config.toml
34
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
35
systemctl restart containerd
36
systemctl enable containerd >/dev/null 2>&1
37

38
echo "[TASK 6] Add apt repo for kubernetes"
39
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - >/dev/null 2>&1
40
apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main" >/dev/null 2>&1
41

42
echo "[TASK 7] Install Kubernetes components (kubeadm, kubelet and kubectl)"
43
apt install -qq -y kubeadm=1.28.0-00 kubelet=1.28.0-00 kubectl=1.28.0-00 >/dev/null 2>&1

After the script finishes, you can verify the installation of kubeadm, kubelet, and kubectl. If all of them return a version string, the installation was successful.

1
kubeadm version
2
kubelet --version
3
kubectl version

Initialize the master node#

[admonition title=“Warning” color=“blue”]All of the following operations are done on the master node.[/admonition]

You can pre-pull the images required by the cluster (optional):

1
sudo kubeadm config images pull

If pulling succeeds, you will see output similar to:

1
[config/images] Pulled registry.k8s.io/kube-apiserver:v1.28.2
2
[config/images] Pulled registry.k8s.io/kube-controller-manager:v1.28.2
3
[config/images] Pulled registry.k8s.io/kube-scheduler:v1.28.2
4
[config/images] Pulled registry.k8s.io/kube-proxy:v1.28.2
5
[config/images] Pulled registry.k8s.io/pause:3.9
6
[config/images] Pulled registry.k8s.io/etcd:3.5.6-0
7
[config/images] Pulled registry.k8s.io/coredns/coredns:v1.9.3

Initialize kubeadm

--apiserver-advertise-address: the IP address used by the local node to communicate with other nodes, i.e. the master node IP
--pod-network-cidr: the pod network CIDR

1
vagrant@k8s-master:~$ sudo kubeadm init --apiserver-advertise-address=192.168.56.10  --pod-network-cidr=10.244.0.0/16

Make sure you save the last part of the output. It tells you what to do next:

Prepare .kube
Deploy a pod network add-on
Add worker nodes

1
Your Kubernetes control-plane has initialized successfully!
2

3
To start using your cluster, you need to run the following as a regular user:
4

5
  mkdir -p $HOME/.kube
6
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
7
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
8

9
Alternatively, if you are the root user, you can run:
10

11
  export KUBECONFIG=/etc/kubernetes/admin.conf
12

13
You should now deploy a pod network to the cluster.
14
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
15
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
16

17
Then you can join any number of worker nodes by running the following on each as root:
18

19
kubeadm join ip:6443 --token 75tjpr.mekbas8r3yvimqen \
20
        --discovery-token-ca-cert-hash sha256:3366cd93860eaac418ce9b35dda601c88676fe6ba99388672aba4e76cc7369e4

Configure .kube#

1
mkdir -p $HOME/.kube
2
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
3
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Check status#

1
$ kubectl get nodes
2
$ kubectl get pods -A

Shell auto-completion (Bash)#

1
source <(kubectl completion bash)
2
echo "source <(kubectl completion bash)" >> ~/.bashrc

Deploy a pod network add-on#

Go to https://kubernetes.io/docs/concepts/cluster-administration/addons/ and choose a network add-on, then deploy it according to the linked documentation.

Here we choose the overlay network add-on named flannel. Deployment steps are as follows:

Download the file https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml and make the following adjustments:

1
curl -LO https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

Make sure the network matches the --pod-network-cidr we configured, 10.244.0.0/16:

1
net-conf.json: |
2
  {
3
    "Network": "10.244.0.0/16",
4
    "Backend": {
5
      "Type": "vxlan"
6
    }
7
  }

In the args for the kube-flannel container, make sure there is an iface= argument whose value is the interface name corresponding to our --apiserver-advertise-address=ip.

1
ip a

1
- name: kube-flannel
2
 #image: flannelcni/flannel:v0.18.0 for ppc64le and mips64le (dockerhub limitations may apply)
3
  image: rancher/mirrored-flannelcni-flannel:v0.18.0
4
  command:
5
  - /opt/bin/flanneld
6
  args:
7
  - --ip-masq
8
  - --kube-subnet-mgr
9
  - --iface=网卡名

Example:

1
vagrant@k8s-master:~$ ip a
2
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
3
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
4
    inet 127.0.0.1/8 scope host lo
5
      valid_lft forever preferred_lft forever
6
    inet6 ::1/128 scope host
7
      valid_lft forever preferred_lft forever
8
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
9
    link/ether 02:9a:67:51:1e:b6 brd ff:ff:ff:ff:ff:ff
10
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3
11
      valid_lft 85351sec preferred_lft 85351sec
12
    inet6 fe80::9a:67ff:fe51:1eb6/64 scope link
13
      valid_lft forever preferred_lft forever
14
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
15
    link/ether 08:00:27:59:c5:26 brd ff:ff:ff:ff:ff:ff
16
    inet 192.168.56.10/24 brd 192.168.56.255 scope global enp0s8
17
      valid_lft forever preferred_lft forever
18
    inet6 fe80::a00:27ff:fe59:c526/64 scope link
19
      valid_lft forever preferred_lft forever

Check the result. If you see output like the following, with all pods in Running state (especially coredns and flannel), it means the network add-on was deployed successfully.

Add worker nodes #

Adding worker nodes is very simple: just run the join command on each worker node. Pay attention to --token.

1
sudo kubeadm join 192.168.56.10:6443 --token 0pdoeh.wrqchegv3xm3k1ow \
2
  --discovery-token-ca-cert-hash sha256:f4e693bde148f5c0ff03b66fb24c51f948e295775763e8c5c4e60d24ff57fe82

Finally, check the nodes and pods from the master node (for example, with two worker nodes):

1
kubectl get nodes