[Kubernetes] VirtualBox Node INTERNAL-IP 설정 방법

오늘은 Kubernetes VirtualBox Node INTERNAL-IP 설정 방법에 대해 포스팅하려고 합니다.

VirtualBox의 사용하여 Kubernetes를 설치할 경우 NAT 어댑터는 enp0s3(10.0.2.15)
호스트 전용 어댑터는 enp0s8(192.168.1.5) 으로 설정되어 있습니다.

$ ip -a

2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:00:59:dc brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enp0s3
       valid_lft 85489sec preferred_lft 85489sec
    inet6 fe80::99d8:eaff:a5d3:bf5b/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:53:60:5f brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.5/24 brd 192.168.1.255 scope global dynamic noprefixroute enp0s8
       valid_lft 520sec preferred_lft 520sec
    inet6 fe80::e6db:d0d1:20e3:b3b3/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

 

 

node 별로 호스트 전용 어댑터는 중복되지 않게 IP가 구성되지만, 
NAT 어댑터는 IP가 10.0.2.15로 아래 내용과 같이 master node, worker node가 모두 동일하게 되어있으며 

클러스터 구성시 INTERNAL-IP는 NAT 어댑터인 enp0s3(10.0.2.15)으로 설정되기 때문에 

CoreDNS 또는 CNI인 weave, calico 등에 문제가 발생합니다.

$ kubectl get node -o wide

NAME           STATUS   ROLES           AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
kube-master    Ready    control-plane   3d    v1.24.2   10.0.2.15   <none>        Ubuntu 20.04.4 LTS   5.13.0-52-generic   containerd://1.6.6
kube-worker1   Ready    <none>          3d    v1.24.2   10.0.2.15   <none>        Ubuntu 20.04.4 LTS   5.13.0-52-generic   containerd://1.6.6
kube-worker2   Ready    <none>          3d    v1.24.2   10.0.2.15   <none>        Ubuntu 20.04.4 LTS   5.13.0-52-generic   containerd://1.6.6

 

 

 

그러므로 Kubernetes의 기본으로 설정되어있는 NAT 어댑터인 enp0s3(10.0.2.15)을 

호스트 전용 어댑터인 enp0s8(192.168.1.5)으로 변경해야 합니다.
해당 설정은 master node, worker node 모두 각각 진행 해야 합니다. 

방법은 다음과 같습니다.

먼저 10-kubeadm.conf파일을 확인합니다. 

$ cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
#Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

 

 

위 내용 중에서 EnvironmentFile=-/etc/default/kubelet라는 설정이 있는데
kubelet arg를 재정의 할 수 있는 파일입니다. 
EnvironmentFile가 정의되어있는 파일을 열어서 호스트 전용 어댑터인 enp0s8의 IP를 입력하고 저장하면 됩니다.

$ vi /etc/default/kubelet

KUBELET_EXTRA_ARGS=--node-ip=<노드의 호스트 전용 어댑터 IP 주소>

 

 

그리고 kubelet service를 재시작 합니다.

$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet

 

 

master node, worker node 모두 진행을 했다면, master node에서 get node를 했을 때, 

INTERNAL-IP가 서로 다른 호스트 전용 어댑터 IP로
설정이 된 것을 확인할 수 있습니다.

$ kubectl get node -o wide

NAME           STATUS   ROLES           AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
kube-master    Ready    control-plane   3d    v1.24.2   192.168.1.5   <none>        Ubuntu 20.04.4 LTS   5.13.0-52-generic   containerd://1.6.6
kube-worker1   Ready    <none>          3d    v1.24.2   192.168.1.6   <none>        Ubuntu 20.04.4 LTS   5.13.0-52-generic   containerd://1.6.6
kube-worker2   Ready    <none>          3d    v1.24.2   192.168.1.7   <none>        Ubuntu 20.04.4 LTS   5.13.0-52-generic   containerd://1.6.6

 

 

 

지금까지 Kubernetes VirtualBox Node INTERNAL-IP 설정 방법에 대한 포스팅이었습니다.