Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Title
n
narrow-noon-75604
08/24/2022, 5:04 AMHi,
I am trying to deploy a HA RKE2 cluster on baremetal machines with ubuntu20 using kube-vip. The first server node is up and running and be able to list the VIP so I have added 2 more server nodes and a worker node.
The installation is successful without any issue and the VIP has appeared after the overall RKE2 installation. Then I have deployed a sample nginx application to test if the service is accessible through VIP address.
I am able to access the application through all the server node IP Addresses but could not be able to access it using VIP. Then I tried to list the IP addresses in the interface that I am using and found that the VIP has disappeared.
Right after the installation is finished,
ip a list $INTERFACE
Output:
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:9b:3a:cb brd ff:ff:ff:ff:ff:ff
inet 192.168.10.71/24 brd 192.168.10.255 scope global ens160
valid_lft forever preferred_lft forever
inet 192.168.10.74/32 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe9b:3acb/64 scope link
valid_lft forever preferred_lft foreverip a list $INTERFACE
Output:
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:9b:3a:cb brd ff:ff:ff:ff:ff:ff
inet 192.168.10.71/24 brd 192.168.10.255 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe9b:3acb/64 scope link
valid_lft forever preferred_lft foreverc
creamy-pencil-82913
08/24/2022, 5:59 AMHave you looked at the kube-vip logs? This is all kube-vip stuff, not rke2 specific really.
n
narrow-noon-75604
08/24/2022, 6:00 AMI guess this might be because of etcd latency as mentioned in a similar issue - https://github.com/Azure/aks-hci/issues/167
Please find the logs of 3 VIP pods -
$ kubectl logs kube-vip-ds-554th -n kube-system
time="2022-08-24T06:03:39Z" level=info msg="Starting <http://kube-vip.io|kube-vip.io> [v0.5.0]"
time="2022-08-24T06:03:39Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[true], Services:[true]"
time="2022-08-24T06:03:39Z" level=info msg="prometheus HTTP server started"
time="2022-08-24T06:03:39Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-08-24T06:03:39Z" level=info msg="beginning services leadership, namespace [kube-system], lock name [plndr-svcs-lock], id [<http://master3.192.168.10.73.nip.io|master3.192.168.10.73.nip.io>]"
I0824 06:03:39.600009 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-svcs-lock...
time="2022-08-24T06:03:39Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [<http://master3.192.168.10.73.nip.io|master3.192.168.10.73.nip.io>]"
I0824 06:03:39.600649 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-08-24T06:03:39Z" level=info msg="Node [<http://master2.192.168.10.72.nip.io|master2.192.168.10.72.nip.io>] is assuming leadership of the cluster"
time="2022-08-24T06:03:39Z" level=info msg="new leader elected: <http://master2.192.168.10.72.nip.io|master2.192.168.10.72.nip.io>"
--------------------------------------------------------------------------------------------------------
$ kubectl logs kube-vip-ds-rqhn5 -n kube-system
time="2022-08-24T06:05:17Z" level=info msg="Starting <http://kube-vip.io|kube-vip.io> [v0.5.0]"
time="2022-08-24T06:05:17Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[true], Services:[true]"
time="2022-08-24T06:05:17Z" level=info msg="prometheus HTTP server started"
time="2022-08-24T06:05:17Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-08-24T06:05:17Z" level=info msg="beginning services leadership, namespace [kube-system], lock name [plndr-svcs-lock], id [<http://master2.192.168.10.72.nip.io|master2.192.168.10.72.nip.io>]"
I0824 06:05:17.604621 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-svcs-lock...
time="2022-08-24T06:05:17Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [<http://master2.192.168.10.72.nip.io|master2.192.168.10.72.nip.io>]"
I0824 06:05:17.605137 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-08-24T06:05:17Z" level=info msg="new leader elected: <http://master1.192.168.10.71.nip.io|master1.192.168.10.71.nip.io>"
time="2022-08-24T06:05:17Z" level=info msg="Node [<http://master3.192.168.10.73.nip.io|master3.192.168.10.73.nip.io>] is assuming leadership of the cluster"
--------------------------------------------------------------------------------------------------------
$ kubectl logs kube-vip-ds-vdwdq -n kube-system
time="2022-08-24T06:03:33Z" level=info msg="Starting <http://kube-vip.io|kube-vip.io> [v0.5.0]"
time="2022-08-24T06:03:33Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[true], Services:[true]"
time="2022-08-24T06:03:33Z" level=info msg="prometheus HTTP server started"
time="2022-08-24T06:03:33Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-08-24T06:03:33Z" level=info msg="beginning services leadership, namespace [kube-system], lock name [plndr-svcs-lock], id [<http://master1.192.168.10.71.nip.io|master1.192.168.10.71.nip.io>]"
I0824 06:03:33.407599 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-svcs-lock...
time="2022-08-24T06:03:33Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [<http://master1.192.168.10.71.nip.io|master1.192.168.10.71.nip.io>]"
I0824 06:03:33.408068 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-08-24T06:03:33Z" level=info msg="new leader elected: <http://master2.192.168.10.72.nip.io|master2.192.168.10.72.nip.io>"
time="2022-08-24T06:03:33Z" level=info msg="Node [<http://master2.192.168.10.72.nip.io|master2.192.168.10.72.nip.io>] is assuming leadership of the cluster"
time="2022-08-24T06:05:07Z" level=info msg="Node [<http://master3.192.168.10.73.nip.io|master3.192.168.10.73.nip.io>] is assuming leadership of the cluster"
I0824 06:05:11.368625 1 leaderelection.go:258] successfully acquired lease kube-system/plndr-svcs-lock