Rancher-managed RKE2 control-plane node keeps old node IP after DHCP IP change Hi Rancher community, I have a Rancher-provisioned RKE2 production cluster running on VMware vSphere with 3 control-plane/etcd nodes and 3 worker nodes. My control-plane VMs use DHCP. One CP VM currently has: Hostname:

rke2-prod1-cp-k4wl9-cqd2t

Current VM/Linux IP:

192.168.168.117

MAC:

00:50:56:95:c3:b6

vCenter,

hostname -I

, and

ip addr

all correctly show

192.168.168.117

. However, the kubelet process is still running with the old IP:

--hostname-override=rke2-prod1-cp-k4wl9-cqd2t
--node-ip=192.168.168.111

Rancher therefore also shows the node with the old

.111

address instead of the VM's current

.117

address. I checked:

grep -Rni -- '--node-ip\|node-ip:' /etc/rancher/rke2 ...

No result.

/etc/rancher/rke2/config.yaml

does not exist, and:

systemctl show rke2-server --property=Environment

returns an empty

Environment=

. The cluster uses a stable HAProxy/Keepalived API/registration VIP (

192.168.168.20

) for ports 6443/9345, so replacement control-plane VMs can receive new DHCP addresses. Question: In a Rancher-provisioned RKE2 cluster, where is the kubelet

--node-ip

generated/stored when it is not present in

config.yaml

or the systemd environment? More importantly, what is the Rancher-supported way for a control-plane VM to automatically update its Kubernetes/RKE2 node IP when its DHCP address changes, without manually modifying RKE2 configuration on every node? I want to understand whether Rancher/RKE2 expects a control-plane node's IP to remain stable for the lifetime of that Machine, and whether changing the IP of an existing etcd/control-plane Machine is supported. Thanks for any guidance.

Don't run kubernetes with dhcp, unless you can guarantee stable IPs. Etcd cluster depend on fixed IPs.

I've put in a reserved address back to what it originally registered as and rebooted to fix this before.

Rancher is usually replacing nodes if it's e.g. resizing them or changing an image. k8s nodes (and therefore RKE2 nodes too) assume that their IP address is not changing over the node lifetime.... regardless of whether it's DHCP or not.

@many-summer-65979 - i have same setup of rancher managed using DHCP working env same as yours, never got into this issue. 1. Can you check in DHCP server to see what exactly the ip reserved for this VM? do you perform fixedaddres reservation for these vms? probably we are hitting ip overalap issue 2. what does command k get endpoints shows?

No, you don't. Dhcp assignments come with a lease-time. Clients confirming they're still using that assigned IP reset the lease-time to a full new period, called dhcp renew. A machine not renewing its IP in time looses it, for example when it's offline for too long, or when dhcp got disabled after initial setup. A dhcp server with a short lease time can cause such issues. A (properly functioning) dhcp server will not hand out an actively used IP, if the user didn't mess it up, like by using fixed IP addresses from a dhcp pool. What to look for:

❯ ip a show dev br0
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 70:85:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 192.168.xx.11/24 metric 1024 brd 192.168.xx.255 scope global dynamic br0
       valid_lft 58367sec preferred_lft 58367sec
    inet6 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 86209sec preferred_lft 86209sec
    inet6 fe80::7285:c2ff:fef1:1148/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever

valid_lft 58367sec

means the currently assigned IP is valid for another 16.2 hours before the dhcp server considers it as unassigned. Your client will ask the dhcp server for a renew after 50% of the lease-time has passed, and a second time after 87.5%. Issues are usually caused by: • clients using the dhcp assigned IP after installation as fixed ip without renewal • users assigning fixed ips from dhcp pools • dhcp servers loosing their stored leases-list, for example by rebooting or restarting without persistent storage