https://rancher.com/ logo
#rke
Title
# rke
h

happy-photographer-15068

03/03/2023, 5:24 PM
Hello, I’m hoping someone could assist or might have additional context to an issue we had. We have a cluster with separate nodepools for controlplane, etcd and workers. Our nodes are hosted in Openstack and we need to rebuild some of the nodes with updated nodeTemplate values. When we deleted 1x ControlPlane and 1x Etcd, new nodes were active but “waiting to register with k8s” but Rancher was stuck on error (I don’t have the specific error unfortunately but I recall with “Failed to communicate with API server” and also “connection refused in xyz session”). We tried to create redudancy by applying controlplane and etcd roles to one of other healthy nodepool in the same cluster via Rancher GUI. Doing so made things worst and Rancher started to “Remove” (but unable to complete and stuck) all etcd nodes. This caused our cluster to go into locked state and we followed [this](https://www.suse.com/support/kb/doc/?id=000020695) to restore our etcd from snapshot Would anyone seen similar issues or possibly be able to assist in understanding why Rancher was “removed” all etcd nodes? Info: Rancher: 2.4.8 etcd: 3.4.15