This message was deleted.
# rke2a
adamant-kite-43734
05/15/2023, 10:54 PMThis message was deleted.
m
magnificent-vr-88571
05/15/2023, 10:59 PMFacing following error
Copy code
Normal SandboxChanged 6h18m (x26 over 6h23m) kubelet Pod sandbox changed, it will be killed and re-created.
Warning FailedMount 6h15m (x7 over 6h16m) kubelet MountVolume.SetUp failed for volume "webhook-cert" : object "kube-system"/"rke2-ingress-nginx-admission" not registered
Warning NetworkNotReady 6h15m (x18 over 6h16m) kubelet network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
Warning FailedMount 6h2m (x7 over 6h2m) kubelet MountVolume.SetUp failed for volume "webhook-cert" : object "kube-system"/"rke2-ingress-nginx-admission" not registered
Warning NetworkNotReady 5h52m (x302 over 6h2m) kubelet network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
Warning FailedMount 5h46m kubelet MountVolume.SetUp failed for volume "webhook-cert" : failed to sync secret cache: timed out waiting for the condition
Normal SandboxChanged 5h39m (x25 over 5h46m) kubelet Pod sandbox changed, it will be killed and re-created.
Warning FailedMount 5h34m kubelet MountVolume.SetUp failed for volume "webhook-cert" : failed to sync secret cache: timed out waiting for the condition
Normal SandboxChanged 5h32m (x12 over 5h34m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal SandboxChanged 38m (x10 over 40m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal SandboxChanged 4m (x28 over 9m41s) kubelet Pod sandbox changed, it will be killed and re-created.magnificent-vr-88571
05/15/2023, 10:59 PMI see above error in many pods
magnificent-vr-88571
05/15/2023, 11:00 PMany recommended steps to recover
magnificent-vr-88571
05/15/2023, 11:01 PMI am using cilium as CNi, but cilium itself not coming up 😞
magnificent-vr-88571
05/18/2023, 11:43 PMMistake I made in the environment. Missed CNI update in the agent node config file to be converted to server node. This lead to default cni helm installed and entire environment got trouble.
Recovered entire environment by following steps.
1) restored last required etcd in the server.
2) added all server node after cleaning up etcd eb and snapshot
3) added all agents
It’s back to normal and entire environment is stable
248 Views