This message was deleted.
# k3sa
adamant-kite-43734
02/04/2025, 4:52 AMThis message was deleted.
c
creamy-pencil-82913
02/04/2025, 5:14 AMWhat you you mean by “got removed from the cluster unexpectedly”. There is nothing built into k3s that deletes nodes. What removed the node?
b
brash-cpu-62691
02/04/2025, 5:17 AMYup we are also not sure how it got removed automatically from the cluster
Logs also dont show any error
Logs just contain network drivers went in promiscuous and disabled state and after that node not found
"Error updating node status, will retry" err="error getting node "node-worker-5": nodes "node-worker-5" not found"c
creamy-pencil-82913
02/04/2025, 5:19 AMwhat cloud provider are you using? usually thats the only thing that will delete nodes
creamy-pencil-82913
02/04/2025, 5:19 AMeither that or someone did it manually
b
brash-cpu-62691
02/04/2025, 5:19 AMits AWS
c
creamy-pencil-82913
02/04/2025, 5:19 AMand you have deployed the aws cloud provider?
b
brash-cpu-62691
02/04/2025, 5:19 AMNode is still there
We restarted the VM and it joined back the clsuter
c
creamy-pencil-82913
02/04/2025, 5:20 AMyes the node will join automatically but if there were localpath volumes or other things bound specifically to that node, recreating it won’t help.
👀 1
creamy-pencil-82913
02/04/2025, 5:20 AMyou need to figure out what deleted it and stop that from happening.
b
brash-cpu-62691
02/04/2025, 5:20 AMThe days count started from scratch
Previously it was 418 days and now its 3 days
c
creamy-pencil-82913
02/04/2025, 5:21 AMfind out what deleted it. it is not k3s, it is something else you have deployed. sounds like probably aws cloud controller
b
brash-cpu-62691
02/04/2025, 5:22 AMok got it
I will debug more in logs
I will update here if I find anything
Also can we have a call tomorrow if you are available?
I can show you logs
c
creamy-pencil-82913
02/04/2025, 6:40 AMno, sorry. You can follow up here on Slack though.