Well this is interesting... the 3rd control node just came back and is healthy now
future-fountain-82544
05/22/2024, 2:54 PM
Alright - went to the 2nd control node in /etc/systemd/system/k3s.service and updated the command line args to specify the 3rd control node with --server and that node is now healthy
future-fountain-82544
05/22/2024, 2:55 PM
aaaaand I went to the 1st control node, added the --token and --server lines and now it's healthy
future-fountain-82544
05/22/2024, 2:55 PM
Props to the kubernetes/etcd/k3s devs. I thought that cluster was smoking charred remains
future-fountain-82544
05/22/2024, 2:56 PM
Anyway -- LMK if there are other resources I could have / should have looked at first. I'm curious what others would have done to recover the cluster