This message was deleted.
# k3sa
adamant-kite-43734
09/12/2024, 9:36 PMThis message was deleted.
f
future-fountain-82544
09/12/2024, 9:37 PMThe purple line is resource=events, and the high green line is resource=secrets
c
creamy-pencil-82913
09/12/2024, 9:39 PMsomething (probably a pod) is trying to watch a future revision. It is likely very confused because the current revision is not retained when you switch datastores.
f
future-fountain-82544
09/12/2024, 9:39 PMahhhh ok
c
creamy-pencil-82913
09/12/2024, 9:39 PMRestart the nodes, or just do a k3s-killall.sh and then start the services again.
f
future-fountain-82544
09/12/2024, 9:39 PMkk
future-fountain-82544
09/12/2024, 9:39 PMOr I can go dig around and start deleting pods 😆
c
creamy-pencil-82913
09/12/2024, 9:40 PMyeah if you want to be more surgical you can go see what pod has watch errrors in its logs
f
future-fountain-82544
09/12/2024, 9:40 PMIs that the same revision that shows up in metadata when you do
kubectl get ...future-fountain-82544
09/12/2024, 9:40 PMI might just out of curiosity
future-fountain-82544
09/12/2024, 9:40 PMty for the help
c
creamy-pencil-82913
09/12/2024, 9:40 PMno
✅ 1
creamy-pencil-82913
09/12/2024, 9:41 PMrevision is a global change counter for etcd that is incremented any time any resource is created/altered/deleted
f
future-fountain-82544
09/12/2024, 9:46 PMAhhh ok. Hah, so if I just wait long enough it'll fix itself once the change counter gets up high enough :P
j/k - I'm not willing to wait that long. Plus I kinda want to see what's doing it. Wonder if that'd show up in the pod logs too
future-fountain-82544
09/12/2024, 9:48 PMIt's interesting how some errors/warnings bubble up in kube API clients and come through almost the same as they show on the server. Guessing that's something that's built into the rpc libraries.
future-fountain-82544
09/12/2024, 9:55 PMso far: longhorn-manager, rancher-webhook, fleet-agent
21 Views