I am trying to roll through my rke2 cluster and do OS updates. After rebooting each of my 3 server nodes independently I have found that the longhorn pods are not coming up on those hosts, but are working on the other 7 hosts. Not even seeing it attempt to bring the pods up, No image errors or crash loops. Rebooting one of my worker nodes has no problem restarting the longhorn engine/manager/csi, etc.

by design longhorn pods will only run on agent (worker) nodes. https://longhorn.io/docs/1.9.0/concepts/#1-design

Thank you for the response. Ok, well then that behavior seems counter to what I have seen on 2 other clusters, but I imagine a reboot of those servers would yield the same result. There are no issues reported by kubectl for the longhorn namespace. I have one worker node that had a taint on it that was preventing longhorn startup, but removing the taint and applying it after longhorn started seems to have made that node happy. I asked Google AI and it gave me this nonsense: "Yes, Longhorn can and should run on RKE2 server nodes. It's a common practice to deploy Longhorn as a storage backend for Kubernetes clusters, including those running RKE2." So I was confused. It doesnt seem necessary to me to run it on a server node that has no workloads, but maybe it makes sense for some of the control plane pods to run on the server nodes?

I am not surprised; there are number of things GoogleAI has been wrong on

Yeah I saw an article on how to run it across all nodes, including server, but I am not looking to do anything non-standard.