This message was deleted.
# k3s
a
This message was deleted.
c
That shouldn't happen. You should be able to restart kubelet/containerd/k3s without affecting the pods. I would probably investigate what specifically is causing it to go into an error state.
b
So the pod's process will continue running even after k3s/containerd dies or is stopped?
c
Correct. That's why we bundle a killall script with k3s.
b
interesting peak behind the covers! Good info.
c
that is how things should work on all distros
stopping containerd or the kubelet won’t stop pods
e
right but restarting k3s server process restarts the kubelet which makes device plugins re-register their devices, thus pods using those "old devices", get rediscovered/created, my question is quite specific to device plugins, and given k3s is often times run single-node, curious if others have seen this
c
why do the device plugins re-register whenever the kubelet starts? there’s no way for the devices to persist across kubelet runs?
e
it's supposed to, i believe it's due to this bug https://github.com/kubernetes/kubernetes/pull/118635
which looks to have a backport made to 1.27 (https://github.com/kubernetes/kubernetes/pull/119432)
altho I sure wish I could rebuild rke2 locally and ensure this actually fixes our issue
@creamy-pencil-82913 you all offer a way to build pre-releases of rke2 ?
c
its all on GH…
e
what i mean is, can a non suse employee build a local release, the docs I see appear to be for internal use
c
you can’t kick one off on our infrastructure, no. But you can fork/clone the repos and build your own locally.
e
okay, do you have a doc you can point me to that would allow me to more or less take (https://github.com/kubernetes/kubernetes/pull/119432) and build a binary locally that I can then put somewhere to test. That would give this week a beautiful ending 🙂
c
you would need to clone and build https://github.com/rancher/image-build-kubernetes from a fork of kubernetes that has that commit applied
then push those images to a docker registry
then clone and build rancher/rke2, with the hardened-kubernetes image references modified to use the images from your docker registry