https://rancher.com/ logo
Title
a

agreeable-hamburger-18931

02/24/2023, 6:49 PM
Hi everyone! I need to deploy K3s in rootless mode (security reasons) and I'm facing a lot of problems making it work as should be. Anyone tried to have a K3s rootless working on production despite being an experimental feature?
m

melodic-umbrella-19641

02/24/2023, 7:48 PM
I did some test deployments of it for someone, I had trouble getting agent nodes to connect. I only tested the server node for a few hours.
a

agreeable-hamburger-18931

02/24/2023, 8:12 PM
Hi Philip, thanks for your answer. I'm surprised that only a few people tested this feature, specially because in production environments, securing the host by avoiding using the root on the Kubernetes processes is a no-brainer thing. I hope more people can join us on this discussion so we can share experiences. Technically speaking, I managed to make it work, with a lot of effort but I'm dealing with some things that are not working (like for example, the deployment of the apps on the cluster, exposing the NVIDIA GPUs on the containers , and making the metrics server work...among others) by reading the logs. Let's see how far I can go with this.
c

creamy-pencil-82913

02/24/2023, 8:51 PM
You can’t run multinode clusters rootless
It is essentially a toy option at this point. Many things are broken.
@best-accountant-68201 is the primary maintainer of the Rootless Kubernetes stuff, he might have some advice but for anything beyond a very simple single-node installation it is unlikely to do what you want.
b

best-accountant-68201

02/24/2023, 9:05 PM
Multi-node rootless Kubernetes is possible with flannel. You can find a Docker Compose demo here: https://github.com/rootless-containers/usernetes Not ported to k3s yet though
c

creamy-pencil-82913

02/24/2023, 9:15 PM
Last time I tried that with K3s I couldn’t get it working 100% - the join itself was OK as was the CNI, but stuff like metrics-server that wants to connect to the kubelet doesn’t work because it can’t reach the other network ns.
which is I think what was reported up above
I am also curious about stuff like GPU attachment, I wonder if thats possible without root.
b

best-accountant-68201

02/24/2023, 10:00 PM
Theoretically yes, as it has been possible with Docker and Podman
m

melodic-umbrella-19641

02/24/2023, 11:06 PM
This is really interesting. Thanks Brandon and Akihiro