where does k3s store the client ca and in how much pain woul rancher-users #k3s

#k3s

Title

nutritious-apartment-10061

09/28/2022, 5:16 PM

where does k3s store the client-ca and in how much pain would I be if I rotate it?

nutritious-tomato-14686

09/29/2022, 9:01 PM

K3s can auto rotate the certs https://docs.k3s.io/advanced#certificate-rotation

The cert is located in

var/lib/rancher/k3s/server/tls/client-ca.crt

if you want to go messing around with it, but I wouldn't recommend it

You can also use the

k3s certificate

subcommand to do the rotation on demand

Process is not documented (TODO for me), but you 1. stop the k3s server 2. run

k3s certificate rotate

3. start the K3s server

nutritious-apartment-10061

09/30/2022, 5:56 AM

k3s certificate rotate specifically doesn’t rotate that one, that's why I’m here :)

literally removing that file makes k3s recreate it

nutritious-tomato-14686

09/30/2022, 7:11 PM

Yeah I chatted with @fancy-guitar-13855 (he wrote the cert rotation command) and he mentioned that you can just delete that file and K3s will make a new one.

nutritious-apartment-10061

09/30/2022, 7:12 PM

https://github.com/k3s-io/k3s/blob/master/pkg/cli/cert/cert.go#L125-L184 I can't find the place in the code where it removes the client ca though

and in here https://github.com/k3s-io/k3s/issues/5147#issuecomment-1048269049 it's stated that the client ca is indeed not rotated

I'm fine if I need to go thorugh the hoops to rotate it; I just want to know which those involve 🙂

nutritious-tomato-14686

09/30/2022, 7:14 PM

So I got more info and we specifically don't rotate any ca* certs because its a Back-Compat thing with RKE1. When you removed the file, K3s regenerated a new one?

nutritious-apartment-10061

09/30/2022, 7:16 PM

nope; it restored the old cert from someplace

I stopped k3s, rm-rf'ed the client-ca*, started it and got the same client-ca back

that's still confusing as to how that works

nutritious-tomato-14686

09/30/2022, 8:36 PM

Okay I did some more digging. Currently, there is no easy way to regenerate the client or server -ca keys/crts. This is because we cache the keys in the etcd/db. They are static for the life of that cluster. If you kill the cluster and remove the files, on restart, k3s will simple pull down the cached version from the DB and regenerate them.

The only way to get new keys is to completely wipe out the K3s cluster...

k3s server --cluster-reset

or a k3s-uninstall.sh.... but that defeats the point of trying to "rotate" the ca keys.

nutritious-apartment-10061

09/30/2022, 9:57 PM

aha, I thought they'd be in etcd; that explains it.

a bit unfortunate; but yeah; wiped the cluster, learned my lesson. No more CA auth.

144 Views