let me list the steps I took

on one control node:

mkdir -p /opt/k3s/server/tls
curl -sL <https://github.com/k3s-io/k3s/raw/master/contrib/util/generate-custom-ca-certs.sh> | DATA_DIR=/opt/k3s bash -
k3s certificate rotate-ca --path=/opt/k3s/server

I didn't copy over the credentials as I wanted a new root CA I then ran

systemctl restart k3s

I expected things to start breaking by now, so nothing happened, when and ran

systemctl restart k3s

on the other 2 control nodes on the cluster, still nothing

figured maybe it only rotated the root CA but not the other certs in use, so I went and ran the normal cert rotate on all the control nodes again, still nothing

I thought maybe I screwed something up so I re-did the whole thing, made sure path

/opt/k3s/server/tls

did not exist, repeated the same steps, no issues on cluster

ok so looking at

/var/lib/rancher/k3s/server/tls/server-ca.crt

it seems the certs were never rotated, they're still issued at the day I created the server

not sure if relevant, but also found the certs from my previous attempt when I followed rotating self-signed certs and the certs there do not match either, so I assume that did not change it either? at least on the file system

another thing I tried is deleting moving

/var/lib/rancher/k3s/server/tls/

into another folder, and it generated the same old certificates, so I assume the old certs in the store didnt actually get rotated

so I've done more testing and looked at the source code, and I think the problem is that im using embedded etcd, and rotate-ca doesn't seem to be working with it? the rotate-ca calls to cluster.Save() here which is defined here and the comment states it does not work with embedded etcd

opened an issue here https://github.com/k3s-io/k3s/issues/8952