Whats the value of encrypting secrets with the AES-CBC provider when the encryption key secret is stored in plaintext? As far as I can tell, that is the default behaviour for RKE2?

they’re encrypted at rest, so not retrievable from an etcd snapshot, or by someone who gains direct access to the etcd datastore. Not without the key at least.

i see, that makes sense, but if the host node persistent storage hardware is compromised then secrets are also compromised, correct?

if someone compromises the servers then yes, it is game over for your cluster as a whole.

just to clearify, i meant compromise the storage devices, not a live server. As all volatile memory is gone.

what do you mean by “storage devices”, and how are you using them.

as in harddrives. if at the end of the lifecycle we dispose of the harddrives. The data on them should not be decipherable.

… you should always scrub your devices before getting rid of them, this doesn’t change that any. The servers (by necessity) have a copy of all the CA private keys, secrets encryption config, admin credentials, and so on. There’s no getting around that, at some point you need to store privileged data on disk as part of the system configuration.

yes, i agree, we are not planning to sell the device, but still we don't want to rely on that process alone. Also there is the scenario of theft of the physical device. WE also may be looking at cases where we repurpose the hardware internally. Regarding the TLS private keys, are stored as kubernetes secrets?

no, the private keys exist outside the cluster. They are the keys for the Kubernetes internal PKI

okay makes sense, thank you. So reading that content, if I understand it correctly, if we provide a custom root-ca, and then rotate the ca any servers that where the ca wasn't rotated will permanently loose access.

I would personally just burn the cluster if you have a server node compromised. Cattle not pets. Don’t make your life more complicated than it needs to be.

that works too, although much harder to execute, especially as the size of the cluster grows, but the data stored on the persistent volumes (harvester - longhorn) still needs to be encrypted and there should be no key on the same disk.