https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# rke2
a
This message was deleted.
c
they’re encrypted at rest, so not retrievable from an etcd snapshot, or by someone who gains direct access to the etcd datastore. Not without the key at least.
There is no support for storing the key in anything other than plaintext. That is just how the non-HSM encryption providers work.
p
i see, that makes sense, but if the host node persistent storage hardware is compromised then secrets are also compromised, correct?
we are actually trying to set up harvester with KMS provider, but so far couldn't get it to work.
c
if someone compromises the servers then yes, it is game over for your cluster as a whole.
that is true regardless of whether you use secrets encryption though. even if you use KMS, someone can just use kubectl to dump all the secrets out.
p
just to clearify, i meant compromise the storage devices, not a live server. As all volatile memory is gone.
c
what do you mean by “storage devices”, and how are you using them.
p
as in harddrives. if at the end of the lifecycle we dispose of the harddrives. The data on them should not be decipherable.
c
… you should always scrub your devices before getting rid of them, this doesn’t change that any. The servers (by necessity) have a copy of all the CA private keys, secrets encryption config, admin credentials, and so on. There’s no getting around that, at some point you need to store privileged data on disk as part of the system configuration.
Many companies crush or shred their server disks instead of selling them as surplus because they don’t want to worry about it. But again this is not a concern unique to RKE2 or any other Kubernetes distro.
p
yes, i agree, we are not planning to sell the device, but still we don't want to rely on that process alone. Also there is the scenario of theft of the physical device. WE also may be looking at cases where we repurpose the hardware internally. Regarding the TLS private keys, are stored as kubernetes secrets?
someone can just use kubectl to dump all the secrets out.
But would that require that they can get behind the firewall before we invalidate cluster access for the node by rotating the certificates.
c
no, the private keys exist outside the cluster. They are the keys for the Kubernetes internal PKI
That content also applies to RKE2, we just haven’t got around to migrating the content to the rke2 docs yet.
p
okay makes sense, thank you. So reading that content, if I understand it correctly, if we provide a custom root-ca, and then rotate the ca any servers that where the ca wasn't rotated will permanently loose access.
So that combined with using a KMS, and volume encryption, means we can can permanently lock out a node from a cluster render all its data as useless.
Or we could we could use an external etcd.
c
I would personally just burn the cluster if you have a server node compromised. Cattle not pets. Don’t make your life more complicated than it needs to be.
p
that works too, although much harder to execute, especially as the size of the cluster grows, but the data stored on the persistent volumes (harvester - longhorn) still needs to be encrypted and there should be no key on the same disk.
9 Views
Open in Slack
PreviousNext
Powered by