08/10/2022, 4:05 PM
Hi - I’m wondering if anyone can enlighten me regarding the coupling between e.g. persistent volumes in rke2 clusters running on top of VMs provisioned in harvester by rancher. We’ve gotten ourselves into the misfortunate situation of having all new nodes in our cluster and seeing the backing longhorn volumes disappear in the process. We still have backups from s3, but we’re unable to get the rke2 cluster to use volumes restored from backups (with pv/pvcs created via the action menu in longhorn).
after much struggling around I’ve attempted to: 1. Restore from backup via longhorn UI, create a volume (not pvc) 2. Create a new pvc in my downstream workload cluster 3. Delete the resulting pvc in the harvester cluster 4. Create a new pvc with the expected name(?) in harvester, pointing to a longhorn-restored volume (restored volumes for some reason are of Filesystem type)
I’ve then changed the workload claim ref to point to this new pvc, and I see the volume being mounted in a hp-pod in harvester, on the same harvester node as my downstream cluster workload runs on
this leaves me with an event
MountVolume.SetUp failed for volume "pvc-d5036b85-5c1b-4a2a-89be-fea0000ff149" : rpc error: code = Aborted desc = The hot-plug volume pvc-d5036b85-5c1b-4a2a-89be-fea0000ff149 is not ready
on the workload pod