https://rancher.com/ logo
Title
l

loud-daybreak-83328

09/19/2022, 2:56 PM
Hi. I'm attempting to get Kasten backing up our environment and it seems to be having Longhorn snapshotting issues. I've looked through the various discussions and am not seeing an issue similar to what we have. We're on Rancher 2.6.8, Longhorn 1.3.1, and Kasten 5.0.8. The error I'm seeing when backing something up that is on a Longhorn PVC is: Failed to create snapshot: failed to take snapshot of the volume pvc-af7ba930-e103-468c-85e3-cdc28d2bebde: "rpc error: code = NotFound desc = volume id pvc-af7ba930-e103-468c-85e3-cdc28d2bebde does not exist in the volumes list". Any help you can offer would be fantastic. @bland-byte-60612
I also tried to manually create a snapshot with the CSI snapshooter (described https://longhorn.io/docs/1.1.0/snapshots-and-backups/csi-snapshot-support/create-a-backup-via-csi/). I got these errors: Failed to create snapshot content with error snapshot controller failed to update test-snapshot-pvc on API server: cannot get claim from snapshot. Not sure what that's telling me either.
Manually running a backup in longhorn does work, so my backup area is functional.
I got further on the manual snapshotting by making sure I identified the namespace that the pvc was on, but it's stuck Waiting for a snapshot namespace/test-snapshot-pvc to be created by the CSI driver.
Still no luck with doing a CSI snapshot of a longhorn volume. I recreated the cluster with RKE1 instead of RKE2 and no difference. I followed the specific directions on the latest longhorn documentation: https://longhorn.io/docs/1.3.1/snapshots-and-backups/csi-snapshot-support/enable-csi-snapshot-support/. Used the same version of the snapshotter, and nothing. Kubernetes accepts the snapshot config but it just sits, and longhorn does not actually do a snapshot.
I think I'm in business now. Part of my issue was that I forgot a couple of namespace changes in the external-snapshotter configuration and it was making it not work across the entire cluster.
👍 1
m

modern-pilot-62631

12/13/2022, 11:02 PM
Hi KT, I am having issues as well. What were the namespace changes that you needed to use in the external-snapshotter configuration? thanks
l

loud-daybreak-83328

12/14/2022, 6:01 PM
Not sure if this is correct or not (it's been awhile), I modified (in the external-snapshotter directory) deploy/kubernetes/snapshot-controller/*.yaml and changed all instances of namespace: default to namespace: kube-system
🙌 1
m

modern-pilot-62631

12/14/2022, 10:20 PM
Ah ok in fact that rings a bell, I must have seen something similar. I will give that a try thanks
It seems now by default it is set to kube-system anyways. I will try again and check the logs.