https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
m
Have you checked the connectivity to the NFS server by using the Harvester UI? If running v1.2.x, in the Harvester UI go to 
Settings >> backup-target >> Test connection
You can also access the Longhorn UI and check the status of this volume.
b
Hi, yes the nfs server connectivity is working fine and i have attached the screenshot FYR. Also, their is no volume created related to this backup. @many-nightfall-1806 Thanks for the reply
m
Can you access the NFS server and check if the image exists?
What does 
kubectl get storageclasses
look like?
b
I have share the result
m
I am not sure why the storage class is missing.
b
Thanks, what are the steps to follow to take VM backup with the image? So i will try one time to take backup and restore.
m
This is the procedure to restore a backup. https://docs.harvesterhci.io/v1.3/vm/backup-restore#restore-a-new-vm-using-a-backup I am still checking this issue....
Can you also share the Harvester support bundle?
b
m
Have you checked after completing the backup that the task finalized successfully? I can see this 
Pendind
status to the PVC.
Copy code
╰$ kubectl get pvc -A                                                                                              
NAMESPACE                  NAME                                                                                             STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
cattle-monitoring-system   alertmanager-rancher-monitoring-alertmanager-db-alertmanager-rancher-monitoring-alertmanager-0   Bound     pvc-67c718e0-3b14-4bcc-b631-5d2e06863327   5Gi        RWO            harvester-longhorn     27m
cattle-monitoring-system   prometheus-rancher-monitoring-prometheus-db-prometheus-rancher-monitoring-prometheus-0           Bound     pvc-8fcc91bd-6dac-4454-82cc-c5222f265183   50Gi       RWO            harvester-longhorn     27m
cattle-monitoring-system   rancher-monitoring-grafana                                                                       Bound     pvc-d6d0ad3b-d248-41ae-848a-a7ebb9d7c529   2Gi        RWO            harvester-longhorn     27m
default                    restore-ubuntu-32d9f19e-349c-4aa7-bccc-66390e5e33af-disk-0                                       Pending                                                                        longhorn-image-26bm7   27m
default                    restore-ubuntuurl-a69d8ab3-9719-4bc4-bce6-c7395e5e8eea-disk-0                                    Pending                                                                        longhorn-image-wrv9w   27m
I can also find this error.
Copy code
status:
    complete: false
    conditions:
    - lastTransitionTime: "2024-06-05T10:32:07Z"
      message: 'admission webhook "<http://mutator.harvesterhci.io|mutator.harvesterhci.io>" denied the request: <http://network-attachment-definitions.k8s.cni.cncf.io|network-attachment-definitions.k8s.cni.cncf.io>
        "vm-network" not found'
      reason: Error
      status: "False"
      type: InProgress
    - lastTransitionTime: "2024-06-05T10:32:07Z"
      message: 'admission webhook "<http://mutator.harvesterhci.io|mutator.harvesterhci.io>" denied the request: <http://network-attachment-definitions.k8s.cni.cncf.io|network-attachment-definitions.k8s.cni.cncf.io>
        "vm-network" not found'
      reason: Error
      status: "False"
      type: Ready
    restores:
    - persistentVolumeClaimSpec:
        metadata:
          creationTimestamp: "null"
          name: restore-ubuntuurl-a69d8ab3-9719-4bc4-bce6-c7395e5e8eea-disk-0
          namespace: default
        spec:
          accessModes:
          - ReadWriteMany
          resources:
            requests:
              storage: 30Gi
          storageClassName: longhorn-image-wrv9w
          volumeMode: Block
          volumeName: pvc-01cb7f8b-65d0-4447-a60a-7500324a60e9
      volumeBackupName: ubuntuurl-volume-ubuntuurl-disk-0-7qo2o
      volumeName: disk-0
"vm-network" not found'
I found only the mgmt network in the cluster.
b
yes, after seeing this error i have created a network cluster and VM network.
thanks for guide. so what i have did is that i have created the backing image with "*longhorn-image-wrv9w*" name then i have restored the VM's and you suggested to create a network cluster, finally i have my backup VM's are restored. thank you so much...
🙌 1
i will try same with this and i will let you know.
m
great! I am glad to help!
b
for your information
im getting this error
m
can you send a new support bundle?
b
sure
supportbundle_c602d554-9f34-45f5-a9d2-80814cadf5f1_2024-06-05T14-29-12Z.zip
supportbundle_c602d554-9f34-45f5-a9d2-80814cadf5f1_2024-06-05T14-29-12Z.zip
m
Both VMs ubuntu and ubuntuurl are in ErrorUnschedulable state. How are you trying to restore the backup? Are you trying to replace the existing VM or create a new one?
Copy code
╰$ k get virtualmachine -A -o wide          
NAMESPACE   NAME        AGE   STATUS               READY
default     ubuntu      16m   ErrorUnschedulable   False
default     ubuntuurl   16m   ErrorUnschedulable   False
a
the pvc is still pending. It looks like there is no storage class for this PVC neither a PV
The PVC needs to be bound to a PV to work. Do you know why there is no PV for the backup pvc?
The PVCs are 
Pending
Copy code
default                    restore-ubuntu-32d9f19e-349c-4aa7-bccc-66390e5e33af-disk-0                                       Pending                                                                        longhorn-image-26bm7   155m
default                    restore-ubuntuurl-a69d8ab3-9719-4bc4-bce6-c7395e5e8eea-disk-0                                    Pending                                                                        longhorn-image-wrv9w   155m
This is because there is no PV
Copy code
╰$ kubectl get pv   
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                                                                                     STORAGECLASS         REASON   AGE
pvc-67c718e0-3b14-4bcc-b631-5d2e06863327   5Gi        RWO            Delete           Bound    cattle-monitoring-system/alertmanager-rancher-monitoring-alertmanager-db-alertmanager-rancher-monitoring-alertmanager-0   harvester-longhorn            159m
pvc-8fcc91bd-6dac-4454-82cc-c5222f265183   50Gi       RWO            Delete           Bound    cattle-monitoring-system/prometheus-rancher-monitoring-prometheus-db-prometheus-rancher-monitoring-prometheus-0           harvester-longhorn            159m
pvc-d6d0ad3b-d248-41ae-848a-a7ebb9d7c529   2Gi        RWO            Retain           Bound    cattle-monitoring-system/rancher-monitoring-grafana                                                                       harvester-longhorn            159m
Checking your backup screenshot, it shows 3 pvc not bounded to a pv
Here is an example from a healthy backup
m
the sc 
longhorn-image-wrv9w
is missing, this SC is created for the matched image
Did you backup from one Harvester cluster and restore to another?
b
Hi Webber, I have only one host
as per the harvester backup guide, the backup is stored in the NFS server.
im using the Ubuntu cloud image & cloud-init config to run the VM.
m
@better-optician-77140 de backup restore is expecting for the 
storage class
named as 
longhorn-image-wrv9w
and 
longhorn-image-26bm7
. When the backup was created both storage classes were used to the VM and must be present to restore the backup. Can you try to created the storage classes manually?
Copy code
allowVolumeExpansion: true
apiVersion: <http://storage.k8s.io/v1|storage.k8s.io/v1>
kind: StorageClass
metadata:
  name: longhorn-image-wrv9w
parameters:
  migratable: "true"
  numberOfReplicas: "1"
  staleReplicaTimeout: "30"
provisioner: <http://driver.longhorn.io|driver.longhorn.io>
reclaimPolicy: Delete
volumeBindingMode: Immediate
Copy code
allowVolumeExpansion: true
apiVersion: <http://storage.k8s.io/v1|storage.k8s.io/v1>
kind: StorageClass
metadata:
  name: longhorn-image-26bm7
parameters:
  migratable: "true"
  numberOfReplicas: "1"
  staleReplicaTimeout: "30"
provisioner: <http://driver.longhorn.io|driver.longhorn.io>
reclaimPolicy: Delete
volumeBindingMode: Immediate
Wait for some minutes and the VMs should start.
Did you manually delete the storage classes?
b
@many-nightfall-1806 Yes, while trying to restore the VM that i found the error in the events that the "longhorn-image-wrv9w" image is not found, so I tried that I have created with storage-class manually "longhorn-image-wrv9w" with the URL "https://cloud-images.ubuntu.com/jammy/20240514/jammy-server-cloudimg-amd64.img"(earlier i have used same url to VM).
FYI - Backing image
image.png
This is how I have created a VM and backup the VM.
Screen sharing - 2024-06-07 9:55:26 AM.webm
m
The storage class and backing image should be created by the VM image controller, because it builds the connection between the VM image, backing image, and storage class. If you manually create a storage class for an image, you must ensure the connection among resources are correct.
b
@magnificent-pencil-261 yes, I got your point.
I have created a VM and created a backup as well and storaged in a NFS drive, I will reinstall the harvester and I will show you the exact issue.
Hi, I have reinstalled the harvester and tried to restore it, but now it is showing the "0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.."
m
Did you ever have the backing image 
image-ws4fp
and sc 
longhorn-image-ws4fp
in your cluster?
b
Hi Webber/Rodolfo de Almeida, i found the solution in the Longhorn doc : https://longhorn.io/docs/1.6.2/advanced-resources/backing-image/backing-image-backup/ i have restored the backing image by creating a yaml file with the below content and it was restored successfully. Thanks for helping me on this. Yaml File:
Copy code
apiVersion: longhorn.io/v1beta2
kind: BackingImage
metadata:
    name: parrot-restore
    namespace: longhorn-system
spec:
    sourceType: restore
    sourceParameters:
        # change to your backup URL
        # backup-url: <nfs://longhorn-test-nfs-svc.default>:/opt/backupstore?backingImage=parrot
        backup-url: <s3://backupbucket@us-east-1/?backingImage=parrot>
        concurrent-limit: "2"
    checksum: 304f3ed30ca6878e9056ee6f1b02b328239f0d0c2c1272840998212f9734b196371560b3b939037e4f4c2884ce457c2cbc9f0621f4f5d1ca983983c8cdf8cd9a
21 Views
Open in Slack
PreviousNext
Powered by