https://rancher.com/ logo
Title
s

stale-painting-80203

02/27/2023, 9:29 PM
After rebooting my harvester node, I am unable to restart a VM. I see the following error, but not sure what caused this.
Events:
  Type     Reason              Age                    From                     Message
  ----     ------              ----                   ----                     -------
  Normal   Scheduled           5m57s                  default-scheduler        Successfully assigned default/virt-launcher-sle-15-sp4-base-9zjw2 to harvester-01
  Warning  FailedMount         3m54s                  kubelet                  Unable to attach or mount volumes: unmounted volumes=[disk-0 disk-1], unattached volumes=[private disk-0 cloudinitdisk-ndata container-disks hotplug-disks sockets cloudinitdisk-udata public ephemeral-disks libvirt-runtime disk-1]: timed out waiting for the condition
  Warning  FailedAttachVolume  102s (x10 over 5m57s)  attachdetach-controller  AttachVolume.Attach failed for volume "pvc-87fe9694-9c15-49fd-adc5-2bbe7337ce85" : PersistentVolume "pvc-87fe9694-9c15-49fd-adc5-2bbe7337ce85" is marked for deletion
  Warning  FailedAttachVolume  102s (x10 over 5m57s)  attachdetach-controller  AttachVolume.Attach failed for volume "pvc-e9108404-e61d-402a-a553-62610bd987a9" : PersistentVolume "pvc-e9108404-e61d-402a-a553-62610bd987a9" is marked for deletion
  Warning  FailedMount         99s                    kubelet                  Unable to attach or mount volumes: unmounted volumes=[disk-0 disk-1], unattached volumes=[public hotplug-disks libvirt-runtime cloudinitdisk-ndata sockets disk-0 private ephemeral-disks container-disks cloudinitdisk-udata disk-1]: timed out waiting for the condition

Did a describe on the pvc:
Name:          sle-15-sp4-base-disk-0-6oyol
Namespace:     default
StorageClass:  longhorn-image-d48sr
Status:        Bound
Volume:        pvc-87fe9694-9c15-49fd-adc5-2bbe7337ce85
Labels:        <none>
Annotations:   <http://harvesterhci.io/imageId|harvesterhci.io/imageId>: default/image-d48sr
               <http://harvesterhci.io/owned-by|harvesterhci.io/owned-by>: [{"schema":"kubevirt.io.virtualmachine","refs":["default/sle-15-sp4-base"]}]
               <http://pv.kubernetes.io/bind-completed|pv.kubernetes.io/bind-completed>: yes
               <http://pv.kubernetes.io/bound-by-controller|pv.kubernetes.io/bound-by-controller>: yes
               <http://volume.beta.kubernetes.io/storage-provisioner|volume.beta.kubernetes.io/storage-provisioner>: <http://driver.longhorn.io|driver.longhorn.io>
               <http://volume.kubernetes.io/storage-provisioner|volume.kubernetes.io/storage-provisioner>: <http://driver.longhorn.io|driver.longhorn.io>
Finalizers:    [<http://kubernetes.io/pvc-protection|kubernetes.io/pvc-protection> <http://provisioner.storage.kubernetes.io/cloning-protection|provisioner.storage.kubernetes.io/cloning-protection>]
Capacity:      20Gi
Access Modes:  RWX
VolumeMode:    Block
Used By:       virt-launcher-sle-15-sp4-base-9zjw2
Events:        <none>


Name:          sle-15-sp4-base-disk-1-upd7u
Namespace:     default
StorageClass:  harvester-longhorn
Status:        Bound
Volume:        pvc-e9108404-e61d-402a-a553-62610bd987a9
Labels:        <none>
Annotations:   <http://harvesterhci.io/owned-by|harvesterhci.io/owned-by>: [{"schema":"kubevirt.io.virtualmachine","refs":["default/sle-15-sp4-base"]}]
               <http://pv.kubernetes.io/bind-completed|pv.kubernetes.io/bind-completed>: yes
               <http://pv.kubernetes.io/bound-by-controller|pv.kubernetes.io/bound-by-controller>: yes
               <http://volume.beta.kubernetes.io/storage-provisioner|volume.beta.kubernetes.io/storage-provisioner>: <http://driver.longhorn.io|driver.longhorn.io>
               <http://volume.kubernetes.io/storage-provisioner|volume.kubernetes.io/storage-provisioner>: <http://driver.longhorn.io|driver.longhorn.io>
Finalizers:    [<http://kubernetes.io/pvc-protection|kubernetes.io/pvc-protection> <http://provisioner.storage.kubernetes.io/cloning-protection|provisioner.storage.kubernetes.io/cloning-protection>]
Capacity:      60Gi
Access Modes:  RWX
VolumeMode:    Block
Used By:       virt-launcher-sle-15-sp4-base-9zjw2
Events:        <none>
f

full-plastic-79795

02/28/2023, 7:42 AM
Hi, what is the version of Harvester?
s

stale-painting-80203

02/28/2023, 7:43 AM
1.1.0
f

full-plastic-79795

02/28/2023, 2:55 PM
There are many possible reasons. Could you please generate a support bundle and put it into an issue for further investigation? Thank you.