This message was deleted.
# harvester
a
This message was deleted.
b
.
b
It was always a bit of a gamble for me. Certain parts of the UI bound the correct device, and others did not. Are you using the the Harvester UI directly or the plugin via Rancher?
✅ 1
n
cc: @great-bear-19718 for inputs
g
a support bundle would be nice to help us understand the state of the cluster
the error indicates it cannot find the device to vfio driver and it could be for any number of reasons.. like nvidia driver toolkit running which is blocking gpu from being bound to vfio..
as always info like harvester version etc are also useful
b
Currently, I am using Harvester UI , | Harvester version: v1.8 nvidia-driver-toolkit is enabled but its not running state, because of image
@great-bear-19718, Thanks for taking a look. Could you please share your email details? I will share the support bundle there
g
please create a GH issue and attach it to the issue
b
for now this has been ok , The fix was enabling passthrough on all non-GPU siblings in the IOMMU group (including a PCIe card that wasn't initially obvious), then rebooting the node to get clean vfio-pci binding. After that the VM powered on fine. Notice below points along the way worth keeping in mind: • Stale device allocation annotations linger on the VM even after removal • Duplicate PCIDevice CRs can auto-generate • Removed PCI devices stay in the VM YAML and block other VMs from claiming them
b
I've seen that before where some cards have an audio device that has to have pci pass through enabled (though they don't have to be assigned to the vm)
g
would still have been nice to check the support bundle to figure out why it ended up in that state
i also have access to L40s GPU's and have never run into such an issue
👍 1
b
@great-bear-19718 Fair point, there may be a few areas I'm overlooking
what I've repeatedly noticed is that the PCI device entries keep appearing in the VM's
deviceAllocationDetails
annotation even after the devices are removed from the spec.
In some cases, the device I attach in the UI is not the device that actually shows up at launch the annotation carries a different (older) entry.