This message was deleted.

I don't think Harvester exposes a user-friendly way for configuring this. @great-bear-19718 do you know?

Have you done much testing with SR-IOV in Harvester with multiple interfaces and/or VMs? During our testing, we ran into issues with KubeVirt mapping the wrong SR-IOV interfaces to VMs or mapping them in the wrong order due to a couple bugs that are fixed in later 0.54 versions, but not rolled into the images used by Harvester. This is just a heads up in case you run into issues that you can't explain. Here is one KubeVirt bugfix link: https://github.com/kubevirt/kubevirt/issues/7444

that seems a different use case.. because we pass the VF as a pcidevice the kubevirt networking is not leveraged and it is upto the guest OS to configure the same

The bug we ran into has to do with the mapping of the VFs to the VMs. If you have more than one physical VF or more than one VM, the mapping may be incorrect and the wrong VF may get presented to the wrong VM. Just a heads up. Your situation may not run into the issue, the bug I linked was one of the issues. We had 2 physical, multiple VF's on each and multiple VMs and sometimes one VM would get mapped 2 VFs from the same physical when we wanted 1 VF from each physical. Examining the libvirt xml file inside the container confirmed we were hitting the KubeVirt bug.

I can try and check it out.. i assume that could happen since device plugin advertises resource as VENDOR_TYPE.. and if you have a lot of VF's with same underlying PF types they just show up under the same device type

The other bug linked in that one has more details about the mixups. https://github.com/kubevirt/kubevirt/issues/6351

i can see what this would happen.. just the way the resource request is mapped when launcher pod is created..

The bugfix was backported to KubeVirt 0.54, but as far as I can tell, the bugfix was never pulled into the 0.54 image(s) that Harvester uses.

i will check.. 1.3.0 will bump up kubevirt to 1.x

Nice. 🙂

so might have to wait till then.. i will try and check internally with a custom build and see if the fix helps

Check out the new link I posted, it gives more details about the mapping fix. I have not tried it, I'm still in the early stages of learning Kubernetes, so didn't even attempt to upgrade KubeVirt on my own to test.

that link is not relevant since that is not how harvester implements this

OK, I'm not 100% sure about the core issue/fix, but I did confirm things were not mapped correctly on our system by spinning up multiple VMs with multiple VFs from different physical NICs. I had someone using a VM which wasn't getting hooked up to the network it was supposed to be hooked to and tracked it down by looking at the libvirt xml file. The mapping in the VM's XML did not match the mapping in the Harvester UI.

thanks indeed.. I think i know what is causing this behaviour i will see if we can do something differently about it

@thousands-action-31843 no I have just started using Harvester and also quite new to SR-IOV. I have another system that does VLAN tagging this way, and looking to replicate that with Harvester. @great-bear-19718 do you have any comment on the original question in the thread about SR-IOV VLAN tagging?

the host cannot do anything to the VF.. as it is passed like any other pcidevice

Thankyou for the clarification. I think this might be a common use case for anyone wanting to do multi-tenancy where SR-IOV is required (e.g. telco). However it seems like it would require a major change to the way SR-IOV is handled by Harvester... Should I create a feature request ticket?

sure.. as much info on what you are trying to achieve would really help

fyi https://github.com/harvester/harvester/issues/4800