Hi I have Harvester 1 4 1 installed and and I have enabled a Rancher Users #harvester

Hi. I have Harvester 1.4.1 installed and and I hav...

flat-librarian-14243

03/03/2025, 8:32 PM

Hi. I have Harvester 1.4.1 installed and and I have enabled a PCI device but I can't remove it. I can click on the X but nothing happens. Yes I have tested with the machine turned off. Also it seems like the VM does not mount the PCI device. Is there any way I can mount it from the CLI? Or get some better logs?

Copy code

{"component":"virt-launcher","level":"warning","msg":"MDEV_PCI_RESOURCE_AMD_COM_AQUA_VANJARAM_INSTINCT_MI300X not set for resource <http://amd.com/AQUA_VANJARAM_INSTINCT_MI300X%22,%22pos%22:%22addresspool.go:51%22,%22timestamp%22:%222025-03-03T19:59:53.182416Z%22}|amd.com/AQUA_VANJARAM_INSTINCT_MI300X","pos":"addresspool.go:51","timestamp":"2025-03-03T19:59:53.182416Z"}>
{"component":"virt-launcher","level":"warning","msg":"USB_RESOURCE_AMD_COM_AQUA_VANJARAM_INSTINCT_MI300X not set for resource <http://amd.com/AQUA_VANJARAM_INSTINCT_MI300X%22,%22pos%22:%22addresspool.go:51%22,%22timestamp%22:%222025-03-03T19:59:53.182500Z%22}|amd.com/AQUA_VANJARAM_INSTINCT_MI300X","pos":"addresspool.go:51","timestamp":"2025-03-03T19:59:53.182500Z"}>

flat-librarian-14243

03/03/2025, 8:35 PM

I use this when booting:

Copy code

Command line: BOOT_IMAGE=(loop0)/boot/vmlinuz console=tty1 root=LABEL=COS_STATE cos-img/filename=/cOS/active.img panic=0 net.ifnames=1 rd.cos.oemlabel=COS_OEM rd.cos.mount=LABEL=COS_OEM:/oem rd.cos.mount=LABEL=COS_PERSISTENT:/usr/local rd.cos.oemtimeout=120 audit=1 audit_backlog_limit=8192 intel_iommu=on amd_iommu=on iommu=pt multipath=off modprobe.blacklist=amdgpu vfio-pci.ids=1002:74a1

👍 1

flat-librarian-14243

03/03/2025, 9:00 PM

Verified that sr-iov is enabled in the BIOS as well.

flat-librarian-14243

03/04/2025, 12:35 AM

IOMMU also enabled in BIOS.

flat-librarian-14243

03/04/2025, 12:43 AM

iommu from dmesg:

Copy code

dmesg | grep -i IOMMU
[    0.000000] Command line: BOOT_IMAGE=(loop0)/boot/vmlinuz console=tty1 root=LABEL=COS_STATE cos-img/filename=/cOS/active.img panic=0 net.ifnames=1 rd.cos.oemlabel=COS_OEM rd.cos.mount=LABEL=COS_OEM:/oem rd.cos.mount=LABEL=COS_PERSISTENT:/usr/local rd.cos.oemtimeout=120 audit=1 audit_backlog_limit=8192 intel_iommu=on amd_iommu=on iommu=pt multipath=off modprobe.blacklist=amdgpu vfio-pci.ids=1002:74a1
[    0.020470] Kernel command line: BOOT_IMAGE=(loop0)/boot/vmlinuz console=tty1 root=LABEL=COS_STATE cos-img/filename=/cOS/active.img panic=0 net.ifnames=1 rd.cos.oemlabel=COS_OEM rd.cos.mount=LABEL=COS_OEM:/oem rd.cos.mount=LABEL=COS_PERSISTENT:/usr/local rd.cos.oemtimeout=120 audit=1 audit_backlog_limit=8192 intel_iommu=on amd_iommu=on iommu=pt multipath=off modprobe.blacklist=amdgpu vfio-pci.ids=1002:74a1
[    0.020599] DMAR: IOMMU enabled
[    2.064999] iommu: Default domain type: Passthrough (set via kernel command line)
[    2.117934] pci 0000:60:00.2: AMD-Vi: IOMMU performance counters supported
[    2.117979] pci 0000:40:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118000] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118029] pci 0000:20:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118076] pci 0000:e0:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118122] pci 0000:c0:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118160] pci 0000:80:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118199] pci 0000:a0:00.2: AMD-Vi: IOMMU performance counters supported
[    2.118351] pci 0000:00:00.3: Adding to iommu group 0
[    2.118425] pci 0000:00:01.0: Adding to iommu group 1
[    2.118469] pci 0000:00:01.1: Adding to iommu group 2
[    2.118542] pci 0000:00:02.0: Adding to iommu group 3
[    2.118614] pci 0000:00:03.0: Adding to iommu group 4
[    2.118687] pci 0000:00:04.0: Adding to iommu group 5
[    2.118773] pci 0000:00:05.0: Adding to iommu group 6
[    2.118907] pci 0000:00:07.0: Adding to iommu group 7
[    2.118949] pci 0000:00:07.1: Adding to iommu group 7
[    2.118990] pci 0000:00:07.2: Adding to iommu group 7
[    2.119092] pci 0000:00:14.0: Adding to iommu group 8
[    2.119135] pci 0000:00:14.3: Adding to iommu group 8
[    2.119425] pci 0000:00:18.0: Adding to iommu group 9
[    2.119466] pci 0000:00:18.1: Adding to iommu group 9
[    2.119508] pci 0000:00:18.2: Adding to iommu group 9
[    2.119550] pci 0000:00:18.3: Adding to iommu group 9
[    2.119591] pci 0000:00:18.4: Adding to iommu group 9
[    2.119633] pci 0000:00:18.5: Adding to iommu group 9
[    2.119674] pci 0000:00:18.6: Adding to iommu group 9
[    2.119716] pci 0000:00:18.7: Adding to iommu group 9
[    2.120007] pci 0000:00:19.0: Adding to iommu group 10
[    2.120051] pci 0000:00:19.1: Adding to iommu group 10
[    2.120094] pci 0000:00:19.2: Adding to iommu group 10
[    2.120136] pci 0000:00:19.3: Adding to iommu group 10
[    2.120179] pci 0000:00:19.4: Adding to iommu group 10
[    2.120224] pci 0000:00:19.5: Adding to iommu group 10
[    2.120267] pci 0000:00:19.6: Adding to iommu group 10
[    2.120309] pci 0000:00:19.7: Adding to iommu group 10
[    2.120351] pci 0000:01:00.0: Adding to iommu group 11
[    2.120396] pci 0000:02:00.0: Adding to iommu group 12
[    2.120438] pci 0000:02:01.0: Adding to iommu group 13
[    2.120480] pci 0000:02:02.0: Adding to iommu group 14
[    2.120527] pci 0000:02:1f.0: Adding to iommu group 15
[    2.120578] pci 0000:03:00.0: Adding to iommu group 16
[    2.120630] pci 0000:04:00.0: Adding to iommu group 17
[    2.120691] pci 0000:05:00.0: Adding to iommu group 18
[    2.120744] pci 0000:07:00.0: Adding to iommu group 19
[    2.120797] pci 0000:08:00.0: Adding to iommu group 20
[    2.120806] pci 0000:09:00.0: Adding to iommu group 7
[    2.120814] pci 0000:09:00.1: Adding to iommu group 7
[    2.120822] pci 0000:09:00.4: Adding to iommu group 7
[    2.120831] pci 0000:09:00.5: Adding to iommu group 7
[    2.120840] pci 0000:0a:00.0: Adding to iommu group 7
[    2.120848] pci 0000:0a:00.1: Adding to iommu group 7
[    2.120983] pci 0000:20:00.3: Adding to iommu group 21
[    2.121058] pci 0000:20:01.0: Adding to iommu group 22
[    2.121100] pci 0000:20:01.1: Adding to iommu group 23
[    2.121173] pci 0000:20:02.0: Adding to iommu group 24
[    2.121246] pci 0000:20:03.0: Adding to iommu group 25
[    2.121288] pci 0000:20:03.1: Adding to iommu group 26
[    2.121361] pci 0000:20:04.0: Adding to iommu group 27
[    2.121433] pci 0000:20:05.0: Adding to iommu group 28
[    2.121537] pci 0000:20:07.0: Adding to iommu group 29
[    2.121585] pci 0000:20:07.1: Adding to iommu group 29
[    2.121628] pci 0000:21:00.0: Adding to iommu group 30
[    2.121671] pci 0000:22:00.0: Adding to iommu group 31
[    2.121713] pci 0000:22:01.0: Adding to iommu group 32
[    2.121755] pci 0000:22:02.0: Adding to iommu group 33
[    2.121807] pci 0000:24:00.0: Adding to iommu group 34
[    2.121858] pci 0000:25:00.0: Adding to iommu group 35
[    2.121920] pci 0000:26:00.0: Adding to iommu group 36
[    2.121971] pci 0000:27:00.0: Adding to iommu group 37
[    2.122015] pci 0000:28:00.0: Adding to iommu group 38
[    2.122057] pci 0000:29:00.0: Adding to iommu group 39
[    2.122099] pci 0000:29:01.0: Adding to iommu group 40
[    2.122141] pci 0000:29:02.0: Adding to iommu group 41
[    2.122184] pci 0000:29:03.0: Adding to iommu group 42
[    2.122227] pci 0000:29:04.0: Adding to iommu group 43
[    2.122269] pci 0000:29:05.0: Adding to iommu group 44
[    2.122311] pci 0000:29:1f.0: Adding to iommu group 45
[    2.122382] pci 0000:2e:00.0: Adding to iommu group 46
[    2.122452] pci 0000:2e:00.1: Adding to iommu group 47
[    2.122504] pci 0000:30:00.0: Adding to iommu group 48
[    2.122513] pci 0000:31:00.0: Adding to iommu group 29
[    2.122521] pci 0000:31:00.1: Adding to iommu group 29
[    2.122657] pci 0000:40:00.3: Adding to iommu group 49
[    2.122739] pci 0000:40:01.0: Adding to iommu group 50
[    2.122784] pci 0000:40:01.1: Adding to iommu group 51
[    2.122857] pci 0000:40:02.0: Adding to iommu group 52
[    2.122931] pci 0000:40:03.0: Adding to iommu group 53
[    2.123004] pci 0000:40:04.0: Adding to iommu group 54
[    2.123077] pci 0000:40:05.0: Adding to iommu group 55
[    2.123181] pci 0000:40:07.0: Adding to iommu group 56
[    2.123233] pci 0000:40:07.1: Adding to iommu group 56
[    2.123276] pci 0000:41:00.0: Adding to iommu group 57
[    2.123319] pci 0000:42:00.0: Adding to iommu group 58

[    2.128643] pci 0000:c9:00.1: Adding to iommu group 127
[    2.128779] pci 0000:e0:00.3: Adding to iommu group 135
[    2.128854] pci 0000:e0:01.0: Adding to iommu group 136
[    2.128897] pci 0000:e0:01.1: Adding to iommu group 137
[    2.128970] pci 0000:e0:02.0: Adding to iommu group 138
[    2.129043] pci 0000:e0:03.0: Adding to iommu group 139
[    2.129118] pci 0000:e0:04.0: Adding to iommu group 140
[    2.129193] pci 0000:e0:05.0: Adding to iommu group 141
[    2.129328] pci 0000:e0:07.0: Adding to iommu group 142
[    2.129396] pci 0000:e0:07.1: Adding to iommu group 142
[    2.129465] pci 0000:e0:07.2: Adding to iommu group 142
[    2.129509] pci 0000:e1:00.0: Adding to iommu group 143
[    2.129552] pci 0000:e2:00.0: Adding to iommu group 144
[    2.129595] pci 0000:e2:01.0: Adding to iommu group 145
[    2.129638] pci 0000:e2:02.0: Adding to iommu group 146
[    2.129681] pci 0000:e2:1f.0: Adding to iommu group 147
[    2.129736] pci 0000:e3:00.0: Adding to iommu group 148
[    2.129791] pci 0000:e4:00.0: Adding to iommu group 149
[    2.129859] pci 0000:e5:00.0: Adding to iommu group 150
[    2.129948] pci 0000:e6:00.0: Adding to iommu group 151
[    2.130039] pci 0000:e6:00.1: Adding to iommu group 152
[    2.130096] pci 0000:e8:00.0: Adding to iommu group 153
[    2.130105] pci 0000:e9:00.0: Adding to iommu group 142
[    2.130114] pci 0000:e9:00.1: Adding to iommu group 142
[    2.130123] pci 0000:e9:00.4: Adding to iommu group 142
[    2.130132] pci 0000:ea:00.0: Adding to iommu group 142
[    2.130141] pci 0000:ea:00.1: Adding to iommu group 142
[    2.135653] pci 0000:60:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135670] pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135684] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135696] pci 0000:20:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135709] pci 0000:e0:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135722] pci 0000:c0:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135734] pci 0000:80:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.135747] pci 0000:a0:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.144795] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    2.144826] perf/amd_iommu: Detected AMD IOMMU #1 (2 banks, 4 counters/bank).
[    2.144865] perf/amd_iommu: Detected AMD IOMMU #2 (2 banks, 4 counters/bank).
[    2.144899] perf/amd_iommu: Detected AMD IOMMU #3 (2 banks, 4 counters/bank).
[    2.144928] perf/amd_iommu: Detected AMD IOMMU #4 (2 banks, 4 counters/bank).
[    2.144959] perf/amd_iommu: Detected AMD IOMMU #5 (2 banks, 4 counters/bank).
[    2.144991] perf/amd_iommu: Detected AMD IOMMU #6 (2 banks, 4 counters/bank).
[    2.145019] perf/amd_iommu: Detected AMD IOMMU #7 (2 banks, 4 counters/bank).

6 Views

Open in Slack

Previous Next