[solved] Service LoadBalancer stuck pending waiting for external IP. The problem was that I'd selected "default" namespace when I created the vip-pool in Harvester UI Settings, but the Service resource wasn't in default namespace. For my case, the best solution was to change the vip-pool to "global" so the IPs can be allocated for any Harvester namespace. I hope this helps the next person.

Does anyone have an idea on how to view SMART value? It seems directly install smartmontools via SSH is not an option.

doc issues on https://docs.harvesterhci.io/v1.1/networking/clusternetwork The diagram does not seem to match the text description of what hosts belong to what network configs and I'm not sure what else. Could be improved.

Hello, I had only one master VM in the cluster. When I tried to add 2 more masters, they got stuck on “waiting for kubernetes register…” for a day. Then I tried to remove those 2 VMs (reduced the pool size to back 1) Now this message won’t go away and new worker also won’t register (screenshot) : “waiting for nj-k8s-master2 to finish provisioning” If I do

k get nodes

, this nodes does not show up. Can someone advise on how to get out of this state ?

Hi

Is there any way to install harvester into an existing rancher cluster instead of installing it on baremetal with ISOs?

I found some older docs that reference install Harvester as a Rancher Catalog App, but now it seems like this is no longer an option

Quick question: Has anyone gotten MetalLB working with Harvester?

Re: arm64 support I'm curious about how you would use arm64 support. In a separate cluster or multi-arch cluster? Multi-arch created in Rancher? The most useful implementation of this feature would allow a multi-arch Harvester cluster where kubevirt VMs are scheduled according to arch taints and tolerations. I would like to use this to manage multi-arch clusters with Rancher on Harvester cloud provider. That way, I could team all available aarch64 and x86_64 metal with Harvester. A complication is that the arm64 and amd64 cloud image variants must be very similar to ensure node uniformity. Is there a multi-arch manifest format for cloud images we could use? I'm not aware of one, but it seems like a necessary standard to describe cloud image artifacts of the same version for each target architecture. This could be metadata only like a checksum file describing cloud image artifacts that were built from the same source files.

Hello, There is an open issue on failure to install harvester 1.1.1 from ISO file using IDRAC (virtual CD mount) It was supposed to be solved in 1.1.2. It’s not.

How to add a new disk to longhorn? - I guess I have to manually format as ext4 and create a mountpoint from cli or?

image.png

and will it do “stripe” automatically on each nodes?

after ext4 format and mount I can select disk - but it shows 2 entries?

Hello, anyone came across this issue ? After adding Harvester to Rancher, I tried to provision a new CLUSTER with the Harvester cloud provider and facing this:

@hundreds-evening-84071 has left the channel

How can I install xfs on the nodes? - filesystem is RO

Dell Poweredge R440,H350 Raid Controller, RAID 10, CPU 10c/20th, 128GB RAM. I have to reinstall Harvester to get an error message though. Same issue as @lively-zoo-40381 didn't see before posting.

I'm installing via virtual DVD over a fiber connection.

Can Harvesters kubernetes setup be used as a normal cluster or is it adviced only to have it running for the purpose of “harvester”?

Hi Everyone: 🎉 Announcement: 🎉 I am happy to announce the release of Harvester CLI v0.2.0. CHANGELOG: 🚀 Enhancements Added the possibility to pass on files to cloud-init using new flags called --user-data-filepath and --network-data-filepath. The previous flags --cloud-init-user-data and --cloud-init-network-data (which were not clearly a configMap reference) become user-data-cm-ref and network-data-cm-ref. the *-cm-ref and *-filepath are mutually exclusive! -- Issue #24 Added support for the vm-import-controller, a plugin to import VMs from VMWare vCenter or OpenStack environments. Using Harvester CLI, you can enable the Addon with harvester import enable, define or delete a source cluster, as well as create or delete a VM import -- Issue #21 🐛 Bugs Fixed a bug where the harvester CLI could not create VMs with more than 6 CPUs. Side effect of the change: now, the harvester vm list command does not show weird memory values (caused by overCommit settings in Harvester). -- Issue #23 Please test it and let me know if I can do any improvements! (edite

Deployment:

harvester-cloud-provider

has two pods. One is Running, the other is Containercreating with events:

Unable to attach or mount volumes: unmounted volumes=[cloud-config], unattached volumes=[cloud-config kube-api-access-7mqbm]: timed out waiting for the condition
MountVolume.SetUp failed for volume "cloud-config" : hostPath type check failed: /etc/kubernetes/cloud-config is not a file

This occurred after modifying the RKE2 guest cluster layout in Rancher.

/etc/kubernetes/cloud-config

doesn't exist on any node, so it's not clear why the provider pod is a able to start in some but not all cases.

Hello, When trying to add a node pool to and RKE cluster (Harvester) I see that there is a limitation to 32 cores for Harvester Vms… Is this by design ?

Hello. I am looking for some help with the pci Devices. Yesterday I installed three nodes and enabled the pci device controller. On two nodes it worked fine but on one the pod constantly restarts with an error I dont really understand. If I run lspci in the Pod it works perfectly fine. The log of the Pod:

time="2023-05-08T08:44:02Z" level=info msg="Applying CRD <http://pcidevices.devices.harvesterhci.io|pcidevices.devices.harvesterhci.io>"
time="2023-05-08T08:44:02Z" level=info msg="Applying CRD <http://pcideviceclaims.devices.harvesterhci.io|pcideviceclaims.devices.harvesterhci.io>"
time="2023-05-08T08:44:11Z" level=info msg="Registering PCI Device Claims controller"
time="2023-05-08T08:44:12Z" level=info msg="Loading driver vfio-pci"
time="2023-05-08T08:44:12Z" level=info msg="Loading driver vfio_iommu_type1"
time="2023-05-08T08:44:12Z" level=info msg="Registering PCI Devices controller"
time="2023-05-08T08:44:12Z" level=info msg="add mutation handler for [pods]. (Pod)"
time="2023-05-08T08:44:12Z" level=info msg="add mutation handler for [virtualmachines].<http://kubevirt.io|kubevirt.io> (VirtualMachine)"
time="2023-05-08T08:44:12Z" level=info msg="Active TLS secret harvester-system/pcidevices-webhook-tls (ver=25321) (count 1): map[<http://listener.cattle.io/cn-pcidevices-webhook.harvester-system.svc:pcidevices-webhook.harvester-system.svc|listener.cattle.io/cn-pcidevices-webhook.harvester-system.svc:pcidevices-webhook.harvester-system.svc> <http://listener.cattle.io/fingerprint:SHA1=06E69757A946BC22422ACA20E153E1DBBCBCF903]|listener.cattle.io/fingerprint:SHA1=06E69757A946BC22422ACA20E153E1DBBCBCF903]>"
time="2023-05-08T08:44:12Z" level=info msg="Listening on :8443"
I0508 08:44:13.628762       1 trace.go:205] Trace[911902081]: "DeltaFIFO Pop Process" ID:harv3-0000ff0c0,Depth:30,Reason:slow event handlers blocking the queue (08-May-2023 08:44:13.430) (total time: 197ms):
Trace[911902081]: [197.83893ms] [197.83893ms] END
time="2023-05-08T08:44:20Z" level=info msg="Starting /v1, Kind=Node controller"
time="2023-05-08T08:44:20Z" level=info msg="Starting <http://network.harvesterhci.io/v1beta1|network.harvesterhci.io/v1beta1>, Kind=VlanConfig controller"
time="2023-05-08T08:44:20Z" level=info msg="Starting <http://devices.harvesterhci.io/v1beta1|devices.harvesterhci.io/v1beta1>, Kind=PCIDevice controller"
time="2023-05-08T08:44:20Z" level=info msg="Starting <http://devices.harvesterhci.io/v1beta1|devices.harvesterhci.io/v1beta1>, Kind=PCIDeviceClaim controller"
time="2023-05-08T08:44:33Z" level=info msg="Reconciling PCI Devices list"
I0508 08:44:42.430276       1 trace.go:205] Trace[646203300]: "Reflector ListAndWatch" name:<http://github.com/harvester/pcidevices/go/pkg/mod/k8s.io/client-go@v0.23.7/tools/cache/reflector.go:167|github.com/harvester/pcidevices/go/pkg/mod/k8s.io/client-go@v0.23.7/tools/cache/reflector.go:167> (08-May-2023 08:44:17.628) (total time: 24801ms):
Trace[646203300]: ---"Objects listed" error:<nil> 24801ms (08:44:42.429)
Trace[646203300]: [24.801374748s] [24.801374748s] END
I0508 08:44:42.628726       1 trace.go:205] Trace[1747278511]: "DeltaFIFO Pop Process" ID:<http://recurringjobs.longhorn.io|recurringjobs.longhorn.io>,Depth:43,Reason:slow event handlers blocking the queue (08-May-2023 08:44:42.528) (total time: 100ms):
Trace[1747278511]: [100.017068ms] [100.017068ms] END
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1344034]

goroutine 31 [running]:
<http://github.com/harvester/pcidevices/pkg/util/nichelper.IdentifyHarvesterManagedNIC({0xc00005a00a|github.com/harvester/pcidevices/pkg/util/nichelper.IdentifyHarvesterManagedNIC({0xc00005a00a>, 0x5}, {0x1f0c1f0, 0xc001cb64b0}, {0x1f0bc40, 0xc00279bb30})
	/go/src/github.com/harvester/pcidevices/pkg/util/nichelper/helper.go:52 +0x454
<http://github.com/harvester/pcidevices/pkg/controller/pcidevice.Register({0x1f0b968|github.com/harvester/pcidevices/pkg/controller/pcidevice.Register({0x1f0b968>, 0xc002719fc0}, {0x7f1403b36f28, 0xc0004222a0}, 0xc00050c0d8, 0xc00050c0e0)
	/go/src/github.com/harvester/pcidevices/pkg/controller/pcidevice/pcidevice_controller.go:69 +0x345
main.run.func2()
	/go/src/github.com/harvester/pcidevices/main.go:148 +0x72
<http://golang.org/x/sync/errgroup.(*Group).Go.func1()|golang.org/x/sync/errgroup.(*Group).Go.func1()>
	/go/src/github.com/harvester/pcidevices/go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:75 +0x64
created by <http://golang.org/x/sync/errgroup.(*Group).Go|golang.org/x/sync/errgroup.(*Group).Go>
	/go/src/github.com/harvester/pcidevices/go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:72 +0xa5
time="2023-05-08T08:45:18Z" level=info msg="Applying CRD <http://pcidevices.devices.harvesterhci.io|pcidevices.devices.harvesterhci.io>"
time="2023-05-08T08:45:19Z" level=info msg="Applying CRD <http://pcideviceclaims.devices.harvesterhci.io|pcideviceclaims.devices.harvesterhci.io>"
time="2023-05-08T08:45:29Z" level=info msg="Registering PCI Device Claims controller"
time="2023-05-08T08:45:29Z" level=info msg="Loading driver vfio-pci"
time="2023-05-08T08:45:30Z" level=info msg="Loading driver vfio_iommu_type1"
time="2023-05-08T08:45:30Z" level=info msg="Registering PCI Devices controller"
time="2023-05-08T08:45:30Z" level=info msg="add mutation handler for [pods]. (Pod)"
time="2023-05-08T08:45:30Z" level=info msg="add mutation handler for [virtualmachines].<http://kubevirt.io|kubevirt.io> (VirtualMachine)"
time="2023-05-08T08:45:30Z" level=info msg="Active TLS secret harvester-system/pcidevices-webhook-tls (ver=25321) (count 1): map[<http://listener.cattle.io/cn-pcidevices-webhook.harvester-system.svc:pcidevices-webhook.harvester-system.svc|listener.cattle.io/cn-pcidevices-webhook.harvester-system.svc:pcidevices-webhook.harvester-system.svc> <http://listener.cattle.io/fingerprint:SHA1=06E69757A946BC22422ACA20E153E1DBBCBCF903]|listener.cattle.io/fingerprint:SHA1=06E69757A946BC22422ACA20E153E1DBBCBCF903]>"
time="2023-05-08T08:45:30Z" level=info msg="Listening on :8443"
time="2023-05-08T08:45:36Z" level=info msg="Starting <http://network.harvesterhci.io/v1beta1|network.harvesterhci.io/v1beta1>, Kind=VlanConfig controller"
time="2023-05-08T08:45:36Z" level=info msg="Starting <http://devices.harvesterhci.io/v1beta1|devices.harvesterhci.io/v1beta1>, Kind=PCIDevice controller"
time="2023-05-08T08:45:36Z" level=info msg="Starting /v1, Kind=Node controller"
time="2023-05-08T08:45:36Z" level=info msg="Starting <http://devices.harvesterhci.io/v1beta1|devices.harvesterhci.io/v1beta1>, Kind=PCIDeviceClaim controller"
I0508 08:45:36.429320       1 trace.go:205] Trace[208240456]: "DeltaFIFO Pop Process" ID:harv3-000000043,Depth:108,Reason:slow event handlers blocking the queue (08-May-2023 08:45:36.229) (total time: 200ms):
Trace[208240456]: [200.154497ms] [200.154497ms] END
time="2023-05-08T08:45:51Z" level=info msg="Reconciling PCI Devices list"
I0508 08:45:56.328655       1 trace.go:205] Trace[646203300]: "Reflector ListAndWatch" name:<http://github.com/harvester/pcidevices/go/pkg/mod/k8s.io/client-go@v0.23.7/tools/cache/reflector.go:167|github.com/harvester/pcidevices/go/pkg/mod/k8s.io/client-go@v0.23.7/tools/cache/reflector.go:167> (08-May-2023 08:45:35.629) (total time: 20601ms):
Trace[646203300]: ---"Objects listed" error:<nil> 20601ms (08:45:56.230)
Trace[646203300]: [20.601417428s] [20.601417428s] END
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1344034]

goroutine 121 [running]:
<http://github.com/harvester/pcidevices/pkg/util/nichelper.IdentifyHarvesterManagedNIC({0xc00005a00a|github.com/harvester/pcidevices/pkg/util/nichelper.IdentifyHarvesterManagedNIC({0xc00005a00a>, 0x5}, {0x1f0c1f0, 0xc0028381b0}, {0x1f0bc40, 0xc0032552c0})
	/go/src/github.com/harvester/pcidevices/pkg/util/nichelper/helper.go:52 +0x454
<http://github.com/harvester/pcidevices/pkg/controller/pcidevice.Register({0x1f0b968|github.com/harvester/pcidevices/pkg/controller/pcidevice.Register({0x1f0b968>, 0xc002f81d40}, {0x7fdb05fbb640, 0xc00029fe30}, 0xc000704650, 0xc000704658)
	/go/src/github.com/harvester/pcidevices/pkg/controller/pcidevice/pcidevice_controller.go:69 +0x345
main.run.func2()
	/go/src/github.com/harvester/pcidevices/main.go:148 +0x72
<http://golang.org/x/sync/errgroup.(*Group).Go.func1()|golang.org/x/sync/errgroup.(*Group).Go.func1()>
	/go/src/github.com/harvester/pcidevices/go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:75 +0x64
created by <http://golang.org/x/sync/errgroup.(*Group).Go|golang.org/x/sync/errgroup.(*Group).Go>
	/go/src/github.com/harvester/pcidevices/go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:72 +0xa5

"Disk cd is CD-ROM, needs to be ejected before migration" Is this new with v1.1.2 - I've always been able to migrate this VM in its current state before, and this worries me for future upgrades.

is longhorn doing raid0 between nodes or is the maxium volume claim < biggest drive?

Speaking of RAID. Is it best to set up hardware RAID in the bios and use it as one drive in Harvester, or just leave them as separate drives and let Harvester manage them?

how can we configure software raid for harvester installation target?

how big is the VM cidr subnet? - and if using masquarde is it possible to deploy a VM/pod with in the same /24 subnet? - I see all my VMs have the same 10.0.2.2/24

can I run a sidecar container with a VM?