heya, we recently had a problem with a node outage and in the process of getting the guest cluster back to work properly we ended up deleting a vm (read= guest cluster node). there are however volumes still attached to the node that was deleted. the clean-up process was somewhat manual and now the harvester csi driver is reporting

Failed to remove volume pvc-306e5ac4-ca7a-4938-af0b-e549279bb97e from node test-worker-1-6c79d7c7-hwdxc: <http://virtualmachine.kubevirt.io|virtualmachine.kubevirt.io> \“test-worker-1-6c79d7c7-hwdxc\” not found”

for certain volumes. This makes sense as the node no longer exists, can we manually mark them as detached?

Hello, I was hoping some of you might be able to provide some advice on the intended maintenance/automation workflow to a newcomer. Any pointers or criticisms are welcome and greatly appreciated. Apologies if this isn't the right forum for this discussion. Background: I'm experimenting with Harvester for running some basic development infrastructure (i.e. gitlab, harbor, artifactory). What I'm trying to achieve is a workflow something like: 1. Create hardened VM ISO(s) (rocky linux), 2. Upload ISO(s) to Harvester, 3. Create VM(s) in harvester, 4. Perform application specific install (i.e. install package, open firewall, enable services). My initial thinking was to use: 1. Packer, 2+3. Terraform, 4. ansible. From a few hours of research I haven't seen much in the way of packer plugin support for Harvester. The current alternative is: 1. Kickstart creation or manually create the hardened snapshot, 2. Terraform+cloud-init to deploy the ISOs and setup VMs, 3. Ansible with additional hardening steps and application install. I'm not horribly against the second approach, but would ideally like to avoid maintaining kickstart configs. Basically my questions are: 1. Am I completely wrong in my overall approach, 2. Is either option feasible, or is there something else you'd recommend?

@great-magician-20621 i've had some luck using packer+qemu to harden an existing cloud image. if you do that and upload it to harvester, terraform + some inline cloud-init configuration will get you a VM ready for applications quickly.

Is anyone able to shed some light on where this validating function is run? Trying to add an NFS backup target, and have verified that both the harvester-webhook pods and longhorn-manager pods can reach the server/port, so a bit stumped here

admission webhook "<http://validator.harvesterhci.io|validator.harvesterhci.io>" denied the request: cannot mount nfs nfs.infra:/data/harvester-test: vers=4.0: mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t nfs4 -o nfsvers=4.0,actimeo=1,soft,timeo=300,retry=2 nfs.infra:/data/harvester-test /var/lib/longhorn-backupstore-mounts/nfs_infra/data/harvester-test
Output: mount.nfs4: Network is unreachable
: vers=4.1: mount failed: exit status 32

Does anyone know if the Rancher integration with Harvester is one-to-one, or can it be many-to-one? My use case: our lab has a single Harvester instance. We have a Rancher "test/dev" deployment in our lab that is integrated with the single lab Harvester instance. We have recently deployed a Rancher "prod" deployment in our lab, and would like to use the same Harvester instance as a cloud provider for the new Rancher "prod" instance as well as the Rancher "test/dev" instance. Is this possible, or would there be a conflict within the Rancher "Virtualization Management" section (two Rancher instances controlling one Harvester instance) as I have to import the cluster into the second Rancher instance?

Does Harvester support Intel SGX, TDX, and GPU functionalities?

One of my three Harvester nodes keeps on being set as cordoned. I'd like to be able to investigate myself. Other than seeing that events have happened, there's nothing much in the way of debugging tools in the Harvester UI. What's the best way for me to investigate?

Quick question. Is anyone looking at building a Vagrant provider for Harvester? I use Vagrant quite a bit for automated integration testing and can't find any info as to whether this (Vagrant provider for Harvester) is a 'thing'.

Howdy howdy; during an upgrade from 1.0.3 to 1.1.1 everything has worked up to the point where the last worker node is stuck running post-drain. The rke2 agent on the stuck node connects just fine to the rke2 server, however the rke2 agent's API proxy on port 6443 times out with no tls handshake received whenever anything on the node tries to talk. As a result kubelet (configured to connect via rke2 api proxy on 127.0.0.1:6443) wont start. Any ideas on how to resolve? The troubleshooting guide says dont start another upgrade from this state: The upgrade is completely finished except for this worker node. Have tried adjusting the kubelet config in /var/lib/rancher/rke2/agent/kubelet.kubeconfig to point to the harvester VIP, which allows the node to join the cluster again, however much still expects the API proxy to work, so the post-drain job cannot complete

it seems harvester could not support AMD hypervisor hosts with nested virtualization enabled. Could someone please confirm this point? after reading slacks/issues reported, it sees it only supports Intel hypervisors hosts with nested virtualization enabled? this seems to be a major bug??

Is it possible to install cluster api kubevirt provider on top of Harvester as is, and then use that to create vanilla k8s cluster on top of Harvester?

I mistakenly created this post on #general - https://rancher-users.slack.com/archives/C3ASABBD1/p1686427812327499 Any thoughts here?

Hi all, anyone met the situation when I tried to install Harvester-1.1.2rc1 via USB to raw server，in the network setting screen, I selected NIC then pressed 'Enter' ，it would back to the previous screen…

HI everyone, I have an issue with storage :

so question. i have a vm that has 64gb of memory assigned to it. it's a worker node in an rke2 cluster (provisioned through harvester) if i put a heavy load on that machine, and cause the memory to balloon on a specific pod, instead of rke2 killing that pod, harvester ends up killing the entire vm due to oom. any recommendations on how i can make it so rke kills it instead of harvester?

Hi I have a management node and it has two IP's one is the VIP and the other is the node IP, when I join the node it says that the x509 doesn't have one IP, I added this in the /etc/rancher/rke2/ config dir under sni but how do I regenerate the certs is bit confusing Also for the advanced docs do we see Harvester docs or rancher docs? Thanks

I tried rancherd bootstrap --force

Is anyone encountering instances of stalled longhorn snapshots during harvester backups?

Can I expect post-drain job for a node to restart if I kill the executing job? I have one that’s been sat at “Finished moving /run/initramfs/cos-state/cOS/active.img to /run/initramfs/cos-state/cOS/passive.img” for half an hour where previous nodes were done within a couple of seconds after that message

I am trying to register rancher on a bare metal harvester install and I am getting this error. Any suggestions where to troubleshoot this? Do i need to open the required ports within the docker container running on Ubutu server full 22.04, which is running on harvester?

quick question: is it possible to mount an HDD directly to a VM bypassing Longhorn?

Just a question, currently there is no XFS stuff on the host nodes, so I can't actually mount my downloaded disks is the recommended way something that's mentioned above a VM and then pci passthrough or can we just mount it to the host.

also is there a way to directly add the disks I have on the system into the images store?

Just installed harvester and am checking it out. I am wondering, where can I get VM images? What images will work and is there any repositories of images that you guys would recommend? I've heard of bitnami, will images from there work?

I recently installed harvester and imported it into rancher. I've noticed that there is two storage classes, harvester-longhorn and longhorn. What is the difference? Why are there two storage classes?

Does SAP VMs hosted on Harvester supported by SAP? (Does Hardester certified by SAP as a virtualization platform?)

Hello! I am trying to import vm's from ESXi and the server says that Harvester logged into it. But when I get the vcsim it says clusterNoReady. I saw an old post in the archive about this with no resolution on March 3rd.

harvey-06:~ # kubectl get VmwareSource  vcsim
NAME    STATUS
vcsim   clusterNotReady

Hi, how can I demote and promote node manually in a 3+ nodes cluster?

Also, is it possible to store VMs across multiple drives?