rancher-users #harvester

harvester

dry-tiger-79777

06/19/2022, 8:28 AM

Howdy y'all, I am working on a PoC with Harvester to try to do some GPU pass through into a VM, and I believe that I am either missing a step or this is misconfigured, because I have done this loads of times in Proxmox or just bare QEMU KVM. I have followed the docs on changing GRUB params, kubevirt's docs on enabling physical device passthrough, and someone's gist on the subject (for nvidia's plugin, since the one on the official repo is outdated now). Attaching some debug info, IOMMU is enabled fine, but it seems that i'm not able to get VFIO assignment working in Harvester, hoping to see if anyone else has successfully navigated this or has pointers.

nvidia-pod.txt

broad-crayon-52589

06/24/2022, 12:16 AM

how to rese harvester dashboard password? or find password ?

sparse-art-48043

06/28/2022, 7:28 PM

i spun up a quick coredns to fix the routing issue but I can't get harvester to trust the self-signed cert. ugh

jolly-musician-63586

06/29/2022, 7:35 PM

Hi community, new to harvester and was trying to spin up an ubuntu VM instance after installing the OS on the bare metal. However, I ran into the following issues:

0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.

I did some basic hardware checks, such as disk storage (the machine has 512 nvme for OS and 4T for VM data), but still have no idea why it still give me this error. Anyone know what could go wrong here? Everything is default out of box for now. Really appreciate it!🙏

🤔 1

freezing-farmer-7156

06/30/2022, 2:59 PM

I'm pretty new to this and love your project "Harvester" but I'm missing some things, and do not find doc on these topics. I posted this also in the channel #harvester-dev So I have a couple of questions on my harvester setup : 1. I have a node but not all local disk storage was assigned during install. Is there a ways to extend my current local disk/storage with the unused/un-assigned disk storage? If so can this be done from within the UI or can/do I have to do this through the CLI? 2. Is there a way to make a shared disk pool between all my nodes as shared storage, like a NFS/CIFS/SMB. Do I need Longhorn? if so, on all nodes? Thanks

colossal-vr-30482

06/30/2022, 5:38 PM

Hi! I'm trying to add a cluster of harvester nodes in Rancher but I cannot do it. I didnt finde the "`cluster-registration-url"`

colossal-vr-30482

06/30/2022, 5:45 PM

And it is possible to change dns server on harvester nodes?

quick-waitress-21632

07/04/2022, 12:58 PM

Hi all, Is it possible create nodes groups (

dev

prod

, etc..) with different kind of hardware and via rancher spin up clusters in

dev

prod

? Can i do it with node taints? and how to configure the toleration in rancher?

bumpy-portugal-40754

07/06/2022, 2:06 PM

I'm trying to debug a problem with downloading a support bundle in the browser... since it's not working from the browser, I checked the pods and found that there are still pod left with the generated bundles in the /tmp directory. I tried to copy them out of the pod with `kubectl cp``... but this didn't work because there is no

tar

binary on the havester nodes. So please, can you add

tar

to the next harvester release? Yes, I know there is a workaround with

kubectl exec ... -- cat file >localfile

but this is simply unnecessary complicated. Is it possible to generate a support bundle from cli? How? And finally: Is there a way to upload the support bundle from command line also?

damp-vegetable-48645

07/07/2022, 4:43 PM

I have been unable to find documentation in regards to removing/replacing physical hosts within a harvester cluster. I built out a test cluster with undersized drives and didn't realize I could just add a drive to expand to, so took a node out and rebuilt it on the new drive. Even after numerous rebuilds it won't join the cluster (the node has been completely removed from the cluster), and putting the original drive back in isn't yielding any better results. As I can see node failures/replacements being a normal part of a lifecycle, the ability to roll nodes in/out of a running cluster would be imperative and something I'd like to test to get a LOE feel on it.

lemon-bear-35731

07/08/2022, 1:40 PM

Hello all; I have a Windows VM crashing and I do not understand how/where to find the logs that could help indicate why. In the harvester UI, I can see the crash event; where can I find the kubevirt logs that correspond? The log window accessible via the UI on the vm that looks like it should show the pod logs is always blank. Can I fetch them via kubectl directly or find them on disk more easily?

sticky-summer-13450

07/11/2022, 1:29 PM

All of the metrics in Harvester's dashboard say there is no data. I don't know when it started happening, but I went to look just now and it's all blank, and no data.

stale-painting-80203

07/11/2022, 6:45 PM

I installed v0.13.3 MetalLB as outlined in https://github.com/SUSE/suse-at-home/blob/main/install/metalLB/README.md. Pods and service in metallb-system are all running, but the whoami app is unable to get an external address (remains pending). I am using Calico CNI and seems MetalLB site has posted an issue with Calico https://metallb.universe.tf/configuration/calico/ . Configuring calicoctl https://www.suse.com/support/kb/doc/?id=000020158 is complicated. Is there an easier way to solve/ fix calico either from Rancher or Harvester?

crooked-scooter-58172

07/12/2022, 3:30 PM

Team: We need to build a monitoring solution around Harvester Nodes where a trigger needs to be created whenever a Node down or restart. Does Harvester offer any out of the box API which I can use for the same?

polite-megabyte-63476

07/15/2022, 2:36 PM

Jul 15 14:34:53  rancher-system-agent[3390]: W0715 14:34:53.242176    3390 reflector.go:437] pkg/mod/github.com/rancher/client-go@v0.21.0-rancher.1/tools/cache/reflector.go:168: watch of *v1.Secret ended with: an error on the server ("unable to decode an event from the watch stream: stream error: stream ID 167; INTERNAL_ERROR") has prevented the request from succeeding

rough-teacher-88372

07/17/2022, 10:58 AM

Hello. I'm currently on Harvester 1.0.2 controlled by a Rancher cluster with version 2.6.3. Trying to spin up an RKE2 cluster gives me the following error:

failed to create fleet-default/cluster1 <http://cluster.x-k8s.io/v1alpha4|cluster.x-k8s.io/v1alpha4>, Kind=Cluster for rke-cluster fleet-default/cluster1: Internal error occurred: failed calling webhook "<http://default.cluster.cluster.x-k8s.io|default.cluster.cluster.x-k8s.io>": failed to call webhook: the server could not find the requested resource, failed to create fleet-default/cluster1-bootstrap1 <http://cluster.x-k8s.io/v1alpha4|cluster.x-k8s.io/v1alpha4>, Kind=MachineDeployment for rke-cluster fleet-default/cluster1: Internal error occurred: failed calling webhook "<http://default.machinedeployment.cluster.x-k8s.io|default.machinedeployment.cluster.x-k8s.io>": failed to call webhook: the server could not find the requested resource

I've been trying to trace the error but I'm having some difficulties finding the correct related objects in the two kubernetes clusters. What I suspect might be the issue is that rancher attempts to create a machinedeployment on the harvester cluster, which is in the api namespace

<http://cluster.x-k8s.io/v1beta1|cluster.x-k8s.io/v1beta1>

while rancher expects it to be in the api namespace

<http://cluster.x-k8s.io/v1alpha4|cluster.x-k8s.io/v1alpha4>

Before I go through the hoops of upgrading rancher I figured I'd check here if this is a known issue. If so, are there plans to support some amount of backwards compatibility from different rancher versions to harvester and vice versa?

faint-oxygen-94310

07/20/2022, 6:40 PM

Hello! Question here, is there a way to update a harvester host vlan config? like at the OS level in Suse?

proud-addition-21854

07/21/2022, 4:38 PM

Is it possible to change the harevest-mgt interface to make it vlan aware? harvester node has been already installed . Since the installer does not support vlan yet , can I change it after its installed?

freezing-farmer-7156

07/21/2022, 11:27 PM

hi harvester team, why are all feature I touch in harevester not working!? Do I really need to create a bug or issue report on every feature I touch? This is really frustrating and I'm really ask my self the question why this is/was not seen!? Is there not any QC done before putting a release out there?

bumpy-portugal-40754

07/28/2022, 4:03 PM

Harvester team, can you please add a longer job timeout for your Harvester update containers? I have a virtual box instance here for testing new version and I'm constantly running into timeouts... because the upgrade vm startup times out, because the image preloding pod times out, because other things time out... just because they are slow. All runs fine after a few restarts, but this is sort of unnecessary and annoying. So please can you adjust the job timeouts to something bigger than now? How about 30 mins? Or make it configurable for admins.

👍 1

bumpy-portugal-40754

07/28/2022, 4:04 PM

Answering my own question: smartctl will be in >=1.0.3

boundless-student-34829

07/28/2022, 8:23 PM

I'm not finding what I seek, so asking here. What is the best method of a node graceful shutdown? 1node testing setup(just got 1st install setup. Have hit 3 different guis and not seeing an option on any) and just need the proper method to get everything shutdown in safe order. Is systemctl poweroff a sufficient method? I see talks of features requesting this, but not an actual method. Also since on here, is there web gui method to get a shell?

swift-market-16829

07/30/2022, 8:14 PM

Hi everyone. I have installed a Digital Ocean single-node RKE2 cluster via Rancher and enabled Multus/Calico with the intention of using this cluster to support a development installation of Harvester using nested virtualization per: https://docs.harvesterhci.io/v1.0/dev/dev-mode/ . The host-check script passes. The instructions say that I have to have “a corresponding

NetworkAttachmentDefinition

CRD”. Are there instructions as to exactly that this CR needs to look like?

stale-painting-80203

07/29/2022, 10:02 PM

I am trying to move and existing Harvester server connected to an old instance of a Rancher to a new instance of a Rancher. The new rancher is behind a Layer-4 Load Balancer. When I plug in the new

cluster-registration-url

, the rancher shows Harvester cluster in Pending state and never connects. I have verified that I can ping the LB IP from the harvester server. The LB is forwarding requests to the rancher on ports 80, 443, 9345 (rancher_supervisor), 6443 (kube_api). These are all working correctly through the LB. Are there any other port listeners required for Harvester to communicate with rancher?

hundreds-evening-84071

08/05/2022, 5:10 PM

hey guys, thinking of trying out harvester for first time... but GPU passthru to VM is important for me. Is this possible? Has anyone tried this?

faint-oxygen-94310

08/08/2022, 5:03 PM

Hello, have a question about DNS on my harvester cluster. I’m trying to add my harvester cluster to my rancher deployment. I’ve set the cluster-registration-url in harvester to the url created from my rancher cluster. In rancher the cluster is stuck in a pending state. Looking at the harvester logs i’m seeing the following error message:

2022-08-08T16:18:03.274063754Z time="2022-08-08T16:18:03Z" level=error msg="error syncing 'cluster-registration-url': handler harvester-setting-controller: Get \"<https://kube.blah.dev/v3/import/zvk4bvccqsvrjh5fpck6vm9tzgrbfjvgtgrh4wtq7r9mmtnk7s9l4j_c-m-mzc7k67k.yaml>\": dial tcp: lookup <http://kube.blah.dev|kube.blah.dev> on 10.53.0.10:53: no such host, requeuing"

its seems like harvester is trying to use 10.53.0.10 for DNS lookup? Thats not the DNS server we have configured on the server. any idea why harvester is trying to use that IP for DNS lookup instead of the server/address configured in the /oem/99_custom.yaml file?

big-judge-33880

08/10/2022, 4:05 PM

Hi - I’m wondering if anyone can enlighten me regarding the coupling between e.g. persistent volumes in rke2 clusters running on top of VMs provisioned in harvester by rancher. We’ve gotten ourselves into the misfortunate situation of having all new nodes in our cluster and seeing the backing longhorn volumes disappear in the process. We still have backups from s3, but we’re unable to get the rke2 cluster to use volumes restored from backups (with pv/pvcs created via the action menu in longhorn).

witty-jelly-95845

08/10/2022, 6:00 PM

https://github.com/harvester/harvester/releases/tag/v1.0.3

🎉 4

👍 1

nutritious-father-35795

08/10/2022, 6:33 PM

harvesterHCI 1.0.3 is available, but it dont see a upgrade button in the right corner. is the upgrade not yet available via the dashboard?

adventurous-engine-93026

08/10/2022, 10:29 PM

I have an on-premise 3 node NUC cluster running Harvester. What's the best way to deploy Rancher onto that?

Title

adventurous-engine-93026

08/10/2022, 10:29 PM

I have an on-premise 3 node NUC cluster running Harvester. What's the best way to deploy Rancher onto that?

square-orange-60123

08/10/2022, 10:57 PM

recommended setup is to have separate nodes running the rancher management cluster. However, you can deploy VMs using harvester, and connect your virtualization cluster (harvester management cluster) to the rancher server running on harvester VMs. I think that is only for R&D (non-production) environments.

adventurous-engine-93026

08/11/2022, 12:41 AM

This is just a home lab setup, so the R&D setup will work. I've tried setting up 3 VMs and installing K3s and I had a crazy challenging time getting networking to work properly. I was really hoping to use the Kubernetes cluster hosting harvester to also run Rancher. That just seems to make so much more sense.

square-orange-60123

08/11/2022, 12:42 AM

gotcha, in that case you can just spin up a VM and use the docker install of rancher to get started. We do not support running rancher and harvester on the same cluster, unless virtualized as I have described above (either on a VM k8s cluster, or docker install of rancher). if that makes sense

👍 1

witty-jelly-95845

08/11/2022, 8:08 AM

I wrote a Community article on using Rancher with Docker in Harvester VM - see https://community.suse.com/posts/how-to-manage-harvester-030-with-rancher-261-running-in-a-vm-within-harvester I need to update for latest versions but hopefully it's still useful - the Rancher/Harvester integration is now easier.

🙌 1

adventurous-engine-93026

08/22/2022, 10:33 PM

I guess I should have mentioned that I was hoping to have a fault tolerant / clustered setup, not just a single node. It is for my lab, so I'm limited on HW, but I'd still like redundancy.

View count: 4