01/02/2023, 12:25 PM
I've been playing with Harvester a bit on my home lab this weekend, and I have some questions based on that experience that I was hoping someone could help with: • Since I'm in a home lab the servers aren't SUPER beefy, but it seems like harvester has 50% of my 32 cores reserved without even running anything else. Is that expected? That seems really high. Is it not really worth running Harvester on lower-specced machines? I also have 14 of my 93GiB reserved (15%), but that at least leaves a lot more headroom. • I'm planning to run Rancher to manage the Harvester cluster, as well as my DO Kubernetes cluster, and it seems like there's at least some version of Rancher running "behind the scenes" of Harvester. Do I need to also create another set of VMs on the Harvester cluster to install Rancher on those as well? Is that a common pattern (managing the Harvester cluster Rancher is running on via the same Rancher)? It seems wasteful to run Rancher in order to run another Rancher. • Longhorn is running on the Harvester cluster, and I was planning to run Longhorn already myself. Is it typical for people to expand Harvester's Longhorn with more storage and use that same Longhorn cluster as a target from a Kubernetes cluster that's running in Harvester VMs? Should I instead be running another Longhorn cluster inside Harvester VMs to be the "Guest Longhorn"? If I were to set up a separate Longhorn cluster (say, bare metal or something) would I be able to use that for Harvester? • Is there any way for me to easily verify Harvester is using the correct NICs for its storage network? I was looking at the IPs to see if it was using the CIDR I provided, but it looks like it's using overlay network IPs? • I'm a big fan of config management and versioned config files for everything (I tend to forget how I set something up so having configs to redo it is great), can I use that with Harvester (and I guess by extension Rancher)? Can I write YAML manifests to define all the VMs/volumes/images/networks/etc and apply those to the cluster to be able to bring up any other Harvester cluster to the same state as another (assuming the same hardware)? I had to dig into the internals a bit in order to troubleshoot some of my Longhorn/Harvester issues when one of my nodes flaked out and it was nice to see Kubernetes "under the hood", but can I also use that and apply CRDs directly to the management cluster (possibly via CI/CD)? Thanks for any help/insight you can give! 😄


01/03/2023, 1:11 AM
same thoughts and problems I have been running into. I understand SUSE focusing on harvester JUST being about HCI even though they use rancher under the covers along with longhorn and the requirement to install Rancher outside of the harvester cluster so the rancher manager itself will not be affected by any disk or network pressure issues (which I have run into) but it does seem like a mess of slimy noodles, doesn't it? I recognize what they are trying to do and it's admirable but it's. 'just not quite there yet'
I am rooting for them though... it will be nice to see an end to end CNCF k8s ecosystem by SUSE to rival RedHat and VMWARE.


01/03/2023, 2:54 AM
im going backwards to Proxmox VE. It was stable and worked well. Since using harvester for 6-8 months, Ive had nothing but issues with crashing VM’s, VM’s constantly pausing, Upgrade not working 5 times in a row, then saying my cluster is up to date despite the hosts all still being on the previous version. Longhorn is an absolute DOG of a storage platform (even with full SSD/NVME disks). Unless you run 10G networking (or much more), dont even bother with Harvester, its too immature. I too hope this platform takes off, as I really want to use it at work, but for home-users its a nightmare. Definitely shouldnt be in stable-production-release.


01/03/2023, 3:12 AM
i found out about harvester only a few hours after I renewed my vmug advanced for this year and I was hoping it would be something I could use at home for free that works like the kubernetes stuff I already use and like
i do have a 10G storage network set up but booting VMs has been pretty slow. i just got it working today after messing around with VLANs and stuff. not harvester or longhorn's fault other than making it hard to figure out what was going wrong


01/03/2023, 8:28 AM
It is usual to run a Rancher VM on Harvester to manage the Harvester cluster, and other clusters, although that is not something I have done.
Regarding config management, Terraform seems to be the way to go - but I only started doing that over this last holiday period.
Regarding Longhorn storage - using the Harvester Longhorn storage yourself is very much discouraged, you should think of Harvester as a black-box product and ignore the fact that it is made from familiar components. But, it seems wasteful to run Longhorn with 3x redundancy on Longhorn with 3x redundancy. Does that mean you're only able a ninth of the total storage? For the time being I've been using an external NAS for persistent storage, but I'd like to migrate to Longhorn running in the Harvester cluster if I can find a recommended low-waste way of doing that.


01/03/2023, 5:02 PM
@flat-evening-58664 same boat I was/am in. I'm a VCP with VMUG as well and have been using an little Intel NUC for years as a nested VMware lab and thought I'd give this harvester thing a shot. Super interested and the bits combined make sense just not mature enough yet in both industry adoption (plugins, ecosystem, etc.) as well as being polished. It's supposed to be a black box but unfortunately I have had to resort to pulling back the covers a bit too often to find things that /should/ be part of the UI. For example, as we speak I am troubleshooting a VM that is stuck stopping in the UI and have had to resort to a kubectl describe to find out what is going on with it. Just not primetime IMHO


01/07/2023, 8:52 PM
@sticky-summer-13450 Is it really usual to run the controlling Rancher on the same Harvester it is managing? I doubt it... at least for production. And I also didn't find a recommendation for that. My advice: run Rancher outside Harvester... on a dedicated rke2 cluster, not a single VM. My 2 cents.