https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
h
By default, the pod network which is the underlying network for the VMs, handles 110, that was increased, I believe to 200? I'm not sure anything over that has been tested. Interested, what's the use case to host 250+ VMs in a single host?
😎 1
w
i have changed networking for pods, that was around 110 yes, and also reserved memory and cpu for RKE after some Kernel panic events.
h
I think the host has 250 available IPs - not sure if there is a way to extend that.
Harvester has overprovisioning when you use ie, 400% overprovision, it will use 1/4 Core per vCPU
w
yes it was a 24 network, but also Harvester internally consumes a few of those so actual limiter is earlier, this was sorted - had to do full reinstall to make that happen
h
What did you change? The pod network?
w
yes, also had to add etcd defragging (daily)
my current challenge is "longhorn" even with a single replica it adds tons of "things" making underlaying iSCSI timeouts, (iam on local a SSD raid set). (raised timeouts to make things work atm) i think i need to drop longhorn, and then i loose some stuff like snaps, migration and backups that is useful even without replication (that i dont need)
but really the "worst" issues are the "symptoms", like kernel panic, UI lost, SSH lost, when Harvester hits those limits (some mentioned above) - it does not feel mature. - ie. how it can not by default reserve RKE cpu/mem to ensure system integrity under any VM load is a mystery to me - (iam on 1.8 currently)
what iam afraid of is that iam choosen the wrong product for the job.... - but i HOPE i have just not managed to SETUP it correctly for the job :)
@happy-cat-90847 - use case = powerful hosts, and small VM's 🙂, total # of VM's being migrated is around 2500....
t
What storage did you use with vmware? I would HIGHLY recommend a 3rd party storage CSI and hardware for that scale.
w
Its the same physical hosts Iam migrating (rolling migration) to, local SSD raid10 sets with HW raid controller
note: iam not clustering the VM's - they are bound to every node, both for storage and process. (so Longhorn is not really adding anything good for me, I have cloned the class and set replicas to 1)
my biggest problem right now seems to be Harvesters RAM handing, it simply lacks the mechanisms ESXI has to balloon/reclaim/compress/manage the VM's memory, that in my case making it using around >200% more (in real-life) RAM for same load, iam at the ceiling now with 241 boxes, where i can run at 500+ on ESXI. (exact same HW)
🐿️ 1
4 Views
Open in Slack
PreviousNext
Powered by