https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
🙃 1
m
IIUC, this configuration is not supported because guest node VMs must support live migration during node maintenance and Harvester upgrades, etc. This is also why RWX volumes are required.
g
Harvester LVM modules does not support Live Migration, for some cases it's not a concern. I am running workloads like Apache Pulsar on the cluster, so live migration will be disabled anyway, moving brokers/bookkeepers/zookeepers between zones should not be supported. Also, there is no need to have replicated storage like Longhorn for systems that handles HA internally.
What I am going to try next is to nvme PCI devices pass through to a cluster created on VMs created from Terraform. It will be important to Rancher, not managed by Rancher but it could be a better fit.
on a guest cluster I'll install TopoLVM, maybe it will work.
anyway devies pass through will block live migration as well, but again not a concern here.
h
@gorgeous-guitar-43479 what's wrong with the LVM CSI we ship?
g
1. It bounds the Storage Class to a specific node, it cannot be used with StatefulSets where you want to have multiple volumes equally distributed across nodes using single storage class and i.e. affinity rules. 2. I didn't work for me - I couldn't attach the volume to the guest cluster (stuck in waiting state), I guess that there are some known issues with WaitForFirstConsumer?
I tried to dynamically change the storage classes using Mutating hooks, but it would be a hack as well.
h
1. Yes, right, nothing across nodes. 2. Yes. TopoLVM should work, but RWX is needed afaik. Check with them?
I can try it later on.
g
They do not support RWX, they explicitly throw an error when RWX is requested On the other hand Harvester hardcodes RWX in harvester-csi-driver What I am trying now (will start) is to pass through the nvme directly to the guest cluster and use TopoLVM there. But to do that I'll need to reimplement my cluster provisioning from Rancher-managed to Terraform-managed
I guess what could work, is the allow RWO here: github.com/harvester/harvester-csi-driver/blob/…/controller_server.go#… if the LiveMigration is not a concern.
2 Views
Open in Slack
PreviousNext
Powered by