https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
b
In general, I normally recommend avoiding pinging specific people in public channels because it pins the whole thing on them. There's probably lot of people that could answer this question, but you're excluding them all when it gets phrased this way. Alejandro is awesome and ends up answering a bunch of these questions, but it's a community effort here. 🙂
That being said, it's k8s underneath so any valid CSI driver lets you attach data volumes
We do it with ceph all the time.
Here's a list of all the upstream ones: https://kubernetes-csi.github.io/docs/drivers.html
t
Ok. Are there docs or guides for adding the CSI that survives a reboot?
b
So how this would work is that you'd install the csi driver into the cluster. (typically this is a helm install)
Then you configure it and add a storage class that uses that driver.
Once you do that, it shows up in the list of available storage classes for VMs.
From that point the PVC gets made and is available to the "vm"
but the PV is in kubernetes and however the driver/storage class determines as far as persistence/retain should be.
Was that helpful Andy? Or more like clear as mud?
a
Two other, documented methods (one is NetApp specific): https://harvesterhci.io/kb/install_iscsi_firmware_install_boot https://harvesterhci.io/kb/install_netapp_trident_csi I have used the NetApp option for VM volumes with success (lab environment).
t
So the question initially was around pinning Longhorn volumes to an iscsi target and still boot from local disk. Replacing Longhorn with another CSI is another approach. no I don’t want to use truenas. I am looking at using harvester with some enterprise grade flash array devices.
b
I believe that the boot volume must be a Longhorn/SuseStorage(or whatever the new name is) volume.
t
ah i forgot that the vm boot volume needs to be local. Hopefully some day there is a chance to have all the VM storage on iscsi.
b
Any other data volumes (not boot) can be any other provisioner (including nfs/smb/etc)
Just as a breadcrumb Longhorn storage != local storage.
Local is an actual storage class type, that longhorn isn't.
t
longhorn --> replica --> local volume… so it is local. indrectly.
b
it's "local" in the sense of to the box, or hyperconverged, but there's another layer there and it's not local as in the actual storage class that kubernetes calls 
local
t
local as in NVME. longhorn is on top. The vm uses longhorn. I was checking to see if the host OS, sle micro, can boot vms that are iscsi attached instead of longhorn. Or a generic iscsi csi driver.
b
What's probably not "supported" but absolutely possible...
You could mount the iscsi drive into the OS and let Longhorn use that instead of an NVME drive
t
so that is part of the question. how to setup the iscsi mount to 
/var/lib/longhorn
that elemental will make permanent.
b
You'd need to edit the 
/oem/90_custom.yaml
to get it to persist.
You might need to add some 
modprobe
commands. under the 
rootfs
section.
Just remember that you'll need to do it on each node.
t
ah makes sense
a
As mentioned above, right now, VM images and boot volumes need to remain on longhorn. There's work in progress on allowing external storage via CSI for VM images and boot volumes in a future release (the HEP is at https://github.com/harvester/harvester/pull/7487/files)
t
NICE
a
getting iscsi in underneath longhorn somehow may work in the meantime, but AFAIK nobody's done any QA on that
t
understandable.
a
also, if you are doing stuff with external storage via iSCSI you might run into trouble with multipath being turned off -- see Gaurav's last two comments in this issue for a workaround https://github.com/harvester/harvester/issues/5150#issuecomment-2517792457
(don't be mislead by the title of that issue where it says "support for iscsi boot disks" -- in this case it's referring to diskless systems where the harvester host disk is on iscsi)
t
thanks
h
I’ve had folks use the nvme cli to load disks on boot (added to the cloud config) and consume them for Lo Longhorn. Good thing is that it doesn’t need multipath L. That was against a Pure FA
Sorry - I missed this post.
t
no worries. I am at Pure and I am getting an FA for my homelab. You don’t know if they have a write up for that ?
🙌 1
8 Views
Open in Slack
PreviousNext
Powered by