https://rancher.com/ logo
Title
b

big-oxygen-89163

05/11/2023, 6:21 AM
is longhorn doing raid0 between nodes or is the maxium volume claim < biggest drive?
b

bulky-sunset-52084

05/11/2023, 7:29 PM
It's a dynamically growing volume - longhorn stores all of its blocks on each node that the volume is assigned to. It replicates to all of them. Max volume claim would probably be the size of the smallest disk because once you fill up the disk the replica would fail.
b

big-oxygen-89163

05/11/2023, 7:32 PM
got it, I guess it would work with an ext4 formatted lvm?
b

bulky-sunset-52084

05/11/2023, 7:34 PM
I guess I have to ask in what context? At the end of the day it's just a raw block volume exposed to k8s so I'd have to say sure you can format in any way you like.
b

big-oxygen-89163

05/11/2023, 7:36 PM
longhorn only supports ext4/xfs from what I know. so if I wanted to create a VM with 2TB disk and my local node SSD are only 1TB - then I would need to create a srtibe/raid0 and add that to longhorn
b

bulky-sunset-52084

05/11/2023, 7:40 PM
Longhorn alone only supports ext4/xfs but harvester uses longhorn as a storage pool for a VM. The VM image is then placed in that longhorn volume and uses the space granted by it. So yes the longhorn volume itself will be ext4/xfs your underlaying VM can be whatever filesystem you like.
b

big-oxygen-89163

05/11/2023, 7:42 PM
yes I know that :) just wanted to know if longhorn could stribe disks. But I will try lvm - if it is included in distro
b

bulky-sunset-52084

05/11/2023, 7:47 PM
Ohh so you want to pool longhorn volumes then with lvm? Can I ask why? That actually is what longhorn is doing under the hood lol so this feels recursive. So you can have hardware raid-0 on 3 nodes for example and longhorn will create a volume and mirror the data to all 3 nodes simultaneously. Longhorn will use whatever is mounted on /var/lib/longhorn on the node itself as storage for the volumes 'block files'
b

big-oxygen-89163

05/11/2023, 7:50 PM
yes but I dont have hardware raid so need to do it in software. so just to understand it, if I have 3 nodes with 2x1TB and a storage class with 3 replica, I can max create 1TB vm disk right? unless there is a raid0 with 2x1TB
💯 1
thanks!
b

bulky-sunset-52084

05/11/2023, 7:52 PM
Yes that is correct!
b

big-oxygen-89163

05/11/2023, 9:22 PM
it seems to work 🙂 great
🙌 1
ok, cluster is not healthy now, so some problems using lvm 😞
failed to update device mount 021f4297329eef8ae98994dada79b2d5: Failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net mount -t ext4 -o journal_checksum,journal_ioprio=0,barrier=1,errors=remount-ro,relatime /dev/dm-0 /var/lib/harvester/extra-disks/021f4297329eef8ae98994dada79b2d5], output , stderr, mount: /var/lib/harvester/extra-disks/021f4297329eef8ae98994dada79b2d5: /dev/mapper/longhorn_lvm_group-longhorn_lvm already mounted on /var/lib/harvester/extra-disks/021f4297329eef8ae98994dada79b2d5. , error exit status 32
a

ancient-pizza-13099

05/12/2023, 2:00 PM
seems not a good option to use raid under longhorn
b

big-oxygen-89163

05/12/2023, 2:01 PM
hardware raid raid0 should be fine, but software raid not so good
a

ancient-pizza-13099

05/12/2023, 2:02 PM
hardware raid works as a total to LH
b

big-oxygen-89163

05/12/2023, 2:03 PM
as a total?
a

ancient-pizza-13099

05/12/2023, 2:03 PM
LH is more working on distributing data with N copis, rather than connecting N disks into a big ONE.
hardware raid, in the view of LH/Harvester, is a disk
b

big-oxygen-89163

05/12/2023, 2:03 PM
yes got it