but you can't migrate anything to the new class, r...
# harvester
r
but you can't migrate anything to the new class, right? i'm not seeing a way to migrate it, let's say you had a functional vm, you want it's storage to be on 5 instead of 3 now since you've added another 2 nodes
b
You can mount the PVCs in a pod and transfer data over with dd or rsync.
t
BUT WHY 5?
b
Do you mean 5 replicas?
r
yea, five replicas
why not five?
t
you are going to get worse performance for ZERO extra redundancy.
b
replicas is just for disaster purposes
if you have best effort set, eventually it'll make sure there's a copy on the local node
no need to make sure there's 5 copies unless you think your cluster and hardware will literally catch fire in the racks or something terrible will happen
If it's a new storage class (you have rust vs nvme or something) then yeah you have to do a real migration.
r
yea i wasn't necessarily concerned about the performance loss but it is for disaster purposes
best effort while replica is only set to 3?
b
That's a pretty sane standard
t
ya. remember that you will have 3 COMPLETE copies of the vm
b
That doesn't include external backup copies if you set that too
r
right
3 complete copies is fine, there may be a case where i want 5/5
b
Sure, but that's extra wear on your disks
r
i agree
b
Just make sure you have the iops and bandwidth for it.
t
IF your data is that valuable, then maybe get it outside the VMs onto an enterprise sotrage.
b
You can also set those on a per Disk basis
The Storage class is just the default setting
Just realize, Let's say you're running a DB. That VM has all the DB traffic the node has to deal with. All those reads come off the attached node, but all the DB writes get duplicated to all the other nodes that have replicas of that disk too. Which takes up bandwidth and disk iops away from other VMs it's trying to do as well.
None of that really makes it as HA as having a DB cluster with just replicas instead of one monolith with 5.
r
yea i don't think i'd put a high iops system with so many replicas unless if i could back it
t
maybe you should test with 3?
b
3 replicas with more frequent backups/snapshots to external storage (s3, nfs, etc) would be better imho.
r
i'm just really doing this for some general knowledge, i don't think this would be close to a production ready cluster, i am running it with 3/5 right now, just testing.
a
FYI, the
StorageClass
is the place to set a customized replica count https://docs.harvesterhci.io/v1.4/advanced/storageclass#parameters-tab
r
yea i know