Has anyone come up with a good solution to use large scale fibrechannel attached SAN storage on Harvester, in the same way vmware VMFS would?
The context is converting/migrating a 15node/1000VM vmware cluster to Harvester, and the majority of workloads are traditional monolithic VMs.
So far I've explored;
• Presenting FC luns to Longhorn, but this means I must present unique LUNs to each node, and either;
a. Replicate all data - which costs VM IO performance, doubles the IO to the SAN, and relies on SAN deduplication.
b. Have a single replica - which solves the performance, but drastically limits VM portability across nodes to optimize CPU/Memory consumption, and means I have a significant outage if I lose a node (or even need to perform maintenance on it).
• SAN CSI driver - Which solves both of the above limitations by provisioning and exporting a dedicated LUN per volume, but does not scale to 1000 VMs (and 1500+ volumes). Each node's FC card limits would quickly be exceeded, and I'm warned that a large concurrent activity like a node failure would likely smash the SAN's API to remount all of the LUNs to new nodes.
Keen to hear any alternatives people have come up with!