I've been thinking about this all night. I was thinking to use the longhorn storage for inside the clusters I create, but really that storage is just for spinning up vms, and clusters, etc ... and doesn't really need to support RWX, though I see that's coming later for the purpose of migrating a vm. Really I'd need another solution for the storage used by the clusters. Not really sure what that'd be just yet, as I had planned to use the local disks for that. Might just be an issue of being a homelab, a bigger setup would of course have remote storage.

oh, i spoke before reading, its right there ... install ceph

This would work perfectly, just ends up using more cpu cause there are two storage solutions running, but that's not the end of the world. Thanks again.