02/16/2023, 1:36 AM
Hi team, We have requirement, where we have provision k3s cluster with x2 master nodes and x3 worker nodes for storage cluster with longhorn and we need this storage cluster to be used / provision in another kubernetes cluster ( workload cluster) PV / PVC claims. The idea is to provide PV/ PVC claims for software stack such as harbor , ELK ( elastic search), custom application workload that need storage for batch processing; Does it make sense to; • Option 1: Connect k3s - storage cluster to workload cluster (2nd k8s cluster) via means of NFS service from ingress ( or any other means) for production work ( i would like to keep longhorn capabilities on top and expose the storage through nfs-server. I saw somewhere that longhorn itself has nfs-server pod as well with longhorn stack) • Option 2: Create workload pods in same k3s - storage cluster having seperate nodes ( by grouping / labeling nodes) to run application workload • Option 3: Any other options?? Any insights/ feedback in to identify the correct approach is highly appreciated.


02/16/2023, 1:39 AM
1. you can’t have a two-server k3s cluster, at least not with etcd. You need an odd number of servers for quorum 2. longhorn does not currently support multi-cluster deployment with storage clusters and workload clusters. It is converged storage, meaning that the storage is colocated within the cluster that uses it.


02/16/2023, 1:55 AM
Thank you for the insights @creamy-pencil-82913. For point 1, Apologies on x2 master node. We intend to increase it to x3 master nodes with etcd or to set it with external datastore with 3 worker nodes for optimal ha availability. For point 2, so it make sense to use option 2 - i.e setup workload nodes within storage cluster ( same cluster) and proceed with different grouping ( via namespace or pod labeling). Is this feature ( i.e. support multi-cluster deployment with storage cluster and workload cluster) is in your roadmap ? If not i think this will be greatly beneficial for devops team operations to offload storage within cloud-native environment which have many beneficial scenarios