I have a multiple kubernetes cluster. And I want ...
# longhorn-storage
h
I have a multiple kubernetes cluster. And I want longhorn to be scheduling volumes for my containers, and running all of the longhorn components on all my nodes except for this one node, we will call it “node-no-longhorn” What I did: 1. I set a node label on all the nodes I want all my longhorn components to run on, and then I edited all the longhorn components deployments, daemon sets etc… so that longhorn components only schedule on nodes with this set node selector/label. And I did not label my “node-no-longhorn“ And I don’t see any longhorn components running on the node “node-no-longhorn” which is good 2. Also, I set a totally different label, this time only on the node “node-no-longhorn”, and for all my all container deployments, I set a required node affinity of “Not In” and the node label, so all my containers are required to not run on the node with this label…..and the node with the label is “node-no-longhorn” 3. I didn’t even install open-iscsi package onto this node “node-no-longhorn” The problem is, for some reason , this one volume of one of my containers keeps wanting to be scheduled onto “node-no-longhorn” so all my work is for naught and I suck. How would I correctly go about configuring Longhorn so it does not stun any longhorn components on this specific node, and how can I also make it so longhorn never tries scheduling volumes on this node. Also, Do I have to install open-iscsi onto this node that I don’t want longhorn using at all? Thanks guys
m
If you don't want longhorn on this node, do not install open-iscsi. You will need to set a nodeSelector using your label. That should suffice, but you can also add affinity/anti-affinity to further set scheduling for longhorn. Make sure you set the nodeSelector on all components of longhorn in the helm chart. Don't do manual edits since the next upgrade it will override your edits with what you have set in values.yaml This example is when I wanted longhorn to only run on control plane nodes with the label set to longhorn.
Copy code
longhornAdmissionWebhook:
  tolerations:
    - key: "node-role.kubernetes.io/master"
      operator: "Exists"
  nodeSelector:
    k8s.storage.local/default: "longhorn"
^ the nodeSelector part you need to add to each component including global in the helm chart.
h
Roger that
Thankyou for response
m
No worries and good luck. If you need any more help, just message this thread and I will see it.
h
I don’t want to use helm chart, I just wanted to use the longhorn.yaml file by the way
m
ohh, ok. Whatever works for you.
h
@mysterious-animal-29850 are you a longhorn dev ?
m
I wish lol. No not at all. Experienced user, used it in production for a few years.
also, using "experienced" here very loosely 🙂