https://rancher.com/ logo
d

damp-magician-5939

11/04/2022, 1:48 PM
Has anyone faced an issue where even setting the taints, tolerations, and node selector, the longhorn workloads still go to the wrong nodes? For instance, we have 2 nodes dedicated to database related workloads, including longhorn, and they are being tainted and labelled correctly. I have made the needed changes on Longhorn-ui, Longhorn manager, and the driver deployer, but still the CSI components for example are on the wrong nodes (non tainted). Not really sure what is going on, I have been looking into this for the past 2 days. I have been trying to get a hold of someone from longhorn team but had no luck so far. I saw some issues opened about that, but all of them were resolved with the taints and tolerations setting.
f

famous-journalist-11332

11/07/2022, 1:04 AM
d

damp-magician-5939

11/07/2022, 11:20 AM
Yes I did, but still not working. I am on the latest version (1.3.2). Our deployment is using
kubectl
, So I have been following the docs, the pages that seemed most relevant are this https://longhorn.io/docs/1.3.2/advanced-resources/deploy/taint-toleration/ and this https://longhorn.io/docs/1.3.2/advanced-resources/deploy/customizing-default-settings/#using-the-longhorn-deployment-yaml-file. I have setup on the ConfigMap the Node selector for system components, but still not able to get it working. I was initially thinking it was a memory/cpu allocation issue, but it is not.
Just confirmed that the same happens if I try to install using
helm
. I am really at a loss here. To clarify, the components I am having trouble with are the CSI related ones. For some reason they are not respecting my nodeSelector and taint settings.
f

famous-shampoo-18483

11/08/2022, 1:14 PM
You mean the related components (like attacher/provisioner/resizer rather than the plugin) do not apply the tolerations you set? To add the tolerations for this kind of System Managed Components (components that are dynamically deployed by longhorn manager after the installation), I would recommend to modify the setting mentioned in the doc rather than directly editing the ConfigMap
longhorn-default-setting
d

damp-magician-5939

11/08/2022, 2:06 PM
You are correct about the components giving me trouble. By the setting mentioned, do you mean the System Managed Components Node Selector setting on the Longhorn UI?
About that, are the admission and conversion webhooks subject to this configuration? or they do not matter in this context?
113 Views