https://rancher.com/ logo
Join Slack
Powered by
hello, for `/var/lib/rancher/rke2/server/manifests...
# rke2
n
hello, for 
/var/lib/rancher/rke2/server/manifests
do I need to replicate custom manifest (and e.g., calico config yaml) among all control plane nodes (where rke2-server is started). Or is it enough to create the manifest at one of the servers?
c
just one is fine. If you do copy it to all the servers, you’re responsible for keeping it in sync.
n
so, does rke2 handle correctly, that e.g., calico config yaml is missing on some nodes, right?
thank you
c
manifests are deployed to the cluster, so they’re not “missing”. They just don’t get synced from nodes where they don’t exist.
remember that this is just an easy way to 
kubectl apply -f
a file
anything you put in there is visible to any node in the cluster
n
I understand, but for e.g., calico, there are two manifests, one calico chart itself maintained by rke (rke2-calico.yml), and 'values' (rke2-calico-config.yml). those values are maintained by me. So I was worried, if rke2 does not redeploy calico with its defaults if the 
rke2-calico-config.yml
is missing at some nodes while 
rke2-calico.yml
is always present.
c
the node does not read that file directly to get calico config. Like I said, it gets applied to the cluster, and then the Helm controller uses those values to configure the Calico chart, which in turn configures Calico on each individual node.
As long as you have it on the first node in the cluster to set the config when the cluster comes up, that is all you need.
n
ok, understand, so just in the case of cluster shutdown, I need to ensure, that the node with the calico-config is started first, and rest does not matter much. right?
c
no
if you shut the cluster down and bring it back up, it doesn’t lose everything that has been created in the datastore…
if you 
kubectl apply
something, and then shut down and start up the cluster again, does it get lost?
no, once you create the resource it stays created, until you delete it. Even if you restart everything.
this is how Kubernetes works. You are just creating Kubernetes resources from a file.
n
I thought that rke2 creates helm chart from the 
rke2-calico.yml
and uses 
rke2-calico-config.yml
as the helm chart values, if this file is present. So I thought, that what is happning is that rke2 triggers 
helm upgrade calico -n calico-system
and adds 
-f values.yaml
from the 
rke2-calico-config.yml
if that file is present and without the 
values.yaml
if is not present which would clearly revert calico to the default values. but if both files (those .yml) are actually kubernetes objects only and some controller trigger helm upgrade job on their change, then I fully understand now.
c
if both files (those .yml) are actually kubernetes objects only and some controller trigger helm upgrade job on their change, then I fully understand
you got it
this is covered in the docs: https://docs.rke2.io/helm#automatically-deploying-manifests-and-helm-charts
Any Kubernetes manifests found in 
/var/lib/rancher/rke2/server/manifests
will automatically be deployed to RKE2 in a manner similar to 
kubectl apply
, both on startup and when the file is changed on disk. Deleting files out of this directory will not delete the corresponding resources from the cluster.
n
is there any limit on filename lenght and/or structure? I had 
server/manifests/calico/calico-hostendpoint-kub-a10.priv.cerit-sc.cz-team0.715.yaml
files (and similar 90 more) but k8s objects was created and removed in quick loop. I removed them and created again and now it seems to be ok.
c
read the rest of that page. that is covered in the section I linked.
n
ok, thanks
ah, I see, it basically must not be distinguished by dots in the file name.. (like file.1.yaml and file.2.yaml, it must be file-1.yaml and file-2.yaml)
thank you for help 🙌
2 Views
Open in Slack
PreviousNext
Powered by