:wave: we're running rke2 with a bare metal cluste...
# kubernetes
h
đź‘‹ we're running rke2 with a bare metal cluster and Cilium. Has anyone found a clear/safe way to modify cilium values and not have them get wiped out by various rancher cluster changes? a super easy repro is to follow this rancher example and modify it with HelmChartConfig. https://docs.rke2.io/networking/networking_services at this point, if you go edit the cluster yaml in the rancher UI (i've tested this a few ways) but the easiest repro is to edit the cluster config, and simply add an Agent Environment Var (foo: bar) once the cluster updates, it'll also bump the rke2-cilium helm rev, flattening your changes via HelmChartConfig.
perhaps a better question is "what's the recommended way to persistently provide a rancher managed helmchart with values"
c
The rancher UI has a spot for CNI config. You need to put CNI config in there. If you go edit the deployed resources by hand, or provide your own rke2-cilium HelmChartConfig, it’ll get overridden by the rancher-managed CNI config values.
h
referring to:
c
yes
that goes into a field in the cluster resource, if you’re editing the yaml
h
a couple things: that section is headered with
Add-On Configuration can vary between Kubernetes versions. Changing the Kubernetes version may reset the values below.
which is a bit scary its also the entire config, so losing the ability to just override like so, is a bummer (also this can be more easily version controlled)
Copy code
apiVersion: <http://helm.cattle.io/v1|helm.cattle.io/v1>
kind: HelmChartConfig
metadata:
  name: rke2-cilium
  namespace: kube-system
spec:
  valuesContent: |-
    kubeProxyReplacement: true
    k8sServiceHost: <KUBE_API_SERVER_IP>
    k8sServicePort: <KUBE_API_SERVER_PORT>
    localRedirectPolicy: true
but ignoring all that - is the documentation just a bit misguided in mentioning the above?
c
its not the entire config. anything you put in there goes into a HelmChartConfig in the downstream cluster. If you don’t want to set something, then don’t. It’s just pre-populated with the default values.yaml content so you have an example to work with.
If you LEAVE the whole config in there then yeah, i’ll be the whole config.
This goes into the cluster yaml as `.spec.rkeConfig.chartValues.rke2-cilium`; if you want to manage the config you should manage it on the cluster object.
The RKE2 docs are just for RKE2. If you are deploying via rancher, the Rancher docs cover the bits of the cluster config that it manages.
h
It’s just pre-populated with the default values.yaml content so you have an example to work with.
that's super good to know and makes this way more clear, and in hindsight is obvious. Thanks a ton for helping walk through this