Does anybody know what is the purpose of system-upgrade-controller in RKE2 spinned up as downstream cluster? I noticed it’s getting installed automatically. From what I can see in the docs https://docs.rke2.io/upgrade/automated_upgrade it can manage automated cluster upgrades. Do I really need it if I initiate upgrades manually, via terraform or UI?

if you’re using Rancher to provision the cluster, the SUC is used to manage version and configuration of the Rancher System Agent as installed on the nodes. The System Agent in turn manages the installation and configuration of RKE2 on the node, including Kubernetes version upgrades. This is done in order to avoid a dependency loop of having the System Agent manage its own updates.

The reason I asked is that the cluster is flooded with the pods apply-system-agent-upgrader-on-*, each of them with similar errors: ---

apply-system-agent-upgrader-on-phygatapp24-lab-with-2d753-zfhvx

+ TMPDIRBASE=/var/lib/rancher/agent/tmp

+ mkdir -p /host/var/lib/rancher/agent/tmp

mkdir: can't create directory '/host/var/lib/rancher/': No such file or directory

--- I thought it is something to do with cis-1.6 profile, but I disabled it and also have unresticted PSP. Same issue. Appreciate any suggestions on how to fix it.

Yeah that’s all managed by Rancher, I’m not sure what might be causing that. Have you looked at the pod logs to see if there are any prior errors? It looks like it’s expecting the host root filesystem to be mounted at /host, is that the case?

ubuntu 21.04

yes that would probably do it

perhaps I’ll need to create a separate fs instead of doing link like I did in RKE1.

yeah, having /var/lib/rancher on another FS should be fine. It’s probably just getting confused by the symlink.

I’ll try hardlink. The philosophy behind is to have a separate FS for k8s stuff (pod logs, kubeliet, etcd, …)

you can’t hardlink across filesystems

yes, I know, meant other way around, bring kubelet and pods logs to /var/lib/rancher which is a dedicated FS