Hi - RKE2 falls back to Canal/10.42 Pod CIDR after reboot Is there a way to restrict Rancher from NOT picking the default CNI & CIDR? I use CNI - calico with a Pod CIDR 192.x.x.x, and this configuration is defined in

/etc/rancher/rke2/config.yaml.d/50-rancher.yaml

. Have encountered this issue twice - once in NPE/Prod during OS maintenance that required VM reboots. However, I haven't been able to reproduce it consistently. In both cases, after stopping

rancher-system-agent.service

and

rke2-server.service

followed by rebooting etcd leader VM, the node came up with default networking configuration (Canal + Pod CIDR 10.42.0.0/16) instead of the configured Calico settings, leaving the cluster unusable. During startup, rancher-system-agent fails with: unable to parse config file: stat /etc/rancher/agent/config.yaml: no such file or directory Even though /etc/rancher/agent/config.yaml exists on the node, for some reasons it is unable to discover. RKE2 then starts with: Running kube-proxy --cluster-cidr=10.42.0.0/16 Log Snippet Jul 13 023800 carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 systemd[1]: Stopping Rancher System Agent... Jul 13 023800 carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 systemd[1]: rke2-server.service: Consumed 2d 21h 20min 44.672s CPU time, 2.5G memory peak. -- Boot 561668e1662345deba257389ac852d58 -- Jul 13 023949 carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 systemd[1]: Started Rancher System Agent. Jul 13 023950 carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 rancher-system-agent[1842]: time="2026-07-13T023950-04:00" level=fatal msg="Fatal error running: unable to parse config file: error gathering file information for file /etc/rancher/agent/config.yaml: stat /etc/rancher/agent/config.yaml: no such file or directory" Jul 13 023954 carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 rke2[1902]: time="2026-07-13T023954-04:00" *level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --*conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=carbon-sad CONFIG.YAML - DOES ADDING CANAL UNDER DISABLE SECTIOIN WILL SOLVE THE ISSUE?

[root@carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 ~]# cd /etc/rancher/rke2/config.yaml.d/
[root@carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 config.yaml.d]# cat 50-rancher.yaml
{
  "agent-token": "7z65h4lpl7f5khjgx55cc6cf5f8mmnxf",
  "cluster-cidr": "192.168.0.0/17",
  "cni": "calico",
  "disable": [
    "rke2-ingress-nginx",
    "rke2-snapshot-controller",
    "rke2-snapshot-controller-crd",
    "rke2-snapshot-validation-webhook"
  ],
  "etcd-arg": [
    "quota-backend-bytes=5368709120",
    "election-timeout=5000",
    "heartbeat-interval=1000"
  ],
  "etcd-expose-metrics": true,
  "etcd-s3": true,
  "etcd-s3-access-key": "GPuoJRFwsu",
  "etcd-s3-bucket": "etcd-backup-dev",
  "etcd-s3-endpoint": "bkpcohprdsasysctrl.com:3000",
  "etcd-s3-endpoint-ca": "/var/lib/rancher/rke2/etc/config-files/s3-endpoint-ca-19131.crt",
  "etcd-s3-folder": "rke2-sadc-dev-devtools-v1",
  "etcd-s3-region": "dev",
  "etcd-s3-secret-key": "iI0wJ_UwoJQzv6XEKPt",
  "etcd-snapshot-retention": 14,
  "etcd-snapshot-schedule-cron": "0 */12 * * *",
  "kube-apiserver-arg": [
    "service-account-lookup=true",
    "audit-log-path=/var/log/kubernetes/audit.log",
    "audit-log-maxage=30",
    "audit-log-maxbackup=10",
    "audit-log-maxsize=100",
    "enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction,Priority,TaintNodesByCondition,PersistentVolumeClaimResize,EventRateLimit,DenyServiceExternalIPs",
    "admission-control-config-file=/etc/rancher/rke2/admission/admission-config.yaml",
    "audit-policy-file=/etc/rancher/rke2/audit/audit-policy.yaml",
    "authentication-token-webhook-config-file=/var/lib/rancher/rke2/kube-api-authn-webhook.yaml"
  ],
  "kube-apiserver-extra-mount": [
    "/etc/rancher/rke2/admission:/etc/rancher/rke2/admission:ro",
    "/etc/rancher/rke2/audit:/etc/rancher/rke2/audit:ro"
  ],
  "kube-controller-manager-arg": [
    "node-cidr-mask-size-ipv4=26",
    "bind-address=0.0.0.0",
    "cert-dir=/var/lib/rancher/rke2/server/tls/kube-controller-manager",
    "secure-port=10257"
  ],
  "kube-controller-manager-extra-mount": [
    "/var/lib/rancher/rke2/server/tls/kube-controller-manager:/var/lib/rancher/rke2/server/tls/kube-controller-manager"
  ],
  "kube-scheduler-arg": [
    "bind-address=0.0.0.0",
    "cert-dir=/var/lib/rancher/rke2/server/tls/kube-scheduler",
    "secure-port=10259"
  ],
  "kube-scheduler-extra-mount": [
    "/var/lib/rancher/rke2/server/tls/kube-scheduler:/var/lib/rancher/rke2/server/tls/kube-scheduler"
  ],
  "kubelet-arg": [
    "tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305"
  ],
  "node-label": [
    "env=dev",
    "platform_name=devtools",
    "rke.cattle.io/machine=5ef1dbcc-3867-4f13-a2e6-ae51fbf33da2",
    "zone=w1c3"
  ],
  "node-taint": [
    "node-role.kubernetes.io/control-plane:NoSchedule",
    "node-role.kubernetes.io/etcd:NoExecute"
  ],
  "private-registry": "/etc/rancher/rke2/registries.yaml",
  "server": "<https://172.12.86.236:9345>",
  "service-cidr": "192.168.128.0/17",
  "system-default-registry": "e-car-onprem-docker-virtual.docker.com",
  "token": "jgzwp9lspkmtnlt9kznc4rlq5dgh52hh9m4gx"

Why is it failing to find the config files?

yes.. /etc/rancher is created as separate partition. carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 ~]# df -h | grep rancher /dev/mapper/appvg-etcrancherlv 2.9G 92K 2.8G 1% /etc/rancher /dev/mapper/appvg-rancherlv 196G 17G 170G 9% /var/lib/rancher

sounds like you don’t have your mount priorities set up properly in systemd, and services are starting without that partition mounted

True, /etc/rancher is very light weighted. Considering my past edge cases with OS/App filesystem combinations created filesystem crashes in RKE1, hence separated OS filesystems and rancher. Is this causing the issue, should i move away from separated file system for /etc/rancher as well? [carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 ~]# df -h | grep rancher /dev/mapper/appvg-etcrancherlv 2.9G 92K 2.8G 1% /etc/rancher /dev/mapper/appvg-rancherlv 196G 17G 170G 9% /var/lib/rancher [root@carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 ~]# cd /etc/rancher/ [carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 rancher]# ls -ltr total 32 drwx------. 2 root root 16384 Apr 21 2025 lost+found drwxr-xr-x. 2 root root 4096 Jan 15 2026 node drwxr-xr-x. 5 root root 4096 Jan 15 2026 rke2 drwx------. 2 root root 4096 Apr 15 16:30 agent [carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 rancher]# du -shx * 16K agent 16K lost+found 8.0K node 52K rke2 [root@carbon-sadc-dev-devtools-v1-np2-w1c3-m-vdw9g-qg5d2 rancher]# cat /etc/fstab /dev/mapper/systemvg-rootlv / ext4 defaults 1 1 UUID=14a21bec-4711-449b-a10d-ca8396380169 /boot ext4 defaults 1 2 /dev/mapper/systemvg-homelv /home ext4 defaults,nodev,nosuid, 0 0 /dev/mapper/systemvg-optlv /opt ext4 defaults 1 2 /dev/mapper/systemvg-tmplv /tmp ext4 defaults,nodev,noexec,nosuid 0 0 /dev/mapper/systemvg-usrlv /usr ext4 defaults 1 2 /dev/mapper/systemvg-usrlocallv /usr/local ext4 defaults 1 2 /dev/mapper/systemvg-varlv /var ext4 defaults,nodev,nosuid, 0 0 /dev/mapper/systemvg-cachednflv /var/cache/dnf ext4 defaults 1 2 /dev/mapper/systemvg-varloglv /var/log ext4 defaults,nodev,noexec,nosuid 0 0 /dev/mapper/systemvg-auditlv /var/log/audit ext4 defaults,noexec,nodev,nosuid 0 0 /dev/toolsvg/splunklv /usr/splunk ext4 defaults 0 0 /dev/toolsvg/taniumlv /opt/Tanium ext4 defaults 0 0 /dev/toolsvg/rapid7lv /opt/rapid7 ext4 defaults,nodev 0 0 /dev/toolsvg/crowdstrikelv /opt/CrowdStrike ext4 defaults,nodev 0 0 tmpfs /dev/shm tmpfs defaults,nodev,noexec,nosuid 0 0 /dev/appvg/containerdlv /var/lib/containerd ext4 defaults 0 0 /dev/appvg/kubeletlv /var/lib/kubelet ext4 defaults 0 0 /dev/mapper/appvg-logkuberneteslv /var/log/kubernetes ext4 defaults 0 0 /dev/mapper/appvg-optrke2lv /opt/rke2 ext4 defaults 0 0 /dev/mapper/appvg-etckuberneteslv /etc/kubernetes ext4 defaults 0 0 /dev/mapper/appvg-etcrancherlv /etc/rancher ext4 defaults 0 0 /dev/toolsvg/fluentlv /opt/fluent ext4 defaults,nodev 0 0 /dev/mapper/appvg-rancherlv /var/lib/rancher ext4 nodev,defaults 1 2

If you are not capable of ensuring it gets mounted reliably, don't do it

thanks for suggestions