This message was deleted.

In the RPi k3s cluster, check the Rancher cluster-agent. Are the Pods running and ready, what do their logs say, and so on. Could be a network connection issue.

Hi Tom thanks for the reply, I'm new to both k3s k8s and rancher. I can ping the VM IP from the raspberry master node. I can see with the few commands I found: emanuele@raspberrypi:~ $ kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system helm-install-traefik-6zlnd 0/1 Completed 1 15h <none> raspberrypi <none> <none> kube-system helm-install-traefik-crd-g6nz7 0/1 Completed 0 15h <none> raspberrypi <none> <none> kube-system svclb-traefik-3ee61a92-6v6dr 2/2 Running 4 (4h23m ago) 15h 10.42.0.17 raspberrypi <none> <none> kube-system svclb-traefik-3ee61a92-qf7zf 2/2 Running 2 (4h22m ago) 13h 10.42.1.3 orangepi3-lts <none> <none> kube-system local-path-provisioner-957fdf8bc-6fzdg 1/1 Running 5 (4h22m ago) 15h 10.42.0.18 raspberrypi <none> <none> kube-system coredns-77ccd57875-5g2pg 1/1 Running 3 (36m ago) 15h 10.42.0.15 raspberrypi <none> <none> cattle-system cattle-cluster-agent-6d97594f4c-bzhx5 0/1 CrashLoopBackOff 17 (3m51s ago) 64m 10.42.0.20 raspberrypi <none> <none> cattle-system cattle-cluster-agent-7fc8bfdd75-llh8d 0/1 Running 9 (6m40s ago) 35m 10.42.0.21 raspberrypi <none> <none> kube-system traefik-64f55bb67d-75gf9 0/1 Running 17 (66s ago) 15h 10.42.0.14 raspberrypi <none> <none> kube-system metrics-server-648b5df564-dbrkn 1/1 Running 11 (6m44s ago) 15h 10.42.0.16 raspberrypi <none> <none> kubectl -n cattle-system logs -l app=cattle-cluster-agent INFO: Environment: CATTLE_ADDRESS=10.42.0.20 CATTLE_CA_CHECKSUM=a7cebb5768bc62340e54ad0968dbe739ae5b6abc9bc5d4aa32f3858f957d5f64 CATTLE_CLUSTER=true CATTLE_CLUSTER_REGISTRY= CATTLE_FEATURES= CATTLE_INGRESS_IP_DOMAIN=sslip.io CATTLE_INSTALL_UUID=de6326b8-3223-404f-ab5c-22e526699587 CATTLE_INTERNAL_ADDRESS= CATTLE_IS_RKE=false CATTLE_K8S_MANAGED=true CATTLE_NODE_NAME=cattle-cluster-agent-6d97594f4c-bzhx5 CATTLE_SERVER=https://192.168.0.248:8443 CATTLE_SERVER_VERSION=v2.5.17 INFO: Using resolv.conf: search cattle-system.svc.cluster.local svc.cluster.local cluster.local nameserver 10.43.0.10 options ndots:5 INFO: https://192.168.0.248:8443/ping is accessible INFO: Value from https://192.168.0.248:8443/v3/settings/cacerts is an x509 certificate time="2023-08-17T115731Z" level=info msg="Listening on /tmp/log.sock" time="2023-08-17T115731Z" level=info msg="Rancher agent version 8bcacfeaf-dirty is starting" time="2023-08-17T115745Z" level=fatal msg="looking up cattle-system/cattle ca/token: no secret exists for service account cattle-system/cattle" INFO: Environment: CATTLE_ADDRESS=10.42.0.21 CATTLE_CA_CHECKSUM=a7cebb5768bc62340e54ad0968dbe739ae5b6abc9bc5d4aa32f3858f957d5f64 CATTLE_CLUSTER=true CATTLE_CLUSTER_REGISTRY= CATTLE_FEATURES= CATTLE_INGRESS_IP_DOMAIN=sslip.io CATTLE_INSTALL_UUID=de6326b8-3223-404f-ab5c-22e526699587 CATTLE_INTERNAL_ADDRESS= CATTLE_IS_RKE=false CATTLE_K8S_MANAGED=true CATTLE_NODE_NAME=cattle-cluster-agent-7fc8bfdd75-llh8d CATTLE_SERVER=https://192.168.0.248:8443 CATTLE_SERVER_VERSION=v2.5.17 INFO: Using resolv.conf: search cattle-system.svc.cluster.local svc.cluster.local cluster.local nameserver 10.43.0.10 options ndots:5 INFO: https://192.168.0.248:8443/ping is accessible INFO: Value from https://192.168.0.248:8443/v3/settings/cacerts is an x509 certificate time="2023-08-17T115629Z" level=info msg="Rancher agent version 8bcacfeaf-dirty is starting" time="2023-08-17T115629Z" level=info msg="Listening on /tmp/log.sock" time="2023-08-17T115630Z" level=fatal msg="looking up cattle-system/cattle ca/token: no secret exists for service account cattle-system/cattle"

Well, the relevant error message is the line with

level=fatal

. I’m not familiar with that error, but googling it gives some possible pointers like this: https://github.com/rancher/rancher/issues/37027#issuecomment-1240935762

Rancher 2.5.17 k3s v1.27.4+k3s1 I'm not familiar so now I'm looking into how to apply that workaround, the only file I have in my /etc/rancher/k3s is k3s.yaml. No mention of that option, I'll need to look where to apply it in the structure.