I am having trouble installing k3s on a 4 node (1 master 3 workers) raspberry pi cluster running 64 bit pi os (Debian Bullseye) and I was able to fix the master install by enabling cgroups via adding

cgroup_memory=1 cgroup_enable=memory

to

/boot/cmdline.txt

but I am running into an issue with the worker nodes not able to join the cluster with a worker role. I run the following command on the worker nodes:

curl -sfL <https://get.k3s.io> | K3S_URL=<https://rpi-kube-master-01:6443> K3S_TOKEN=<token> sh -

I can

curl

the api server just fine from the worker nodes to validate iptables/nftables aren’t an issue. I have also removed them via

sudo apt remove iptables -y && sudo apt install nftables

How can I further troubleshoot how to fix this issue with worker nodes joining with no roles?

It looks like they're joining fine. K3s doesn't use the worker role. Roles are just arbitrary labels with a standard prefix. You're welcome to add it manually if you want, but it doesn't mean anything to k3s or any of it's components.

I was previously running k3s 1.23 on my same pis and while that OS was debian buster 32bit I could have swore I saw worker roles being active for the non masters…

nope. The only thing that would prevent scheduling on a node is the presence of taints

k3s check-config

on my master shows this:

Verifying binaries in /var/lib/rancher/k3s/data/9088e57b1ba3c37820aaba60202af921dbc01b77ec0ad1e08be86b5c7bc9b8c1/bin:
- sha256sum: good
- links: aux/ip6tables should link to iptables-detect.sh (fail)
- links: aux/ip6tables-restore should link to iptables-detect.sh (fail)
- links: aux/ip6tables-save should link to iptables-detect.sh (fail)
- links: aux/iptables should link to iptables-detect.sh (fail)
- links: aux/iptables-restore should link to iptables-detect.sh (fail)
- links: aux/iptables-save should link to iptables-detect.sh (fail)

that’s a different problem than the absent role

all 3 agents hang here

look at the logs in journald, not just the install script output

interesting I see a lot of these

level=error msg="Failed to authenticate request from 192.168.1.146:42378: invalid username/password combination"

so maybe the api key is bad?

do you have a typo in the token?

sudo cat /opt/k3s/data/server/token
K102bbdc04d2df00bar4081d6f269f81d327c055bf890ee565f42dcb3::server:fc967c1c8424624646e9688867c10

In this token,

`fc967c1c8424624646e9688867c10`

is the api key I have the agents use to join right? On my worker nodes I’m running:

curl -sfL <https://get.k3s.io> | K3S_URL=<https://rpi-kube-master-01:6443> K3S_TOKEN=fc967c1c8424624646e9688867c10 sh -

copy the whole thing, not just part of it

so

K3S_TOKEN

should be:

K102bbdc04d2df00bar4081d6f269f81d327c055bf890ee565f42dcb3::server:fc967c1c8424624646e9688867c10

?

yes that is the full token. Either that or the short bit (everything after the last

:

) should work, but if in doubt use the whole thing.

Gotcha. That seemed to work. Thanks Brad 🙏

👍