https://rancher.com/ logo
#k3s
Title
a

acceptable-lunch-74034

02/21/2023, 5:56 PM
I am having trouble installing k3s on a 4 node (1 master 3 workers) raspberry pi cluster running 64 bit pi os (Debian Bullseye) and I was able to fix the master install by enabling cgroups via adding
cgroup_memory=1 cgroup_enable=memory
to
/boot/cmdline.txt
but I am running into an issue with the worker nodes not able to join the cluster with a worker role. I run the following command on the worker nodes:
Copy code
curl -sfL <https://get.k3s.io> | K3S_URL=<https://rpi-kube-master-01:6443> K3S_TOKEN=<token> sh -
I can
curl
the api server just fine from the worker nodes to validate iptables/nftables aren’t an issue. I have also removed them via
sudo apt remove iptables -y && sudo apt install nftables
How can I further troubleshoot how to fix this issue with worker nodes joining with no roles?
c

creamy-pencil-82913

02/21/2023, 6:24 PM
It looks like they're joining fine. K3s doesn't use the worker role. Roles are just arbitrary labels with a standard prefix. You're welcome to add it manually if you want, but it doesn't mean anything to k3s or any of it's components.
a

acceptable-lunch-74034

02/21/2023, 6:29 PM
I was previously running k3s 1.23 on my same pis and while that OS was debian buster 32bit I could have swore I saw worker roles being active for the non masters…
I thought if a node has no roles, nothing can be scheduled on them
c

creamy-pencil-82913

02/21/2023, 6:43 PM
nope. The only thing that would prevent scheduling on a node is the presence of taints
k3s has never used the worker role
a

acceptable-lunch-74034

02/21/2023, 6:53 PM
k3s check-config
on my master shows this:
Copy code
Verifying binaries in /var/lib/rancher/k3s/data/9088e57b1ba3c37820aaba60202af921dbc01b77ec0ad1e08be86b5c7bc9b8c1/bin:
- sha256sum: good
- links: aux/ip6tables should link to iptables-detect.sh (fail)
- links: aux/ip6tables-restore should link to iptables-detect.sh (fail)
- links: aux/ip6tables-save should link to iptables-detect.sh (fail)
- links: aux/iptables should link to iptables-detect.sh (fail)
- links: aux/iptables-restore should link to iptables-detect.sh (fail)
- links: aux/iptables-save should link to iptables-detect.sh (fail)
Does this mean I need to do something with iptables? ( I thought I already removed them)
my other worker nodes are just hanging on “Starting k3s agent” and idk why my master would suddenly stop working as the api server
c

creamy-pencil-82913

02/21/2023, 6:58 PM
that’s a different problem than the absent role
where are the agents hanging? what do the logs show?.
a

acceptable-lunch-74034

02/23/2023, 9:42 PM
all 3 agents hang here
c

creamy-pencil-82913

02/23/2023, 9:46 PM
look at the logs in journald, not just the install script output
a

acceptable-lunch-74034

02/23/2023, 9:53 PM
interesting I see a lot of these
level=error msg="Failed to authenticate request from 192.168.1.146:42378: invalid username/password combination"
so maybe the api key is bad?
c

creamy-pencil-82913

02/23/2023, 9:54 PM
do you have a typo in the token?
a

acceptable-lunch-74034

02/23/2023, 9:54 PM
Copy code
sudo cat /opt/k3s/data/server/token
K102bbdc04d2df00bar4081d6f269f81d327c055bf890ee565f42dcb3::server:fc967c1c8424624646e9688867c10
In this token,
Copy code
`fc967c1c8424624646e9688867c10`
is the api key I have the agents use to join right? On my worker nodes I’m running:
Copy code
curl -sfL <https://get.k3s.io> | K3S_URL=<https://rpi-kube-master-01:6443> K3S_TOKEN=fc967c1c8424624646e9688867c10 sh -
c

creamy-pencil-82913

02/23/2023, 9:55 PM
copy the whole thing, not just part of it
a

acceptable-lunch-74034

02/23/2023, 9:56 PM
so
K3S_TOKEN
should be:
K102bbdc04d2df00bar4081d6f269f81d327c055bf890ee565f42dcb3::server:fc967c1c8424624646e9688867c10
?
c

creamy-pencil-82913

02/23/2023, 10:01 PM
yes that is the full token. Either that or the short bit (everything after the last
:
) should work, but if in doubt use the whole thing.
a

acceptable-lunch-74034

02/23/2023, 10:34 PM
Gotcha. That seemed to work. Thanks Brad 🙏
I think there was something funky going on when copying and pasting in my zsh that must have screwed up the token string.
c

creamy-pencil-82913

02/23/2023, 10:40 PM
👍
57 Views