worried-receptionist-18982
09/18/2023, 11:09 PMkubectl port-forward
i can get any agent node to connect to a k3s service just fine. but if i start a pod, the pod can't seem to access the service. is there a good way to debug this issue? im using default flannel config, maybe it's an iptables issue?creamy-pencil-82913
09/18/2023, 11:15 PMworried-receptionist-18982
09/18/2023, 11:16 PMcreamy-pencil-82913
09/18/2023, 11:16 PMworried-receptionist-18982
09/18/2023, 11:16 PMcreamy-pencil-82913
09/18/2023, 11:16 PMworried-receptionist-18982
09/18/2023, 11:16 PMcreamy-pencil-82913
09/18/2023, 11:17 PMworried-receptionist-18982
09/18/2023, 11:17 PMcreamy-pencil-82913
09/18/2023, 11:17 PMworried-receptionist-18982
09/18/2023, 11:18 PMcreamy-pencil-82913
09/18/2023, 11:18 PMworried-receptionist-18982
09/18/2023, 11:18 PMkubectl port-forward
works tho, it's just pod tries to access service no gocreamy-pencil-82913
09/18/2023, 11:19 PMworried-receptionist-18982
09/18/2023, 11:20 PMkubectl port-forward
and a pod accessing a service? dont they both do the same thing basically via kube proxy?creamy-pencil-82913
09/18/2023, 11:20 PMworried-receptionist-18982
09/18/2023, 11:20 PMcreamy-pencil-82913
09/18/2023, 11:21 PMworried-receptionist-18982
09/18/2023, 11:21 PMcreamy-pencil-82913
09/18/2023, 11:22 PMworried-receptionist-18982
09/18/2023, 11:22 PMcreamy-pencil-82913
09/18/2023, 11:22 PMworried-receptionist-18982
09/18/2023, 11:22 PMcreamy-pencil-82913
09/18/2023, 11:23 PMworried-receptionist-18982
09/18/2023, 11:23 PMflannel-backend=wireguard
then that's tunneling everything thru encrypted wireguard effectively? maybe i should just try that. i dont need the wg tunnels today but some day i might, e.g. hybrid cloud set-upcreamy-pencil-82913
09/18/2023, 11:30 PMethtool --offload eth0 rx off tx off
on all the nodesethtool -K eth0 gso off
helpsworried-receptionist-18982
09/18/2023, 11:34 PMethtool
let me try that out. i kinda wanted to give wg
a spin anyways too. thanks for helping me understand diff between proxy / tunnel and the overlay, i had mistaken that the proxy / tunnel always gets usedcreamy-pencil-82913
09/18/2023, 11:38 PMworried-receptionist-18982
09/19/2023, 10:29 PMethtool
stuff and re-installed 1.25.12 several times so im pretty sure the ethtool stuff didn't fix (tho i do see in github issues, several from years ago, it's a fix for many people).
the issue was that pod ip was routable but not service ip. seems several people have seen this, e.g. https://github.com/k3s-io/k3s/issues/1638#issuecomment-615088410
any idea on what in ... flannel? might have changed in k8s 1.25.12 -> 1.28.1 to mitigate this particular issue? i've got ubuntu 20.04 and i never touched the iptables stuff, which seems to be related. it sounds like flannel had some mac address issues that got merged but that was i think a while ago? will keep testing a bit
welp so i was using a release that was fairly old i guess, v1.28.1+k3s1