hundreds-camera-73369
09/09/2024, 1:51 AMwg0
.
Then I set up K3S with flannel-iface: wg0
, and flannel-backend: host-gw
. For good measure I also used flannel-external-ip: true
. I thought this would cause all the cluster traffic to go over the WireGuard interface.
For the most part this works, but then I started noticing not all pods can talk to each other. Flannel seems to create different /24 subnets for each node, under the overall cluster pod CIDR of 10.42.0.0/24. So one node might get pods under 10.42.0.0/24, another with 10.42.1.0/24, etc. The problem (I think) is that a pod on one node will try to connect to a pod on another node, and under the hood this will try to route via the 10.100.x.x "external" address. But then WireGuard on the other node says "hey, this 10.42.x.x address is not in the valid allowedIPs
for the WireGuard peer" and drops the packet.
This hypothesis seems to be confirmed by the following: I tried manually adding the appropriate subnets to the allowedIPs
for each WireGuard peer, and then everything worked. But of course this isn't a real solution, since Flannel assigns these subnets automatically and they might change.
I was originally thinking/hoping that Flannel would magically work with this setup, but it seems not. Is this a misuse of the host-gw
backend? Should I instead use wireguard-native
or something? Any help much appreciated!