I have some networking issues with a k3s setup Is there a sp rancher-users #k3s

Title

adamant-pencil-35455

03/15/2023, 12:33 PM

I have some networking issues with a k3s setup. Is there a specific way to ask a question here?

hundreds-evening-84071

03/15/2023, 12:47 PM

I don't think there is a specific way; just ask what sort of issues you have? Also, assuming you have looked at the k3s requirements page for OS/Networking? https://docs.k3s.io/installation/requirements

adamant-pencil-35455

03/15/2023, 12:48 PM

I am having this issue... https://github.com/k3s-io/k3s/issues/4683

Trying with --flannel-ipv6-masq

I am bashing my head about this for almost one week now.

certificates with a dns hook api feels like vacation compared to the ip6 masquerading issues 🙂

image.png

That will be a big beer 🙂

plain-byte-79620

03/15/2023, 2:42 PM

Have you configured

net.ipv6.conf.all.accept_ra=2

adamant-pencil-35455

03/15/2023, 3:07 PM

yes

It works now... took me 1 week full of sweat, tears, coffee and running in circles

late-needle-80860

03/17/2023, 6:42 PM

and what did you do … Mr. @adamant-pencil-35455

adamant-pencil-35455

03/19/2023, 12:16 PM

Basically following the tutorials even more strictly

Here my setup

Proxmox Server - eth0 has ipv4 and ipv6

I created a vrbr0 interface - gave it a 10.10.x IP, I also gave it an internal iv6 IP (not one of the official IPs)

I added masquarading ipv6 (yes that's crazy - but this is how the scripts I created will work in any constellaton with all ip4/ip6 combos)

then I created 5 VMs (debian based) assigining a static ip4 and ip6 to all of them

wget -6 <https://foo>

and

wget -4 <https://foo>

is now working from all of the 5 VMs

then I installed k3s - node-ip needs to have the VMs ipv4 and ipv6

and (!) i added this to the server:

--cluster-cidr=10.42.0.0/16,2001:cafe:42:0::/56

--service-cidr=10.43.0.0/16,2001:cafe:42:1::/112

--flannel-ipv6-masq

All IPs that are cluster/service IPs are not in the same subnet as the VMs ipv4/ipv6

ipv4 and ipv6 now work inside of kubernetes

Here my test

kubectl run my-shell --rm -i --tty --image-pull-policy Always --image ubuntu -- bash
apt-get update && apt-get install -y iputils-ping
ping6 <http://dl-cdn.alpinelinux.org|dl-cdn.alpinelinux.org>
root@my-shell:/# ping6 <http://dl-cdn.alpinelinux.org|dl-cdn.alpinelinux.org>
PING <http://dl-cdn.alpinelinux.org|dl-cdn.alpinelinux.org>(2a04:4e42:8e::645 (2a04:4e42:8e::645)) 56 data bytes
64 bytes from 2a04:4e42:8e::645 (2a04:4e42:8e::645): icmp_seq=1 ttl=58 time=5.00 ...

My Proxmox Server has this in /etc/sysctl.conf

net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1
net.ipv6.conf.all.accept_ra=2

16 Views

#k3s Join Slack