https://rancher.com/ logo
Title
a

adamant-pencil-35455

03/15/2023, 12:33 PM
I have some networking issues with a k3s setup. Is there a specific way to ask a question here?
h

hundreds-evening-84071

03/15/2023, 12:47 PM
I don't think there is a specific way; just ask what sort of issues you have? Also, assuming you have looked at the k3s requirements page for OS/Networking? https://docs.k3s.io/installation/requirements
a

adamant-pencil-35455

03/15/2023, 12:48 PM
Trying with --flannel-ipv6-masq
I am bashing my head about this for almost one week now.
certificates with a dns hook api feels like vacation compared to the ip6 masquerading issues 🙂
image.png
That will be a big beer 🙂
p

plain-byte-79620

03/15/2023, 2:42 PM
Have you configured
net.ipv6.conf.all.accept_ra=2
?
a

adamant-pencil-35455

03/15/2023, 3:07 PM
yes
It works now... took me 1 week full of sweat, tears, coffee and running in circles
l

late-needle-80860

03/17/2023, 6:42 PM
and what did you do … Mr. @adamant-pencil-35455
a

adamant-pencil-35455

03/19/2023, 12:16 PM
Basically following the tutorials even more strictly
Here my setup
Proxmox Server - eth0 has ipv4 and ipv6
I created a vrbr0 interface - gave it a 10.10.x IP, I also gave it an internal iv6 IP (not one of the official IPs)
I added masquarading ipv6 (yes that's crazy - but this is how the scripts I created will work in any constellaton with all ip4/ip6 combos)
then I created 5 VMs (debian based) assigining a static ip4 and ip6 to all of them
wget -6 <https://foo>
and
wget -4 <https://foo>
is now working from all of the 5 VMs
then I installed k3s - node-ip needs to have the VMs ipv4 and ipv6
and (!) i added this to the server:
--cluster-cidr=10.42.0.0/16,2001:cafe:42:0::/56
--service-cidr=10.43.0.0/16,2001:cafe:42:1::/112
--flannel-ipv6-masq
All IPs that are cluster/service IPs are not in the same subnet as the VMs ipv4/ipv6
ipv4 and ipv6 now work inside of kubernetes
Here my test
kubectl run my-shell --rm -i --tty --image-pull-policy Always --image ubuntu -- bash
apt-get update && apt-get install -y iputils-ping
ping6 <http://dl-cdn.alpinelinux.org|dl-cdn.alpinelinux.org>
root@my-shell:/# ping6 <http://dl-cdn.alpinelinux.org|dl-cdn.alpinelinux.org>
PING <http://dl-cdn.alpinelinux.org|dl-cdn.alpinelinux.org>(2a04:4e42:8e::645 (2a04:4e42:8e::645)) 56 data bytes
64 bytes from 2a04:4e42:8e::645 (2a04:4e42:8e::645): icmp_seq=1 ttl=58 time=5.00 ...
My Proxmox Server has this in /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1
net.ipv6.conf.all.accept_ra=2