Hey guys, just started to play with k3s some weeks...
# k3s
q
Hey guys, just started to play with k3s some weeks ago and I love it! I want to use k3s to deploy software to restricted and isolated machines in different private networks. Till now the proof of concept worked fine, but I want to confirm that what I want to do makes sense, before using it in production. 1. server nodes are in the same private network and have public ips, ports opened - they can reach each other - everything works fine 2. agent nodes can reach server nodes via k3s tunneling and I'm able to get logs/exec into pods - everything works fine But...
The thing is that: - some agent nodes have the same ips (as long as they are in different networks it's possible) - agent nodes can't use the public ip because no inbound ports are opened (and not allowed to open) and 2 agent nodes deployed in the same private network has the same public ip in my case Is it ok from k3s tunneling point of view to have duplicate node ips? All I need from network point of view is: 1. k3s tunneling kube api server to work (logs/exec into agent pods etc) 2. pod to pod communication inside agent node 3. exposing pods using HostPort (pods have to be reachable just in the private network where the node is located) I don't need: 1. node to node communication 2. cross nodes pod communication 3. kube services Is it a problem from k3s tunneling/networking point of view to have duplicate node ips, as long as that node ips are unique in their private network? I attached here a diagram of my situation and a screenshot with the agent nodes in my cluster.
k3s-topology.jpg
here is a screenshot with my agent nodes list with duplicate ips (I have other duplicate ips too)
the cluster has around 200 agent nodes
I ask this because I encounter randomly failures when I want to exec into pods on machines with the same private ip, but maybe it's just a coincidence
c
Yeah that is going to cause problems. Hostnames, internal IPs, and external IPs, are all expected to be unique.
q
😞
so the only way I see doing it is to create a vpn and join all nodes there, and there I have more control over networking
I'll give some more tries to kairos p2p feature 👍
@creamy-pencil-82913, even if I don't need cluster traffic at all? All I need is k3s server <-> agent tunneling to work, to be able to schedule pods on these agent nodes with duplicate IPs