This message was deleted.

I'd probably try

kubectl exec --stdin --tty ${POD} -- /bin/bash

for a pod running on the agent that has bash and then poke around looking at /etc/resolv.conf & running nslookup, dig, ping, or whatever else was a good test for my particular concern.

for your command i get

Unable to connect to the server: dial tcp: lookup tdin on 127.0.0.53:53: server misbehaving

but

kubectl exec -it <pod> -- bash

works. However,

/etc/resolv.conf

does not include the

nameservers

I listed in the file supplied to

--resolv.conf

. Also

nslookup <http://google.com|google.com>

fails with

;; connection timed out; no servers could be reached

My guess is you're using a distro that runs local DNS and puts 127.0.0.53 in resolv.conf (newer Ubuntu does that). The problem is that localhost on a pod is the pod itself, not the pod's host. There are multiple ways to get around it, but the one I saw was editing the configMap for coredns (it's in the kube-system namespace). Look for a line saying 'forward . /etc/resolv.conf' and add your preferred DNS IP after it, so 'forward . /etc/resolv.conf 8.8.8.8' if you wanted you use one of Google's DNS as an example.

dial tcp: lookup tdin on 127.0.0.5353 server misbehaving

you did

-s tdin

didn’t you

I had

-stdin

instead of

--stdin

🤦‍♂️. That error is gone and I can use the command mentioned by @rough-farmer-49135. The error, however is the same

nslookup <http://google.com|google.com>

fails. My flag with my modified

resolv.conf

is

--resolv-conf=/etc/k3s-resolv.conf

and this also comes up in the k3s logs ( the

k3s-agent.service

also mentions it).

what are you trying to accomplish by changing the resolv.conf? Are you trying to change where coredns goes for upstream lookups, or change the dns search order or what? Most pods are going to be running with cluster DNS so they will be resolving things via the in-cluster dns service and coredns.

I'm guessing he had similar problems we had with running k3d on some newer Ubuntu that pointed to 127.0.0.53 in resolv.conf . For some of those installs, but not all oddly enough, that ended up blocking pods from being able to resolve host names external to the cluster on our internal network and we had some places where it's needed. The solution we found for those was what I wrote above with the coredns ConfigMap, just passing in our internal network DNS instead of 8.8.8.8. I'm sure there could be other solutions, but that worked and was easy to do.

Basically I am trying to connect my pod to the internet 🙂 . My OS on all 4 nodes is Ubuntu 22.04 (k3s 1.25.6) and a pod on one specific node cannot reach pods on other nodes and vice versa. I read that setting the nameserver in a custom resolv.conf could solve it. I also tried https://ranchermanager.docs.rancher.com/troubleshooting/other-troubleshooting-tips/networking which I think is similar to what I am doing

a pod on one specific node cannot reach pods on other nodes and vice versa.

That indicates a problem with your CNI traffic, not a problem with DNS

Not the same network, but ports should all be open. Are you aware of any test script or command that I could use to test/debug the CNI? ( I am using the default k3s network backend).