Hi guys, any idea where I can look for DNS issues? Deployd awx but can't resolve our internal gitlab. I can resolve external addresses and also internal/local IPs - but not internal names. (for example our domain is

domain-xx.local

and

10.2.17.21

is the vm (

server1

running k3s.))

kubectl run -it --rm --restart=Never busybox --image=busybox:1.28 -- nslookup 10.2.17.21
Server:    192.168.246.10
Address 1: 192.168.246.10 kube-dns.kube-system.svc.domain-xx.local

Name:      10.2.17.21
Address 1: 10.2.17.21 server1.domain-xx.local

Lookup with IP is working ✅ Now I'm trying to resolve the same system via name:

kubectl run -it --rm --restart=Never busybox --image=busybox:1.28 -- nslookup server1.domain-xx.local
Server:    192.168.246.10
Address 1: 192.168.246.10 kube-dns.kube-system.svc.domain-xx.local

nslookup: can't resolve 'server1.domain-xx.local'

Lookup with name is not working. ❌ I'm new to k3s, so bear with me. It's a single k3s node, fresh installed. The host can resolve in both ways. When I'm running:

kubectl run -i --restart=Never --rm test-${RANDOM} --image=ubuntu --overrides='{"kind":"Pod", "apiVersion":"v1", "spec": {"dnsPolicy":"Default"}}' -- sh -c 'cat /etc/resolv.conf'

to check for the /etc/resolv.conf it shows me the correct entries for the search domain and our dns server. kube-dns (coredns) is runnig and dns server is

192.168.246.10

pod:
coredns-597584b69b-9hspl   1/1     Running   1 (65m ago)   87m

svc:
kube-dns   ClusterIP   192.168.246.10   <none>        53/UDP,53/TCP,9153/TCP   87m

wouldnt it be

server1.svc.namespace.domain-xx.local

?

Yes, I'm trying to reach our gitlab server, which is another vm (not running k3s). But in this example I'm trying to make a lookup on the k3s host (ip and name). But I can't lookup any other vm with the name.

are you using the same domain for your Kubernetes cluster as your actual domain?

yes, all in one domain

I wouldn’t do that. coredns assumes it is authoritative for the cluster domain. It isn’t going to forward lookups for the cluster domain out to your primary dns server.

ah wait, what exactly you mean with cluster domain? My 'real' domain is domain-xx.local. This is the domain where our gitlab, dns server and also the k3s host is. I did not specify any special domain in k3s installation.

ok. it can be hard to tell when people redact things.

I replaced our real domain with domain-xx.local .

That suggests that you’re using your real domain as the cluster domain then

ah, got it now. Sorry.

which you shouldn’t do

yes, i set up the cluster with --cluster-domain domain-xx.local

yep. don’t do that.

got it, will fix it and try again. Thank you very much!

so

did not specify any special domain in k3s installation.

was not accurate

any ressources for best practices where I can find such info?

you should uninstall and reinstall, the cluster domain can’t be changed after the cluster is brought up.

Yes, it was not. Sorry

np!

Yeah, I thougth setting up the same cluster-domain would help. Did not thought about DNS Zones at the moment. Learned my lesson now. 🙂 Will change it to something different.

it looks like you’ve also customized the cluster and service CIDRs, those don’t overlap with your actual network ranges do they?

I have to do it. The default ranges overlapping some of our used network ranges.

ok. just make sure that the ranges you picked don’t overlap with any actual ranges, and that the cluster-cidr is large enough to hold as many nodes as you plan on adding.