anyone else run into issues with ubuntu 22.04 and coredns dns queries from pods not working? They work fine in 20.04 but on 22.04 nodes the dns entries cant resolve and the pods crash out.

I've seen it sporadically, but not predictably on 20.04 too. It seems like there's some chance of it hitting any system that runs its own DNS resolver so has 127.0.0.x in /etc/resolv.conf We fixed by editing coredns ConfigMap to include the DNS IP it should be pointed to, then things were fine.

hmm, think im getting closer, in my case it looks to be related to flannel etc not properly injecting the ip table routes

That's certainly a worse problem than mine. I can certainly see how coredns would be one of the early canaries for that one, though. I don't recall, but does flannel have pods to poke & check the logs?

yea the flannel pod pokes at the iptables pretty consistantly, but i dont see any of the entries in its logs its trying to add getting through to the actual iptables routes

I don't know if iptables is involved with the routing, I thought that showed up with the

route

command instead.

sorry ip route, your right

Not sure what may have changed with networking and/or network permissions, we ran into some other problems with 22.04 before we even got to Kubernetes and backed off to 20.04 to eliminate the possibility (and our production environment is RHEL clones, so Ubuntu was more just for dev systems and ok to ignore).

this may have fixed it, still tshooting

update-alternatives --set iptables /usr/sbin/iptables-legacy

Nice to hear, I'll keep it in mind if I hear similar.