This message was deleted.
# general
a
This message was deleted.
c
usually indicates CNI (vxlan) traffic between the node running the pod, and the node running coredns, is being dropped. Make sure you’ve opened the correct ports, and if you’re using vmware, that you’ve turned off checksum offload.
e
This is intermittent issue sometime its works also so how can be the port issue here ?
but takes sometime to response on ping or nslookup
c
because sometimes your pod is on the same node as coredns and sometimes its not?
e
i tried to lauch pod on both nodes master as well as worker but having same issue from both the vms
seems canal cni having issue and i am able to ping 8.8.8.8 every time and its working fine but issue with name resolution only sometime IP which is in resolv.conf 10.43.0.10 this ip sometime listen on 53 port and sometime not / # cat /etc/resolv.conf search default.svc.cluster.local svc.cluster.local cluster.local nameserver 10.43.0.10 options ndots:5 / # nz -zv 10.43.0.10 53 /bin/sh: nz: not found / # nc -zv 10.43.0.10 53 10.43.0.10 (10.43.0.10:53) open / # nc -zv 10.43.0.10 53 10.43.0.10 (10.43.0.10:53) open / # nc -zv 10.43.0.10 53 10.43.0.10 (10.43.0.10:53) open / # nc -zv 10.43.0.10 53 ^[[A ^[[A ^[[A
s
Have you already made sure that the overlay network in General functions fine? You can run this test to see if there are issues: https://ranchermanager.docs.rancher.com/troubleshooting/other-troubleshooting-tips/networking#check-if-overlay-network-is-functioning-correctly
e
Thanks let me check
root@rke-master1:~# cat >z #!/bin/bash echo "=> Start network overlay test" kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.nodeName}{"\n"}{end}' | while read spod shost do kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.status.podIP}{" "}{@.spec.nodeName}{"\n"}{end}' | while read tip thost do kubectl --request-timeout='10s' exec $spod -c overlaytest -- /bin/sh -c "ping -c2 $tip > /dev/null 2>&1" RC=$? if [ $RC -ne 0 ] then echo FAIL: $spod on $shost cannot reach pod IP $tip on $thost else echo $shost can reach $thost fi done done echo "=> End network overlay test" root@rke-master1:~# bash z => Start network overlay test rke-master1 can reach rke-master1 rke-master1 can reach rke-worker1 rke-worker1 can reach rke-master1 rke-worker1 can reach rke-worker1 => End network overlay test root@rke-master1:~#
it seems fine here
r
Hi Matt also when inside pod if i trace route to coredns ip. it not going directly. # traceroute -I -p 53 10.43.0.10 traceroute to 10.43.0.10 (10.43.0.10), 30 hops max, 46 byte packets 1 172.25.68.5 (172.25.68.5) 0.007 ms 0.121 ms 0.004 ms 2 172.25.68.254 (172.25.68.254) 0.073 ms 0.042 ms 0.037 ms 3 100.64.192.12 (100.64.192.12) 0.335 ms 0.522 ms 0.166 ms 4 100.103.211.181 (100.103.211.181) 0.676 ms 0.551 ms 0.564 ms 5 100.103.9.254 (100.103.9.254) 27.874 ms 1.288 ms 0.892 ms 6 100.64.99.24 (100.64.99.24) 1.474 ms 1.431 ms 1.087 ms 7 100.64.99.249 (100.64.99.249) 1.229 ms 1.337 ms 1.118 ms 8 * *
152 Views