most-sunset-36476
08/19/2022, 9:36 AMtime="2022-08-19T07:03:38Z" level=info msg="Connecting to <wss://rancher.gandalf.mordor.net/v3/connect> with token starting with 4sfxz9dk54w2hp9xv729k8tbgnf"
time="2022-08-19T07:03:38Z" level=info msg="Connecting to proxy" url="<wss://rancher.gandalf.mordor.net/v3/connect>"
time="2022-08-19T07:03:48Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:03:48Z" level=error msg="Remotedialer proxy error" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:03:50Z" level=warning msg="Error while getting agent config: Get \"<https://rancher.gandalf.mordor.net/v3/connect/config>\": dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:03:58Z" level=info msg="Connecting to <wss://rancher.gandalf.mordor.net/v3/connect> with token starting with 4sfxz9dk54w2hp9xv729k8tbgnf"
time="2022-08-19T07:03:58Z" level=info msg="Connecting to proxy" url="<wss://rancher.gandalf.mordor.net/v3/connect>"
time="2022-08-19T07:04:08Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:08Z" level=error msg="Remotedialer proxy error" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:18Z" level=info msg="Connecting to <wss://rancher.gandalf.mordor.net/v3/connect> with token starting with 4sfxz9dk54w2hp9xv729k8tbgnf"
time="2022-08-19T07:04:18Z" level=info msg="Connecting to proxy" url="<wss://rancher.gandalf.mordor.net/v3/connect>"
time="2022-08-19T07:04:25Z" level=warning msg="Error while getting agent config: Get \"<https://rancher.gandalf.mordor.net/v3/connect/config>\": dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:28Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:28Z" level=error msg="Remotedialer proxy error" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:38Z" level=info msg="Connecting to <wss://rancher.gandalf.mordor.net/v3/connect> with token starting with 4sfxz9dk54w2hp9xv729k8tbgnf"
time="2022-08-19T07:04:38Z" level=info msg="Connecting to proxy" url="<wss://rancher.gandalf.mordor.net/v3/connect>"
time="2022-08-19T07:04:48Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:48Z" level=error msg="Remotedialer proxy error" error="dial tcp 20.42.142.242:443: i/o timeout"
time="2022-08-19T07:04:58Z" level=info msg="Connecting to <wss://rancher.gandalf.mordor.net/v3/connect> with token starting with 4sfxz9dk54w2hp9xv729k8tbgnf"
time="2022-08-19T07:04:58Z" level=info msg="Connecting to proxy" url="<wss://rancher.gandalf.mordor.net/v3/connect>"
time="2022-08-19T07:05:00Z" level=warning msg="Error while getting agent config: Get \"<https://rancher.gandalf.mordor.net/v3/connect/config>\": dial tcp 20.42.142.242:443: i/o timeout"
loud-airline-16221
08/19/2022, 9:46 AMmagnificent-glass-73162
08/19/2022, 9:52 AM$ kubectl -n kube-system get pods -l k8s-app=kube-dns
NAME READY STATUS RESTARTS AGE
coredns-685d6d555d-q9pn4 0/1 CrashLoopBackOff 4 2m40s
If I look at the pod description I see the events as follows:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SandboxChanged 62m (x2 over 62m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal Pulled 62m kubelet Container image "rancher/mirrored-cluster-proportional-autoscaler:1.8.3" already present on machine
Normal Created 62m kubelet Created container autoscaler
Normal Started 62m kubelet Started container autoscaler
Normal SandboxChanged 52m (x2 over 52m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal Pulled 52m kubelet Container image "rancher/mirrored-cluster-proportional-autoscaler:1.8.3" already present on machine
Normal Created 52m kubelet Created container autoscaler
Normal Started 52m kubelet Started container autoscaler
Warning Unhealthy 51m kubelet Readiness probe failed: Get "<http://10.42.0.17:8080/healthz>": dial tcp 10.42.0.17:8080: i/o timeout (Client.Timeout exceeded while awaiting headers)
Warning Unhealthy 51m (x3 over 52m) kubelet Readiness probe failed: Get "<http://10.42.0.17:8080/healthz>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Normal SandboxChanged 37m (x2 over 37m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal Pulled 37m kubelet Container image "rancher/mirrored-cluster-proportional-autoscaler:1.8.3" already present on machine
Normal Created 37m kubelet Created container autoscaler
Normal Started 37m kubelet Started container autoscaler
Warning Unhealthy 36m (x2 over 37m) kubelet Readiness probe failed: Get "<http://10.42.0.93:8080/healthz>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Normal SandboxChanged 22m (x2 over 22m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal Pulled 22m kubelet Container image "rancher/mirrored-cluster-proportional-autoscaler:1.8.3" already present on machine
Normal Created 22m kubelet Created container autoscaler
Normal Started 22m kubelet Started container autoscaler
Warning Unhealthy 21m (x4 over 22m) kubelet Readiness probe failed: Get "<http://10.42.0.161:8080/healthz>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
It seems to be failing the health check, but I’m not sure whyloud-airline-16221
08/19/2022, 10:06 AMnice-needle-20936
08/19/2022, 11:32 AMhelpful-beard-54962
08/19/2022, 11:49 AMstraight-fireman-71417
08/19/2022, 4:55 PMred-musician-8168
08/19/2022, 8:15 PMred-musician-8168
08/19/2022, 8:17 PMbroad-ambulance-17822
08/19/2022, 8:18 PMbroad-ambulance-17822
08/19/2022, 8:18 PMred-musician-8168
08/19/2022, 8:18 PMred-musician-8168
08/19/2022, 8:18 PMbroad-ambulance-17822
08/19/2022, 8:18 PMred-musician-8168
08/19/2022, 8:19 PMbroad-ambulance-17822
08/19/2022, 8:19 PMred-musician-8168
08/19/2022, 8:19 PMred-musician-8168
08/19/2022, 8:24 PMbroad-ambulance-17822
08/19/2022, 8:24 PMhundreds-state-15112
08/19/2022, 10:34 PMbland-area-30120
08/20/2022, 12:01 PMflat-glass-90449
08/20/2022, 1:03 PMplain-portugal-84182
08/21/2022, 8:49 AMhappy-football-1417
08/21/2022, 1:46 PMmost-alarm-2259
08/21/2022, 3:06 PMmost-alarm-2259
08/21/2022, 3:08 PMcareful-optician-75900
08/22/2022, 4:13 AMcareful-optician-75900
08/22/2022, 4:13 AMwitty-jelly-95845
08/22/2022, 9:10 AMtall-doctor-28108
08/22/2022, 9:12 AM