https://rancher.com/ logo
Join Slack
Powered by
i'm getting these x509 errors when trying to creat...
# k3d
r
i'm getting these x509 errors when trying to create a cluster. this is just a regular k3d cluster create test. the only different thing going on here is that im running this inside a k8s pod (eks) with a dind image as the other container in the pod.
Copy code
E0728 16:28:38.880521      87 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"rancher/mirrored-pause:3.6\": failed to pull image \"rancher/mirrored-pause:3.6\": failed to pull and unpack image \"<http://docker.io/rancher/mirrored-pause:3.6\|docker.io/rancher/mirrored-pause:3.6\>": failed to resolve reference \"<http://docker.io/rancher/mirrored-pause:3.6\|docker.io/rancher/mirrored-pause:3.6\>": failed to do request: Head \"<https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6>\": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match <http://registry-1.docker.io|registry-1.docker.io>" pod="kube-system/coredns-ccb96694c-8kdvd"
E0728 16:28:38.880535      87 kuberuntime_manager.go:1170] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"rancher/mirrored-pause:3.6\": failed to pull image \"rancher/mirrored-pause:3.6\": failed to pull and unpack image \"<http://docker.io/rancher/mirrored-pause:3.6\|docker.io/rancher/mirrored-pause:3.6\>": failed to resolve reference \"<http://docker.io/rancher/mirrored-pause:3.6\|docker.io/rancher/mirrored-pause:3.6\>": failed to do request: Head \"<https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6>\": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match <http://registry-1.docker.io|registry-1.docker.io>" pod="kube-system/coredns-ccb96694c-8kdvd"
E0728 16:28:38.880556      87 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-ccb96694c-8kdvd_kube-system(40ca321a-468e-4560-b151-f9ee2541945b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-ccb96694c-8kdvd_kube-system(40ca321a-468e-4560-b151-f9ee2541945b)\\\": rpc error: code = Unknown desc = failed to get sandbox image \\\"rancher/mirrored-pause:3.6\\\": failed to pull image \\\"rancher/mirrored-pause:3.6\\\": failed to pull and unpack image \\\"<http://docker.io/rancher/mirrored-pause:3.6\\\|docker.io/rancher/mirrored-pause:3.6\\\>": failed to resolve reference \\\"<http://docker.io/rancher/mirrored-pause:3.6\\\|docker.io/rancher/mirrored-pause:3.6\\\>": failed to do request: Head \\\"<https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6>\\\": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match <http://registry-1.docker.io|registry-1.docker.io>\"" pod="kube-system/coredns-ccb96694c-8kdvd" podUID="40ca321a-468e-4560-b151-f9ee2541945b"
c
You have a wildcard dns entry, or a pihole, or something else like that, and it is misdirecting requests to docker.io to a host with a broken certificate.
Fix your DNS
❤️ 1
You can try execing into the container and `curl`ing registry-1.docker.io to see what it is hitting instead of docker hub.
r
thank you i'll need to go digging
@creamy-pencil-82913 on a gitlab k8s runner I was able to fix this by having the cluster create inside a $CI_JOB_ID network. Other network names didn’t work, and I don’t really understand why. Sorta a blocker for me as I try to get this to work w/o a dependency on gitlab.
c
I have no idea what that is or why it would be necessary but it does point to there being something broken in the default network, external to k3d
r
After a lot of digging this seems to have been caused by the following istio annotation being missing in dev and active in prod: 
<http://traffic.sidecar.istio.io/kubevirtInterfaces|traffic.sidecar.istio.io/kubevirtInterfaces>
thank you @creamy-pencil-82913 for being responsive and insightful in helping me narrow it down
8 Views
Open in Slack
PreviousNext
Powered by