https://rancher.com/ logo
Title
p

polite-engineer-55788

08/16/2022, 12:21 PM
is there a way to configure
insecure
registry with k3d using registry k8s service on my cluster
this is more of k3s question as well
w

wide-garage-9465

08/16/2022, 12:42 PM
Are you referring to using the
registries.yaml
config file? See https://k3d.io/v5.4.4/usage/registries/#registries-configuration-file
p

polite-engineer-55788

08/16/2022, 12:43 PM
yeah I was using it as my reference but in my case I have Nexus Repo Manager acting as my Container Registry and that gets deployed on the cluster once its up
and I am trying to pull an image from that local kubernetes service
and seeing error like
Failed to pull image "nexus.infra.svc.cluster.local:18081/example/quarkus-springboot-demo:latest@sha256:83d0dbb77648335bd17862081590b65c932977ae97c0fdf4635e47f3523936ae": rpc error: code = Unknown desc = failed to pull and unpack image "nexus.infra.svc.cluster.local:18081/example/quarkus-springboot-demo@sha256:83d0dbb77648335bd17862081590b65c932977ae97c0fdf4635e47f3523936ae": failed to resolve reference "nexus.infra.svc.cluster.local:18081/example/quarkus-springboot-demo@sha256:83d0dbb77648335bd17862081590b65c932977ae97c0fdf4635e47f3523936ae": failed to do request: Head "<https://nexus.infra.svc.cluster.local:18081/v2/example/quarkus-springboot-demo/manifests/sha256:83d0dbb77648335bd17862081590b65c932977ae97c0fdf4635e47f3523936ae>": dial tcp: lookup nexus.infra.svc.cluster.local on 127.0.0.11:53: read udp 127.0.0.1:59808->127.0.0.11:53: i/o timeout
where in my push to same repo works excellently
do you thing something like this will work
mirrors:
"nexus.infra.svc.cluster.local":
  endpoint:
    - "<http://host.k3d.internal>"
configs:
  "nexus.infra.svc.cluster.local":
    auth:
      username: admin
      password: admin123
w

wide-garage-9465

08/16/2022, 12:46 PM
and I am trying to pull an image from that local kubernetes service
You mean creating a pod with an image from that registry?
containerd
is not inside the cluster, so it doesn't know about the service names. You can use the name in the registries.yaml and point it e.g. to a nodeport service.
do you thing something like this will work
host.k3d.internal will redirect pulls to your local machine, so I guess not. It would work if you've exposed the registry using some port-forwarding to your host machine. Then you'd need to add the port to the endpoint config.
p

polite-engineer-55788

08/16/2022, 12:47 PM
You mean creating a pod with an image from that registry?
containerd is not inside the cluster, so it doesn’t know about the service names. You can use the name in the registries.yaml and point it e.g. to a nodeport service.
mirrors:
"nexus.infra.svc.cluster.local":
  endpoint:
    - "<http://nexus.infra.svc.cluster.local>"
configs:
  "nexus.infra.svc.cluster.local":
    auth:
      username: admin
      password: admin123
?
where
nexus.infra.svc.cluster.local
is my Registry service on the cluster and ha node port to it
yeah I do have portforward like
- port: 127.0.0.1:31081:31081
    nodeFilters:
     - loadbalancer
where
31081
is the nodeport of my registry service
w

wide-garage-9465

08/16/2022, 12:49 PM
<http://nexus.infra.svc.cluster.local>
containerd has no idea how to resolve that name. You can make it easy: Use a
type: NodePort
service with
nodePort: 30555
(example), then add
mirrors:
"nexus.infra.svc.cluster.local":
  endpoint:
    - "<http://localhost:30555>"
configs:
  "nexus.infra.svc.cluster.local":
    auth:
      username: admin
      password: admin123
I didn't test this though.
Your last example will do as well, if you adapt the registries.yaml accordingly
p

polite-engineer-55788

08/16/2022, 12:50 PM
I can also strip of the port in the endpoint and add it while pushing/pulling the image right?
me trying … 🤞🏽
@wide-garage-9465 the registry config worked like a charm 🙂
thanks for the help
w

wide-garage-9465

08/16/2022, 3:48 PM
👍