https://rancher.com/ logo
Title
i

incalculable-chef-11448

03/27/2023, 3:03 PM
What do most folks do when utilizing containerd as the container engine for the cluster local registry?
f

fast-garage-66093

03/27/2023, 3:56 PM
Not sure what you mean by "cluster local registry", but you don't necessarily need a registry at all: you can build images directly inside the Kubernetes namespace with
nerdctl -n <http://k8s.io|k8s.io> build ...
and reference them from your pod specs. However you must make sure to set the image pull policy to
Never
if you use the
latest
tag to prevent kubelet from trying to fetch a newer image.
The only other concern I'm aware of is that you need to maintain sufficient free space on the data volume, as kubelet will start deleting unused images when it reaches a garbage collection threshold.
i

incalculable-chef-11448

03/27/2023, 4:15 PM
Hi, @fast-garage-66093, thank you for the reply. I’m specifically working on how to utilize tilt (tilt.dev) to setup a local development environment on top of rancher desktop.
f

fast-garage-66093

03/27/2023, 6:40 PM
Ah, ok. I have never used Tilt, so I don't know, but I guess it needs a local registry to push to? You can definitely run a registry as a container and use that from within the cluster.
i

incalculable-chef-11448

03/27/2023, 6:42 PM
For now, I’ve swapped back to using dockerd/moby instead of containerd … but … to continue the discussion to maybe figure out what would be needed for containerd … one of the tilt folks was suggesting that I needed a way to get the built image “into the cluster”…
f

fast-garage-66093

03/27/2023, 6:42 PM
I have written some bats tests for Rancher Desktop that deploys an internal registry for testing registry access with credentials. It may be a bit complex, but could give you an idea on what can be done: https://github.com/rancher-sandbox/rancher-desktop/blob/main/bats/tests/registry/creds.bats
The way to get an image "into the cluster" is by building inside the cluster namespace (
-n <http://k8s.io|k8s.io>
). Note that these are containerd namespaces, not Kubernetes namespaces.
Containerd organizes images into namespaces, and only images from the
<http://k8s.io|k8s.io>
namespace are visible to the cluster.
i

incalculable-chef-11448

03/27/2023, 6:48 PM
Thanks for this, Jan. This is helping me to see the pieces and how they come together.