you only need a TLS secret if you want to use client certificates to auth to the registry, instead of basic auth

OK, the nodes seem to be pulling images now... looks like I needed

rancher/system-agent-installer-rke2:v1.23.17-rke2r1

to successfully update the RKE servers and agents (which isn't in the RKE2 release bundle on github btw)

Hmm, this seems to occur with certain images. The calico node pod has managed to pull down calico-node from the registry, however now it's throwing a 401 trying to get the

rancher/pause:3.6

image (replaced my registry hostname with {registryaddress}):

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "{registryaddress}/rancher/pause:3. │
│ 6": failed to pull image "{registryaddress}/rancher/pause:3.6": failed to pull and unpack image "{registryaddress}/rancher/pause:3.6": failed to resolve reference "{registryaddress}/rancher/pause: │
│ 3.6": pulling from host {registryaddress} failed with status code [manifests 3.6]: 401 Unauthorized

but yeah, you’ll need that

if registry auth is working for some images it should work for all of them

rancher-system-agent.service logs are suspect:

Oct 05 15:18:06 rancher-system-agent[915]: time="2023-10-05T15:18:06+13:00" level=info msg="[Applyinator] Extracting image rancher/system-agent-installer-rke2:v1.23.17-rke2r1 to directory /var/lib/rancher/agent/work/20231005-151806/352756b4d64bd84d9b8064b13d710a637636d0fbd8094df75c28f56a5ddb838e_0"
Oct 05 15:18:06 rancher-system-agent[915]: time="2023-10-05T15:18:06+13:00" level=info msg="Using private registry config file at /etc/rancher/agent/registries.yaml"
Oct 05 15:18:06 rancher-system-agent[915]: time="2023-10-05T15:18:06+13:00" level=info msg="Pulling image index.docker.io/rancher/system-agent-installer-rke2:v1.23.17-rke2r1"
Oct 05 15:18:06 rancher-system-agent[915]: time="2023-10-05T15:18:06+13:00" level=error msg="error while staging: Get \"<https://index.docker.io/v2/>\": read tcp 172.20.101.204:57676->34.194.164.123:443: read: connection reset by peer: failed to get image index.docker.io/rancher/system-agent-installer-rke2:v1.23.17-rke2r1"
Oct 05 15:18:06 rancher-system-agent[915]: time="2023-10-05T15:18:06+13:00" level=error msg="error executing instruction 0: Get \"<https://index.docker.io/v2/>\": read tcp 172.20.101.204:57676->34.194.164.123:443: read: connection reset by peer: failed to get image index.docker.io/rancher/system-agent-installer-rke2:v1.23.17-rke2r1"

I added

<http://index.docker.io|index.docker.io>

to the list of mirrors in Rancher, but didn't help

Will I need to set the Rancher default registry to point to my private one? Or is it enough to set it at a cluster level? Would I need to rebuild my clusters if I change this value?