Hey there I hit an issue when I start new nodes and start a rancher-users #general

Title

enough-xylophone-4568

08/26/2022, 2:03 PM

Hey there! I hit an issue when I start new nodes and start a old rancher agent version on it. Ideally, it should automatically pull the new rancher image and start a replacement docker container by itself. In my case, it fails at pulling the agent image with

Error response from daemon: Head \"<https://foo.azurecr.io/v2/rancher/rancher-agent/manifests/v2.6.7>\": unauthorized: authentication required, visit <https://aka.ms/acr/authorization> for more information.

I had a look at the source code, and I believe it crashes when trying to pull the

share-mnt

process image, which has no

imageRegistryAuthConfig

settings section in the nodePlan object. How can I ensure

share-mnt

process is configured with the right ``imageRegistryAuthConfig`` , given that all the other processes have the right credentials attached?

I confirm editing the share-mnt process in the nodePlan (from nodes.management.cattle.io) did the trick. I unlocked rancher-agent from pulling the agent image.

I guess https://github.com/rancher/rancher/blob/release/v2.6/pkg/rkenodeconfigserver/nodeserver.go#L288 comes with an empty string (""), is

cluster

null?

So there's clearly a bug on the Rancher side.

What I did is: Add the following annotation to rancher server ingress (since I was not able to find an existing unauthentified endpoint that serves the server version):

<http://nginx.ingress.kubernetes.io/configuration-snippet|nginx.ingress.kubernetes.io/configuration-snippet>: |
                location /rancher-version {
                    return 200 '${rancher_current_version}';
                }

And then, in my cloud-init, add the following step before starting the rancher-agent:

# Pull latest Rancher agent
rancher_current_version=$(curl -sfSL ${rancher_url}/rancher-version)
docker pull "${docker_registry_url}/rancher/rancher-agent:$rancher_current_version"

So, when the Rancher agent connects and checks if the share-mnt image is present, docker says yes and the pull is skipped: https://github.com/rancher/rancher/blob/5b56b17e2515302ad07b9e14e73024455157c546/pkg/rkeworker/docker.go#L138

7 Views

#general Join Slack