I guess https://github.com/rancher/rancher/blob/release/v2.6/pkg/rkenodeconfigserver/nodeserver.go#L288 comes with an empty string (""), is

cluster

null?

So there's clearly a bug on the Rancher side.

What I did is: Add the following annotation to rancher server ingress (since I was not able to find an existing unauthentified endpoint that serves the server version):

<http://nginx.ingress.kubernetes.io/configuration-snippet|nginx.ingress.kubernetes.io/configuration-snippet>: |
                location /rancher-version {
                    return 200 '${rancher_current_version}';
                }

And then, in my cloud-init, add the following step before starting the rancher-agent:

# Pull latest Rancher agent
rancher_current_version=$(curl -sfSL ${rancher_url}/rancher-version)
docker pull "${docker_registry_url}/rancher/rancher-agent:$rancher_current_version"

So, when the Rancher agent connects and checks if the share-mnt image is present, docker says yes and the pull is skipped: https://github.com/rancher/rancher/blob/5b56b17e2515302ad07b9e14e73024455157c546/pkg/rkeworker/docker.go#L138