https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# rke2
a
This message was deleted.
c
you can use ECR from anywhere, sure
but if you’re not on EC2 an don’t have IAM role credentials then you need to figure out how to authenticate to it
which means sticking AWS credentials somewhere on each node
e
I see. Would you say its easier to use something like k8s-ecr-login-review? https://github.com/nabsul/k8s-ecr-login-renew
c
no, I’d probably use the ecr kubelet credential provider
idk where to get binaries for that though
e
Ok. Thank you. I will have a look then,
c
e
Copy code
The Amazon EKS Anywhere Curated Packages are only available to customers with the Amazon EKS Anywhere Enterprise Subscription.
So that means I need license to use it right?
The problem which I face here is that the registry is on AWS ECR. We use AWS Organisations so providing the keys is not enough as token is passed back and its valid only for 12h AFAIK. We dont have EKS Anywhere license. I will try the kubelet-credential-provider
c
that is the kubelet credential provider for ECR
ask your AWS support contact if they make the binaries available anywhere else
if not you can always build it yourself from the cloud-provider-aws repo
e
Ok. I will. Thanks
c
that is what you need to turn AWS creds into an ECR auth token that the kubelet can use to pull images from ECR, regardless of where you are getting your AWS creds from.
a
You may be able to utilize external-secrets deployed to your rke2 cluster and use ECR creds via external-secrets. You can create your creds in AWS Secret Manager then deploy External Secrets Operator and create a Cluster SecretStore and ExternalSecret that fetches the creds from AWS and creates a new secret for you that you will put as your ImagePullSecret.
e
@adamant-portugal-53342 yes, I know what you talking about as I use it in AWS. Unfortunately, the project I got now is cloud agnostic with possibility of air-gapped env. I know, I should not use ECR but spinning my own docker registry is another brick to the pile of bricks I already have 😁
c
do you actually need to push images to it, or do you just need somewhere to load core images from?
a
If your using rancher on top of k3s you should do the registry mirror option during setup can’t remember the option off the top of my head for airgapping and utilizing the bundles
e
All the images are there including the core images as we use docker caching on ECR. I just need RKE2 to be able to pull the images from there.
I have a problem with the credentials though. I put the creds in /home/ubuntu/.aws/config but it doesn’t work. Getting errors about auth to ECR. Where should I put the creds then?
The creds are as well in /root/.aws/config but the same issue. Auth errors
The creds are correct as already checked.
c
where do the docs for the helper say to put them?
Have you tried setting the AWS_CONFIG_FILE and AWS_PROFILE env vars to point at the config file and profile in the plugin config?
I am assuming that you are familiar with aws cli configs and profiles
Are you sure the provider bin is in the correct location and the config file has the correct contents?
e
Thanks @creamy-pencil-82913 - All good - works. Needed to reboot the VM and all works. For those who gets here as well - working config and steps: 1 - download the 
ecr-credential-provider
from google cloud 2 - add credentials-provider-config.yaml with the following:
Copy code
apiVersion: <http://kubelet.config.k8s.io/v1|kubelet.config.k8s.io/v1>
kind: CredentialProviderConfig
providers:
  - name: ecr-credential-provider
    matchImages:
      - "*.dkr.ecr.*.<http://amazonaws.com|amazonaws.com>"
    defaultCacheDuration: "12h"
    apiVersion: <http://credentialprovider.kubelet.k8s.io/v1|credentialprovider.kubelet.k8s.io/v1>
3 - add creds to /home/ubuntu/.aws/config and/or /root/.aws/config (to confirm as not sure which actually works here
Copy code
[default]
aws_access_key_id =
aws_secret_access_key =
4 - add the following args to kubelet
Copy code
image-credential-provider-config=/home/ubuntu/credential_provider_config.yaml
image-credential-provider-bin-dir=/home/ubuntu/ecr-credential-provider
5 - for troubleshooting add the following to kubelet arg
Copy code
logtostderr=false
log-file=/var/log/kubelet.log
6 - IMPORTANT - there should be only one file in 
home/ubuntu/ecr-credential-provider
and that is the 
ecr-credential-provider
. The name MUST be exactly that and it should be exec so 
chmod +x home/ubuntu/ecr-credential-provider
Many thanks for your help guys.
c
why are you putting all this stuff in /home/ubuntu? Not really a great idea to put system-level stuff in user home dirs.
RKE2 already has expected paths for this and configures them for you if you put the files in the correct location
You should put them where they are expected to be
https://docs.rke2.io/reference/linux_agent_config#node
image-credential-provider-bin-dir The path to the directory where credential provider plugin binaries are located “/var/lib/rancher/credentialprovider/bin”
image-credential-provider-config The path to the credential provider plugin config file “/var/lib/rancher/credentialprovider/config.yaml”
e
Thanks @creamy-pencil-82913 - I will move the files as suggested.
60 Views
Open in Slack
PreviousNext
Powered by