I am writing my own cluster driver and am trying to find out rancher-users #general

Title

creamy-accountant-88363

03/30/2023, 5:55 PM

I am writing my own cluster driver, and am trying to find out how cloud credentials are sent to the driver. Does anyone have information around cloud credentials and cluster drivers? Thank you

stocky-account-63046

03/30/2023, 6:21 PM

The official docs are a good place to start https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permiss[…]ion/about-provisioning-drivers/manage-cluster-drivers. There's a linked example as well

I haven't played around with cluster drivers, but if they're similar to node drivers there's more credential info over at https://rancher.github.io/dashboard/code-base-works/machine-drivers#driver-binary,

creamy-accountant-88363

03/30/2023, 8:05 PM

I'm trying to find the link between creating the cloud cred on the UI, and how the driver gets the corresponding creds

it seems like it should happen on the rancher server, since it goes: 1. POST /v3/clusters to create the cluster 2. Rancher does something to add the cloud creds 3. Rancher starts the GRPC server for the driver 4. Rancher invokes the GRPC server

Looking at the built-in drivers (aks, eks, gke), they accept their various creds as driver flags

E.g., for EKS: https://github.com/rancher/rancher/blob/release/v2.7/pkg/kontainer-engine/drivers/eks/eks_driver.go#L334

From the UI side (EKS) it doesn't seem to set the accessKey/secretKey before sending the POST: https://github.com/rancher/ui/blob/master/lib/shared/addon/components/cluster-driver/driver-eks/component.js

stocky-account-63046

03/31/2023, 9:15 AM

To confirm, the correct fields are shown in the ui when creating the cloud credential associated with your cluster driver provider.... but in the POST to create the cloud credential they're missing?

creamy-accountant-88363

03/31/2023, 2:11 PM

That's right, I can set the credential via the UI, but I'm not sure how the creds are sent back to the cluster driver itself

stocky-account-63046

03/31/2023, 3:39 PM

The cluster credentials will be stored as a

<http://provisioning.cattle.io/cloud-credential|provisioning.cattle.io/cloud-credential>

resource containing a property, named after the provider, with the provider specific values in. For instance, DO cloud credentials look like...

{
  "type": "<http://provisioning.cattle.io/cloud-credential|provisioning.cattle.io/cloud-credential>",
  "metadata": {
    "generateName": "cc-",
    "namespace": "fleet-default"
  },
  "_name": "asdasd",
  "annotations": {
    "<http://provisioning.cattle.io/driver|provisioning.cattle.io/driver>": "digitalocean"
  },
  "digitaloceancredentialConfig": {
    "accessToken": "<snip>"
  },
  "_type": "<http://provisioning.cattle.io/cloud-credential|provisioning.cattle.io/cloud-credential>",
  "name": "asdasd"
}

Hopefully you should see a POST to

v2/cloudcredentials

with something like above. However.... i'm not sure how those are found and used within the cluster driver context.

From a UI perspective we supply a cloud credential id when we proxy request to the provider via rancher to fetch things like regions and machine sizes

creamy-accountant-88363

03/31/2023, 3:48 PM

Yes, I found that too. That API is capable of authorizing the user's access to a given cloud credential and querying the cloud credential secret

worst case I could add another API there that can send back the deserialized cloud credential for my ui impl

Or I could try the cloud credentials API

/v3/cloudcredentials

will give the creds, minus any "secret" data

stocky-account-63046

03/31/2023, 4:11 PM

those with the correct permissions should be able to get the associated

v3/secret

by with the same id (swapping : with /)

creamy-accountant-88363

03/31/2023, 4:46 PM

FYI I figured out how cluster drivers are getting the cloud credentials. rancher has a concept of kontainerengine drivers, and "KEv2" drivers, both of with are "cluster drivers"

KEv2 drivers are the only cluster drivers who support cloud credentials. since these are operators (with apis and such), they just lookup the cloud credentail secret by name when processing cluster operations

the management cluster validating webhook ensures a KEv2 driver does not get a cloud credential which is not associated with the creating user...

community drivers are by nature not KEv2 drivers, so they do not have cloud credential support

7 Views

#general Join Slack