This message was deleted.

Hi, did you managed to make the ACE work? How did you configured the part “load-balancer with SSL termination on the load balancer”?

Hi @elegant-candle-74808 You will find all the details here: https://forums.rancher.com/t/authorized-cluster-endpoint-unable-to-access-from-outside-of-the-control-plane-node/43349/10?u=jaepetto Basically, I did setup the load balancer before setting the cluster. Then I reinstalled the cluster with ACE activated. There’s apparently a bug where ACE does not work if it is activated after the initial setup. The HAProxy configuration is straightforward. The configuration can be found here: https://forums.rancher.com/t/authorized-cluster-endpoint-unable-to-access-from-outside-of-the-control-plane-node/43349/10?u=jaepetto I’m now struggling with the authentication / authorisation part. But that’s another story. I’m trying to remove the dependency on Rancher to manage authorisations but I can’t figure out how: https://forums.rancher.com/t/is-there-a-way-to-bypass-rancher-authentication-on-a-downstream-cluster/43422?u=jaepetto cheers.

Thank you very much for the fast and detailed answer!!! Unfortunatelly the clusters for which I want to enable the ACE are already in use so not sure I will be able to reinstall them (I’m looking at the solutions described i the links you kindly provided)

I feel your pain!

Dear Emmanuel, disturbing you again 🙂, maybe you know what cert I need when encountering the following:

E0626 01:20:10.808349   53803 memcache.go:265] couldn't get current server API group list: Get "https://<fqdn>:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate signed by unknown authority

I feel I’m near…. 🙂 but don’t know what should I use here. In the cluster ACE configuration I used the fqdn but i didn’t specified anything in the field “CA certificate” (Certificates required for the client to successfully verify the validity of the certificate returned by the endpoint”)

I’m not completely at ease with TLS… so I might not give you a helpful answer. Did you try to connect using openssl?

openssl s_client -connect <fqdn>:6443

This should give you all the details concerning the certificate chain that is currently used and point out which certificate is not trusted.

gives the same error>

---

SSL handshake has read 1515 bytes and written 437 bytes Verification error: self-signed certificate in certificate chain ---

I guess you should put this self-signed certificate into the “CA certificate” field on the ACE configuration. But again, this is just a guess.

Maybe I used the wrong certificate, because I did tried it few days ago but was not workig. I’ll investigate more in this direction 🙂

Enjoy your investigation

🙂

Good news! I found a fix for that… 🙂 There something slightly misleading in the Rancher concerning ACE. You should probably specify the port where the API is listening. I our case, I ended up putting the value

<load_balancer_fqdn>:6443

in the fqdn field. Then I made sure that our load-balancer is configured as follow: port 6443 sends traffic to port 6443 of the control-plane nodes port 443 sends traffic to port 443 of the worker nodes I hope this helps.

ah… I missed that part, I just added it in the downloaded kubeconfig

it should.

I know it should 😞 I’m deploying a tes rke2 to see how it is working if “ACE is enabled since the creation”

Cheer up! I’m sure you’ll find

So what I think might be happening here, is that your authorized endpoint kubectl config, does not contain a valid certificate. There needs to be a certificate there, in order to be able to connect with the the k8s api endpoint. If you examine your kubectl config, you should see 2 contexts. One is for the cluster connect through Rancher Server.. the other is the contact that connects directly to the cluster.. the 'authorized endpoint' . Both of these context should contain a certificate. What appears to be the case, is that if you do not supply the Authorized Endpoint with a CA certificate for the k8s endpoint, it won't populate the kubectl config context with one. If your RKE2 Cluster certificates are self-signed (which is the default), then the k8s kube-api endpoint certificate is signed using a self-signed CA certificate, generated by RKE2 at cluster creation. This is not documented well, but what you need to do, is add the public part of this self-signed CA, for your cluster, to the CA field of the Authorized Endpoint settings. That will cause it to be populated in the kubectl config. And then, you should be able to connect. You can grap the RKE2 self-signed CA from a control plane node, either through your browser (jut brows to the k8s api endpoint on 6443) and then in your browser you should see a certificate chain.. the endpoint itself.. and its own certificate. But if you go one step 'up' in the chain, you can download the RKE2 CA certificate, and that is the one you need. You can also find all these certificates locally on the control-plane nodes, under /var/lib/rancher/rke2/server/tls. More info: https://docs.rke2.io/security/certificates