i added use-forwarded-header = true in the ingress annotation , also configured custom header X-Forwarded-Proto = https in the nginx controller . still the ranche server is redirecting to https

Hello Everyone, Does anyone here has worked on the prometheus-federator project ? I have some question to ask And if there is some dedicated channels please don't hesitate to add me there ! Thank you

Hello, We have Rancher 2.5.9 with multiple RKE clusters an issue started with kubectl not responding and Rancher UI stuck In rancher Logs we see the errors below, but when we check them from the same nodes they work (connection is opened)

rancher-7859df46-c69nz rancher 2022/11/21 16:27:56 [ERROR] error syncing 'cattle-global-data/rke-cis-1.5': handler cisBenchmarkVersionHandler: cisBenchmarkVersionHandler: Sync: error creating namespace: error while creating namespace security-scan: Internal error occurred: failed calling webhook "<http://owner.namespace.capsule.clastix.io|owner.namespace.capsule.clastix.io>": Post "<https://capsule-webhook-service.liqo.svc:443/namespace-owner-reference?timeout=30s>": dial tcp 10.43.6.146:443: connect: connection refused, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:27:58 [ERROR] error syncing 'cattle-global-data/rke-cis-1.4': handler cisBenchmarkVersionHandler: cisBenchmarkVersionHandler: Sync: error creating namespace: error while creating namespace security-scan: Internal error occurred: failed calling webhook "<http://owner.namespace.capsule.clastix.io|owner.namespace.capsule.clastix.io>": Post "<https://capsule-webhook-service.liqo.svc:443/namespace-owner-reference?timeout=30s>": dial tcp 10.43.6.146:443: connect: connection refused, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:01 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:01 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:01 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:01 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:02 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:02 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:02 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:04 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:06 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:11 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:22 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

rancher-7859df46-c69nz rancher 2022/11/21 16:28:42 [ERROR] error syncing 'c-hjrkx/m-1526f20a73c5': handler rke-worker-upgrader: getNodePlan error for node [m-1526f20a73c5]: failed to find plan for 172.18.50.230, requeuing

ello everybody, We have deployed Rancher server on Kubernetes using certificates signed by our private intermediate CA, its certificate in turn being signed by our private root CA. Before deploying Rancher server, we had created the certificate secret resource for the ingress as well as the CA certificate secret resource as instructed by this guide: https://docs.ranchermanager.rancher.io/v2.5/getting-started/installation-and-upgrade/resources/update-rancher-certificate The ingress secret contains the certificate corresponding to the ingress FQDN, and the intermediate CA certificate, plus the ingress private key. The

https://<FQDN>/v3/settings/_cacerts_

Rancher server was deployed using the following command:

helm install rancher rancher-stable/rancher --namespace cattle-system \
--set hostname=<FQDN> \
--set replicas=3 \
--set ingress.tls.source=secret \
--set privateCA=true

However, when importing an existing generic Kubernetes cluster,

Hello everybody, We have deployed Rancher server on Kubernetes using certificates signed by our private intermediate CA, its certificate in turn being signed by our private root CA. Before deploying Rancher server, we had created the certificate secret resource for the ingress as well as the CA certificate secret resource as instructed by this guide: https://docs.ranchermanager.rancher.io/v2.5/getting-started/installation-and-upgrade/resources/update-rancher-certificate The ingress secret contains the certificate corresponding to the ingress FQDN, and the intermediate CA certificate, plus the ingress private key. Rancher server has been deployed using the following command:

helm install rancher rancher-stable/rancher --namespace cattle-system \
--set hostname=<FQDN> \
--set replicas=3 \
--set ingress.tls.source=secret \
--set privateCA=true

The

https://<FQDN>/v3/settings/_cacerts_

endpoint returns the same root CA with that contained in the corresponding secret. However, Rancher server seems not to have taken into account the ingress secret as its certificate has an invalid CN=dynamic, its signing CA certificate having CN=dynamiclistener-ca@1669050359 and being self signed. Therefore, the CA chain appears to be broken, and as a consequence, the import of an existing generic Kubernetes cluster, results in the infamous x509: certificate signed by unknown authority as shown below:

time="2022-11-21T18:47:26Z" level=error msg="Issuer of last certificate found in chain (CN=dynamiclistener-ca@1669050359,O=dynamiclistener-org) does not match with CA certificate Issuer (CN=...,OU=...,O=...). Please check if the configured server certificate contains all needed intermediate certificates and make sure they are in the correct order (server certificate first, intermediates after)"
time="2022-11-21T18:47:26Z" level=fatal msg="Certificate chain is not complete, please check if all needed intermediate certificates are included in the server certificate (in the correct order) and if the cacerts setting in Rancher either contains the correct CA certificate (in the case of using self signed certificates) or is empty (in the case of using a certificate signed by a recognized CA). Certificate information is displayed above. error: Get \"<https://rancher-demo.mas.local>\": x509: certificate signed by unknown authority"

Can anybody help? Have we missed anything before/during the deployment?

Hi all! So, currently, I've got a server running Docker on Ubuntu 18.04, with a bunch of docker-compose configs running, most behind Traefik. This has been a "home" lab of sorts for myself for a while. Now, I'm being asked to set up a Windows instance for Quickbooks, apparently, and so i need to create a VM, but those are harder to manage in a way that makes sense for non-technical users and i would prefer to manage containers and vms in the same place. I've heard that Rancher can do vms in docker, if so, how can I get this set up, and import my existing stuff? (Traefik is configured with Cloudflare) Noting that, we also have a second server that's yet to be set up - so I would like to connect that as well, later. -- i know that it can manage a VM host, but ideally i want to be able to have all systems be capable of both vms and containers seamlessly

Hi All, I just installed rancher on k3d/windows along with Kafka from the Apps/Charts. I'm struggling to access it from outside the cluster, can anyone drive me on the right direction? (I googled it but did not find any example)

Is there a kubectl / rancher-cli equivalent of selecting a deployment and clicking "redeploy"? All I can find is old scripts in go/php/etc.

And yeah, I don't want to have to change the yamls every single time, I simply want it to pull latest -latest and recreate the pods (exactly what happens when I use "Redeploy" button")

When will Rancher support kubernetes version 1.25+ ?

is upgrading the local k3s not viable?

Hi Team ! I m using Rancher v2.6.9 for creating my EKS cluster. I m having an hard time to create a managed node group In just one AZ. It seems the requirement is to have at least 2 AZ for the ASG. I change the config of the ASG in AWS to spawn the nodes in just one AZ but I have this issue now

[Syncing error] error for cluster [c-vbbr]: health error for node group [test3] in cluster [c-vbbr]: The Amazon AutoScalingGroup eks-test3-xxxxxxxxxx has subnets ([subnet-xxxxx]) which is not expected by Amazon EKS. Expected subnets : ([subnet-xxxxxx, subnet-xxxxx, subnet-xxxx]): node group cannot be updated, must be deleted and recreated

Any idea please ? Is this possible ?

Hi Team! I am currently facing issues with nodes running into PLEG errors after a scheduled node restart. Does anyone know if Rancher does anything particular with the PLEG lifecycle that could be affecting this? The nodes just eventually start timing out on all their health checks which eventually causes the node to be marked as failing.

Hi. I accidentally terminated all nodes (masters and workers) of a cluster in Rancher. All but the last master went away. But I cannot get rid of that one, it is stuck there in Removing state, since some hours. How can I recover from that state? My plan was to start a new master and populate its

etcd

from a snapshot of that cluster. But new nodes won't come up and stay in Registering state, probably trying to join any other master nodes that don't exist anymore.

Cannot import AKS to rancher via rancher cli and also using rancher2 terraform

FATA[0001] Bad response statusCode [403]. Status [403 Forbidden]. Body: [baseType=error, code=Forbidden, message=<http://clusters.management.cattle.io|clusters.management.cattle.io> "test" is forbidden: User "u-v8qr9" cannot get resource "clusters" in API group "<http://management.cattle.io|management.cattle.io>" at the cluster scope: Azure does not have opinion for this non AAD user. If you are an AAD user, please set Extra:oid parameter for impersonated user in the kubeconfig] from [<https://rancher/v3/clusters/test>]

Can someone help with this?

Hey guys! I am trying to push docker image to AWS ECR.

Where should I be looking to troubleshoot if I am unable to use kubectl to view anything in a downstream managed Rancher environment? Every request I send via kubectl comes back with:

I1122 14:06:43.519461   42615 helpers.go:264] Connection error: Get <https://my.rancher.dev/k8s/clusters/c-spjq6/api/v1/namespaces/default/pods?limit=500>: net/http: TLS handshake timeout
Unable to connect to the server: net/http: TLS handshake timeout

increasing to

--v=9

doesn’t really show any more info on why exactly this call is failing….

I can browse to that rancher URL via the UI just fine, no cert errors, etc

but both the local management cluster and a rancher launched RKE cluster both have the issue, but both also work just fine from the UI

Hi, Is there a way to hide the cluster management tab on rancher? I am trying to understand how to limit which users can see this option.

Hello

Hello, Is it possible to download each k8s cluster's user kubeconfig file from Rancher using API or Rancher CLI?

Hi has anyone faced this before ? achanpur@ankit-mac ~ % /Applications/Rancher\ Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl start --tty=false 0 INFO[0000] Creating an instance “0” from template://default (Not from <template://0>) WARN[0000] This form is deprecated. Use

limactl start --name=0 <template://default>

instead FATA[0000] open /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/share/lima/examples/default.yaml: no such file or directory

Did anyone try to use the websocket from the API to connect to the shell of a docker? It seems possible from the code and API spec, but there's no documentation on how to pass the auth tokens in the WS

Hello, Did anyone manage to use the tls=external option when deploying a rancher through Helm in an HA architecture ? When I try to do so, the ingress correctly exposes the service from port 80 and the 3 rancher pods start up but they fail to communicate between each other with this error : "[ERROR] Failed to connect to peer wss://10.42.2.19/v3/*connect* [local ID=10.42.0.22]: websocket: bad handshake" I tried this on 2.7.0 or 2.6.9, with the compatible RKE version and had no luck with both of them. A cert-manager config with self-signed certificates works fine in the same environment but unfortunately that's not what I would like to do.

Hey, we are using the new monitoring with an fresh RKE2 cluster. When we open Grafana we cant see any external nginx traffic, but only the internal traffic. Any config options we forgot? Tried using these boards.

is there a way to run rancher cluster agents in tls verify disabled mode?

Same error here.

Is it possible to give nodes Internal IPs after the cluster has been created?