Hi everyone, I've been able to spin up eks, gke clusters and able to import the existing ones too on rancher. I would like to know if it is possible or has anyone used gp3 volume using CSI driver in rancher for eks cluster? Has anyone tried it in Rancher if yes then how?

Is there any instructions for handling a kubernetes version upgrade if I did not use RKE? I have an eks cluster on AWS thats running 1.19 and want to upgrade to 1.20 or 1.21.

Trying to install 2.6.4 on New EKS cluster. I have installed the nginx-controller. One of the pods is throwing this error:

[ERROR] error syncing 'cattle-fleet-system/helm-operation-gg8n6': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-gg8n6), requeuing

Another pod is showing this error:

[ERROR] error syncing 'rancher-rke2-charts': handler helm-clusterrepo-ensure: git -C /var/lib/rancher-data/local-catalogs/v2/rancher-rke2-charts/675f1b63a0a83905972dcab2794479ed599a6f41b86cd6193d69472d0fa889c9 fetch origin 107ecb60ed851ebd2b4a54a0859c8d8edc046c9c error: exit status 128, detail: error: Server does not allow request for unadvertised object 107ecb60ed851ebd2b4a54a0859c8d8edc046c9c
, requeuing

And there is also this error:

2022/05/03 16:55:54 [ERROR] Failed to connect to peer <wss://10.50.224.213/v3/connect> [local ID=10.50.193.244]: EOF
2022/05/03 16:55:54 [ERROR] Failed to connect to peer <wss://10.50.215.200/v3/connect> [local ID=10.50.193.244]: EOF

The rancher documentation says to use this command to create the ingress controller. But it creates a classic lb. How to fix? Plus I don’t want a public load balancer. I want a private NLB

helm upgrade --install \
  ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --set controller.service.type=LoadBalancer \
  --version 3.12.0 \
  --create-namespace

Hi all. I'm looking at importing existing EKS clusters into rancher (2.6.3) and this page https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/registered-clusters/ describes the process and requirements including cluster rbac. What it doesn't say is what IAM policy is needed for the import. The EKS cluster creation doc specified an IAM policy but it has way too many permissions for existing cluster. Does anyone have details of the policy required?

Hi,Please tell me, How to connect Ingress. 1. installed rancher 2.6. 2. Apped new cluster EKS. 3. Create Deployment and Ingress etc 4. setting ALB and Target group. But 502 Bad Gateway Do I need ingress-nginx ?? Or is there any runbook?

Is Kubernetes version 1.22 on EKS for downstream clusters supported in Rancher 2.6.5?

Hi all. I have created downstream cluster ( EKS provider v1.21, Rancher local cluster v2.6.3). For some reason it stucks on

Waiting for API to be available

, in conditions there are

Cluster agent is not connected

event. At the same time AWS console shows that EKS cluster is in

Active

, so basically it means that EKS cluster is running and ready to use. What can the reason of this issue? I understand that

Waiting for API to be available

is to generic issue, but I have no idea where I need to look for the problem. The interesting thing that there is no rancher agents on any of worker nodes. Rancher is accessible from downstream nodes, so connection shouldn’t be an issue here.

Hi all, I'm after launching a HA Rancher Server setup, I'm using the UI to create a private EKS cluster. The rancher server and EKS cluster are in separate VPC's but with VPC peering, so there is no vpc connectivity issue. However, the cluster is not able to be created. I get the following message:

Failed to communicate with cluster: Post "<https://XXXXXX.gr7.us-east-1.eks.amazonaws.com/api/v1/namespaces>": cluster agent disconnected

On the private cluster there are no rancher pods running. So I went through the instructions to run a command manually to install cattle-cluster-agent, but it is also stuck at the following:

time="2022-05-17T16:59:31Z" level=info msg="Connecting to proxy" url="<wss://xxx.com/v3/connect/register>"

Also, this exact set of steps, but creating an EKS cluster with a public endpoint, works perfectly fine, I don't even need the vpc peering. Any insight as to why the rancher server and this cluster cannot communicate would be greatly appreciated.

Hello, I just installed a brand new Rancher 2.6.5 on an EKS 1.22 cluster. When I go to the import cluster screen and attempt to import / register a EKS cluster, I see boxes to provide cloud credentials and I don't understand why. When I go to the "import generic cluster" option instead, I see the appropriate "agent environment var" settings and can click create to get the correct kubectl commands to run on one of my managed nodes and get it imported / registered. Is that the way it should be? I would have expected the EKS option to be worded the same way as the generic cluster option.

Does anyone know what IAM permissions are needed to import an existing eks cluster? I've given accesskubernetesapi and list,describe cluster. But I'm getting “ClusterUnavailable 503” errors from the eks-operator pod. along with regular “unauthorised, requeuing” errors every 30 seconds.

I've playing with it for few hours now to no avail. I'm sure, that something is missing, but i have no clue what

Has anyone been able to update the max pods limit when using EKS and the VPC-CNI Add-On?

Hi, I have a few rancher clusters deployed on EC2 (RKE1) and now I'm working towards implementing a VPN in my company (Pritunl) and I was looking to restrict access to rancher nodes from outside, looking into the

rancher-nodes

security group, I see a few rules that give access to

0.0.0.0/0

TCP 30000 - 32767 TCP 80 TCP 443 TCP 6443 TCP 2376 Are those rule safe to modify with my VPN server IP ? or are there more steps to it? thanks

is there an ingress that bind the cert to the service?

I updated one of our sandboxs from 1.21 to 1.22 and then 1.23 via the EKS UI now seeing this in Rancher “InvalidParameterException: Unsupported Kubernetes minor version update from 1.23 to 1.22” I don’t see it for another sandbox that I upgraded directly from 1.22 to 1.23. I see a ton of matches on Google for the above (but with old versions of k8s) and don’t see any fixes proposed

verified the node group did get updated as well

2022/08/12 201350 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 2022/08/12 201350 [INFO] Watching metadata for management.cattle.io/v3, Kind=Node 2022/08/12 201350 [INFO] Watching metadata for apps/v1, Kind=StatefulSet 2022/08/12 201353 [INFO] Adding peer wss://192.68.101.82/v3/connect, 192.68.101.82 2022/08/12 201353 [ERROR] Failed to connect to peer wss://192.68.101.82/v3/connect [local ID=192.68.102.155]: websocket: bad handshake 2022/08/12 203803 [ERROR] Error during subscribe websocket: close sent 2022/08/12 203803 [ERROR] Error during subscribe websocket: close sent Is anyone faced this kind of websocket close connection in rancher containers ?

Do you have old rancher stuff ?

Hello team - we are using rancher

v1.6.30

and am looking at configuring a few

sysctl

params for our docker containers, what is the best way to do this in the legacy rancher? any clues/ideas?

Hello, I m using AWS EKS with Classic Load Balancer, Got this kind of issue for web-socket. I really appreciate any help on this.

websocket: close 1006 (abnormal closure): unexpected EOF

That Websocket error is from rancher containers.

When we need to use these annotation ? I have some issues with web-sockets. When i check proxy server logs, i got "*Error* getting SSL certificate "default/*-tls": local SSL certificate default/*-tls was not found. Using default certificate" and it keeps updating ingress to classic load balancer and got disconnected every 1mins after upgrading rancher version.

annotations:
    <http://field.cattle.io/projectId|field.cattle.io/projectId>: ""

Updating local copy of SSL certificate to classic load balancer every 1 mins. How to troubleshoot these issues ? Nginx-ingress logs: 8 controller.go:177] Configuration changes detected, backend reload required. 8 backend_ssl.go:189] Updating local copy of SSL certificate "cattle-system/tls-rancher-ingress" with missing intermediate CA certs I0830 050732.859819 8 controller.go:195] Backend successfully reloaded.A Any ideas for SSL Uploading every 1 mins ? Many thanks

Using EKS to deploy cluster. I want to increase the max pods. I've ensured that the CNI plugin is enabled but I can't get the launch template to correctly updated to add

--max-pods=110

to the kubelet-extra-args. Anyone know how to make that work? I've tried adding it as user data but it seems to be ignored

Rancher version: 2.6.8
Installation method: Helm
Helm repo: <https://releases.rancher.com/server-charts/stable>

After installing rancher, I tried to bootstrap it, using rancher terraform provider and next tf configuration:

resource "rancher2_bootstrap" "admin" {
  provider         = rancher2.bootstrap
  initial_password = "Password1"
  password         = local.rancher_bootstrap_password
  telemetry        = false
}

For some reason after few minutes, creating of this process fails with the following error log:

│ **Error:** **[ERROR] Updating token: Bad response statusCode [403]. Status [403 Forbidden]. Body: [baseType=error, code=Forbidden, message=<http://settings.management.cattle.io|settings.management.cattle.io> "k8s-version" is forbidden: User "user-bfkj6" cannot get resource "settings" in API group "<http://management.cattle.io|management.cattle.io>" at the cluster scope] from [<https://rancher-internal.foo.bar.com/v3/settings/k8s-version]**>

│

│ with rancher2_bootstrap.admin,

│ on <http://main.tf|main.tf> line 48, in resource "rancher2_bootstrap" "admin":

│ 48: resource "rancher2_bootstrap" "admin" {

That user has GlobalBindingRole to admin role, so it should has access to Setting CRD. Also there are a lot of such error messages in rancher pod:

2022/09/09 09:07:41 [ERROR] Failed to connect to peer <wss://10.0.3.184/v3/connect> [local ID=10.0.1.59]: websocket: bad handshake

I’ve also tried to bootstrap rancher via UI, but after entering bootstrap password the only thing I see is the white screen. Any ideas? (edited)

I have Rancher 2.6.1 running on EKS version 1.20. (Both Server and Production Cluster) I want to upgrade both of these. All nodes in the server cluster are running Amazon Linux, and the downstream cluster nodes are using Ubuntu. For the rancher server cluster I launched it with EKS (eksctl) and did not use a custom AMI. The production cluster I created a launch template with a custom AMI and created the cluster with Rancher console. I am looking for advice on the best way to upgrade. I think I can do the following: • Upgrade Rancher to 2.6.8 (can I go directly or do need to do it in steps?) • Upgrade server cluster to 1.21 and then to 1.22 Using AWS Console Upgrade Downstream Cluster would be a multi stage process. • I have to create a new AMI using the correct base AMI + my customizations.. • Upgrade the cluster using the AWS console to 1.21 • Re-launch all nodes with the matching AMI • Upgrade the Cluster using the AWS console to 1.22 • Re-launch all nodes using the matching AMI Do I have it right?

I have a downstream cluster that I created using rancher console. I recently deleted a node group, but I did it from the AWS Console. Don’t know if it matters, but I also added a node group to that cluster using eksctl instead of rancher. Now that cluster is showing this error: _*InvalidParameterException: You cannot specify an AMI Type other than CUSTOM, when specifying an image id in your launch template. { RespMetadata: { StatusCode: 400, RequestID: “c5a5bfb7-70ae-476f-8548-94858b1aa765” }, ClusterName: “pano-prod”, Message_: “You cannot specify an AMI Type other than CUSTOM, when specifying an image id in your launch template.“, NodegroupName: “pool-pvt” }*_

Hi, I want to install rancher on AWS EKS cluster but with istio setup(ingress/egress). I see installation instructions wants nginx-ingress, which I dont want to use for my use case. Any reference document of setup please

Hi I'm new, I installed rancher on my aws eks cluster, when I tried accessing my ingress load balancer url, its returning 504 gateway timeout, pls can anyone help out. thanks

Has anyone successfully imported EKS Clusters v1.23.0 into Rancher 2.6. I have multiple EKS Clusters running in my AWS Account, which I'm trying to import to Rancher 2.6 (running on another EKS and VPC) but I'm getting

Waiting for API to be available

error. I have completed all networking between both clusters, not sure what's happening. Any guidance would be greatly appreciated.

Hi. I’m trying to create downstream EKS cluster using Rancher terraform provider. For some reason EKS nodes created by Rancher have only default EKS security group even though we provided list of additional security groups. These SGs are present in EKS configuration, but nodes still don’t have them. The strangest thing is that during creating EKS cluster, you can see in node group config that they will use auto-generated SG by Rancher, but after cluster became to Active state, config has been changed and there are now list of security groups we specified. We do not have custom launch template, nodes launched from LT created by Rancher as well. And that LT for some reason contains only the default EKS node group and no instance type specified. I thought it may be issue in 2.6.4 version, but after upgrading to 2.6.9 issue still here