https://rancher.com/ logo
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
amazon
  • f

    future-truck-59205

    04/28/2022, 6:01 PM
    Hi everyone, I've been able to spin up eks, gke clusters and able to import the existing ones too on rancher. I would like to know if it is possible or has anyone used gp3 volume using CSI driver in rancher for eks cluster? Has anyone tried it in Rancher if yes then how?
    m
    • 2
    • 3
  • c

    curved-lifeguard-39360

    04/28/2022, 7:34 PM
    Is there any instructions for handling a kubernetes version upgrade if I did not use RKE? I have an eks cluster on AWS thats running 1.19 and want to upgrade to 1.20 or 1.21.
    m
    r
    • 3
    • 6
  • c

    curved-lifeguard-39360

    05/03/2022, 5:37 PM
    Trying to install 2.6.4 on New EKS cluster. I have installed the nginx-controller. One of the pods is throwing this error:
    [ERROR] error syncing 'cattle-fleet-system/helm-operation-gg8n6': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-gg8n6), requeuing
    Another pod is showing this error:
    [ERROR] error syncing 'rancher-rke2-charts': handler helm-clusterrepo-ensure: git -C /var/lib/rancher-data/local-catalogs/v2/rancher-rke2-charts/675f1b63a0a83905972dcab2794479ed599a6f41b86cd6193d69472d0fa889c9 fetch origin 107ecb60ed851ebd2b4a54a0859c8d8edc046c9c error: exit status 128, detail: error: Server does not allow request for unadvertised object 107ecb60ed851ebd2b4a54a0859c8d8edc046c9c
    , requeuing
    And there is also this error:
    2022/05/03 16:55:54 [ERROR] Failed to connect to peer <wss://10.50.224.213/v3/connect> [local ID=10.50.193.244]: EOF
    2022/05/03 16:55:54 [ERROR] Failed to connect to peer <wss://10.50.215.200/v3/connect> [local ID=10.50.193.244]: EOF
    a
    • 2
    • 8
  • c

    curved-lifeguard-39360

    05/03/2022, 8:35 PM
    The rancher documentation says to use this command to create the ingress controller. But it creates a classic lb. How to fix? Plus I don’t want a public load balancer. I want a private NLB
    helm upgrade --install \
      ingress-nginx ingress-nginx/ingress-nginx \
      --namespace ingress-nginx \
      --set controller.service.type=LoadBalancer \
      --version 3.12.0 \
      --create-namespace
    • 1
    • 1
  • b

    boundless-dog-9864

    05/04/2022, 8:39 PM
    Hi all. I'm looking at importing existing EKS clusters into rancher (2.6.3) and this page https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/registered-clusters/ describes the process and requirements including cluster rbac. What it doesn't say is what IAM policy is needed for the import. The EKS cluster creation doc specified an IAM policy but it has way too many permissions for existing cluster. Does anyone have details of the policy required?
    c
    • 2
    • 1
  • a

    adorable-ghost-62021

    05/10/2022, 8:03 AM
    Hi,Please tell me, How to connect Ingress. 1. installed rancher 2.6. 2. Apped new cluster EKS. 3. Create Deployment and Ingress etc 4. setting ALB and Target group. But 502 Bad Gateway Do I need ingress-nginx ?? Or is there any runbook?
    b
    • 2
    • 14
  • r

    rich-address-13891

    05/13/2022, 9:22 AM
    Is Kubernetes version 1.22 on EKS for downstream clusters supported in Rancher 2.6.5?
    c
    • 2
    • 16
  • w

    worried-rain-56725

    05/13/2022, 6:28 PM
    Hi all. I have created downstream cluster ( EKS provider v1.21, Rancher local cluster v2.6.3). For some reason it stucks on
    Waiting for API to be available
    , in conditions there are
    Cluster agent is not connected
    event. At the same time AWS console shows that EKS cluster is in
    Active
    , so basically it means that EKS cluster is running and ready to use. What can the reason of this issue? I understand that
    Waiting for API to be available
    is to generic issue, but I have no idea where I need to look for the problem. The interesting thing that there is no rancher agents on any of worker nodes. Rancher is accessible from downstream nodes, so connection shouldn’t be an issue here.
    c
    • 2
    • 4
  • l

    little-arm-91919

    05/17/2022, 5:51 PM
    Hi all, I'm after launching a HA Rancher Server setup, I'm using the UI to create a private EKS cluster. The rancher server and EKS cluster are in separate VPC's but with VPC peering, so there is no vpc connectivity issue. However, the cluster is not able to be created. I get the following message:
    Failed to communicate with cluster: Post "<https://XXXXXX.gr7.us-east-1.eks.amazonaws.com/api/v1/namespaces>": cluster agent disconnected
    On the private cluster there are no rancher pods running. So I went through the instructions to run a command manually to install cattle-cluster-agent, but it is also stuck at the following:
    time="2022-05-17T16:59:31Z" level=info msg="Connecting to proxy" url="<wss://xxx.com/v3/connect/register>"
    Also, this exact set of steps, but creating an EKS cluster with a public endpoint, works perfectly fine, I don't even need the vpc peering. Any insight as to why the rancher server and this cluster cannot communicate would be greatly appreciated.
    👍 1
    a
    • 2
    • 2
  • c

    clean-painting-58815

    06/28/2022, 9:32 PM
    Hello, I just installed a brand new Rancher 2.6.5 on an EKS 1.22 cluster. When I go to the import cluster screen and attempt to import / register a EKS cluster, I see boxes to provide cloud credentials and I don't understand why. When I go to the "import generic cluster" option instead, I see the appropriate "agent environment var" settings and can click create to get the correct kubectl commands to run on one of my managed nodes and get it imported / registered. Is that the way it should be? I would have expected the EKS option to be worded the same way as the generic cluster option.
    a
    • 2
    • 4
  • b

    boundless-dog-9864

    07/11/2022, 5:31 PM
    Does anyone know what IAM permissions are needed to import an existing eks cluster? I've given accesskubernetesapi and list,describe cluster. But I'm getting “ClusterUnavailable 503” errors from the eks-operator pod. along with regular “unauthorised, requeuing” errors every 30 seconds.
    c
    • 2
    • 3
  • a

    ambitious-motherboard-40337

    07/18/2022, 2:08 PM
    I've playing with it for few hours now to no avail. I'm sure, that something is missing, but i have no clue what
    a
    • 2
    • 11
  • o

    orange-barista-66154

    08/10/2022, 10:03 AM
    Has anyone been able to update the max pods limit when using EKS and the VPC-CNI Add-On?
    • 1
    • 1
  • a

    ancient-energy-15842

    08/10/2022, 6:37 PM
    Hi, I have a few rancher clusters deployed on EC2 (RKE1) and now I'm working towards implementing a VPN in my company (Pritunl) and I was looking to restrict access to rancher nodes from outside, looking into the
    rancher-nodes
    security group, I see a few rules that give access to
    0.0.0.0/0
    TCP 30000 - 32767 TCP 80 TCP 443 TCP 6443 TCP 2376 Are those rule safe to modify with my VPN server IP ? or are there more steps to it? thanks
  • a

    ambitious-motherboard-40337

    08/11/2022, 9:02 AM
    is there an ingress that bind the cert to the service?
    • 1
    • 1
  • h

    helpful-ability-85939

    08/12/2022, 11:54 PM
    I updated one of our sandboxs from 1.21 to 1.22 and then 1.23 via the EKS UI now seeing this in Rancher “InvalidParameterException: Unsupported Kubernetes minor version update from 1.23 to 1.22” I don’t see it for another sandbox that I upgraded directly from 1.22 to 1.23. I see a ton of matches on Google for the above (but with old versions of k8s) and don’t see any fixes proposed
  • h

    helpful-ability-85939

    08/12/2022, 11:54 PM
    verified the node group did get updated as well
  • c

    careful-optician-75900

    08/18/2022, 3:05 AM
    2022/08/12 20:13:50 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 2022/08/12 20:13:50 [INFO] Watching metadata for management.cattle.io/v3, Kind=Node 2022/08/12 20:13:50 [INFO] Watching metadata for apps/v1, Kind=StatefulSet 2022/08/12 20:13:53 [INFO] Adding peer wss://192.68.101.82/v3/connect, 192.68.101.82 2022/08/12 20:13:53 [ERROR] Failed to connect to peer wss://192.68.101.82/v3/connect [local ID=192.68.102.155]: websocket: bad handshake 2022/08/12 20:38:03 [ERROR] Error during subscribe websocket: close sent 2022/08/12 20:38:03 [ERROR] Error during subscribe websocket: close sent Is anyone faced this kind of websocket close connection in rancher containers ?
  • c

    careful-optician-75900

    08/18/2022, 3:24 AM
    Do you have old rancher stuff ?
    f
    • 2
    • 2
  • c

    clever-napkin-33058

    08/25/2022, 6:40 PM
    Hello team - we are using rancher
    v1.6.30
    and am looking at configuring a few
    sysctl
    params for our docker containers, what is the best way to do this in the legacy rancher? any clues/ideas?
  • c

    careful-optician-75900

    08/29/2022, 2:27 PM
    Hello, I m using AWS EKS with Classic Load Balancer, Got this kind of issue for web-socket. I really appreciate any help on this.
    websocket: close 1006 (abnormal closure): unexpected EOF
    That Websocket error is from rancher containers.
  • c

    careful-optician-75900

    09/02/2022, 9:45 AM
    When we need to use these annotation ? I have some issues with web-sockets. When i check proxy server logs, i got "*Error* getting SSL certificate "default/*-tls": local SSL certificate default/*-tls was not found. Using default certificate" and it keeps updating ingress to classic load balancer and got disconnected every 1mins after upgrading rancher version.
    annotations:
        <http://field.cattle.io/projectId|field.cattle.io/projectId>: ""
    Updating local copy of SSL certificate to classic load balancer every 1 mins. How to troubleshoot these issues ? Nginx-ingress logs: 8 controller.go:177] Configuration changes detected, backend reload required. 8 backend_ssl.go:189] Updating local copy of SSL certificate "cattle-system/tls-rancher-ingress" with missing intermediate CA certs I0830 05:07:32.859819 8 controller.go:195] Backend successfully reloaded.A Any ideas for SSL Uploading every 1 mins ? Many thanks
  • o

    orange-barista-66154

    09/08/2022, 3:59 PM
    Using EKS to deploy cluster. I want to increase the max pods. I've ensured that the CNI plugin is enabled but I can't get the launch template to correctly updated to add
    --max-pods=110
    to the kubelet-extra-args. Anyone know how to make that work? I've tried adding it as user data but it seems to be ignored
  • w

    worried-rain-56725

    09/09/2022, 10:12 AM
    Rancher version: 2.6.8
    Installation method: Helm
    Helm repo: <https://releases.rancher.com/server-charts/stable>
    After installing rancher, I tried to bootstrap it, using rancher terraform provider and next tf configuration:
    resource "rancher2_bootstrap" "admin" {
      provider         = rancher2.bootstrap
      initial_password = "Password1"
      password         = local.rancher_bootstrap_password
      telemetry        = false
    }
    For some reason after few minutes, creating of this process fails with the following error log:
    │ **Error:** **[ERROR] Updating token: Bad response statusCode [403]. Status [403 Forbidden]. Body: [baseType=error, code=Forbidden, message=<http://settings.management.cattle.io|settings.management.cattle.io> "k8s-version" is forbidden: User "user-bfkj6" cannot get resource "settings" in API group "<http://management.cattle.io|management.cattle.io>" at the cluster scope] from [<https://rancher-internal.foo.bar.com/v3/settings/k8s-version]**>
    
    │
    
    │ with rancher2_bootstrap.admin,
    
    │ on <http://main.tf|main.tf> line 48, in resource "rancher2_bootstrap" "admin":
    
    │ 48: resource "rancher2_bootstrap" "admin" {
    That user has GlobalBindingRole to admin role, so it should has access to Setting CRD. Also there are a lot of such error messages in rancher pod:
    2022/09/09 09:07:41 [ERROR] Failed to connect to peer <wss://10.0.3.184/v3/connect> [local ID=10.0.1.59]: websocket: bad handshake
    I’ve also tried to bootstrap rancher via UI, but after entering bootstrap password the only thing I see is the white screen. Any ideas? (edited)
  • c

    curved-lifeguard-39360

    09/14/2022, 8:19 PM
    I have Rancher 2.6.1 running on EKS version 1.20. (Both Server and Production Cluster) I want to upgrade both of these. All nodes in the server cluster are running Amazon Linux, and the downstream cluster nodes are using Ubuntu. For the rancher server cluster I launched it with EKS (eksctl) and did not use a custom AMI. The production cluster I created a launch template with a custom AMI and created the cluster with Rancher console. I am looking for advice on the best way to upgrade. I think I can do the following: • Upgrade Rancher to 2.6.8 (can I go directly or do need to do it in steps?) • Upgrade server cluster to 1.21 and then to 1.22 Using AWS Console Upgrade Downstream Cluster would be a multi stage process. • I have to create a new AMI using the correct base AMI + my customizations.. • Upgrade the cluster using the AWS console to 1.21 • Re-launch all nodes with the matching AMI • Upgrade the Cluster using the AWS console to 1.22 • Re-launch all nodes using the matching AMI Do I have it right?
    b
    • 2
    • 2
  • c

    curved-lifeguard-39360

    09/15/2022, 8:18 PM
    I have a downstream cluster that I created using rancher console. I recently deleted a node group, but I did it from the AWS Console. Don’t know if it matters, but I also added a node group to that cluster using eksctl instead of rancher. Now that cluster is showing this error: _*InvalidParameterException: You cannot specify an AMI Type other than CUSTOM, when specifying an image id in your launch template. { RespMetadata: { StatusCode: 400, RequestID: “c5a5bfb7-70ae-476f-8548-94858b1aa765” }, ClusterName: “pano-prod”, Message_: “You cannot specify an AMI Type other than CUSTOM, when specifying an image id in your launch template.“, NodegroupName: “pool-pvt” }*_
    c
    • 2
    • 9
  • f

    flaky-shampoo-86024

    10/17/2022, 7:31 PM
    Hi, I want to install rancher on AWS EKS cluster but with istio setup(ingress/egress). I see installation instructions wants nginx-ingress, which I dont want to use for my use case. Any reference document of setup please
  • r

    rough-london-41454

    10/18/2022, 12:10 PM
    Hi I'm new, I installed rancher on my aws eks cluster, when I tried accessing my ingress load balancer url, its returning 504 gateway timeout, pls can anyone help out. thanks
  • n

    narrow-rose-64653

    10/21/2022, 3:32 PM
    Has anyone successfully imported EKS Clusters v1.23.0 into Rancher 2.6. I have multiple EKS Clusters running in my AWS Account, which I'm trying to import to Rancher 2.6 (running on another EKS and VPC) but I'm getting
    Waiting for API to be available
    error. I have completed all networking between both clusters, not sure what's happening. Any guidance would be greatly appreciated.
  • w

    worried-rain-56725

    10/25/2022, 8:18 AM
    Hi. I’m trying to create downstream EKS cluster using Rancher terraform provider. For some reason EKS nodes created by Rancher have only default EKS security group even though we provided list of additional security groups. These SGs are present in EKS configuration, but nodes still don’t have them. The strangest thing is that during creating EKS cluster, you can see in node group config that they will use auto-generated SG by Rancher, but after cluster became to Active state, config has been changed and there are now list of security groups we specified. We do not have custom launch template, nodes launched from LT created by Rancher as well. And that LT for some reason contains only the default EKS node group and no instance type specified. I thought it may be issue in 2.6.4 version, but after upgrading to 2.6.9 issue still here
Powered by Linen
Title
w

worried-rain-56725

10/25/2022, 8:18 AM
Hi. I’m trying to create downstream EKS cluster using Rancher terraform provider. For some reason EKS nodes created by Rancher have only default EKS security group even though we provided list of additional security groups. These SGs are present in EKS configuration, but nodes still don’t have them. The strangest thing is that during creating EKS cluster, you can see in node group config that they will use auto-generated SG by Rancher, but after cluster became to Active state, config has been changed and there are now list of security groups we specified. We do not have custom launch template, nodes launched from LT created by Rancher as well. And that LT for some reason contains only the default EKS node group and no instance type specified. I thought it may be issue in 2.6.4 version, but after upgrading to 2.6.9 issue still here
View count: 2