https://rancher.com/ logo
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
kubernetes
  • e

    echoing-island-97495

    04/28/2022, 4:38 PM
    How do I remove a Container Default Resource Limit from a Project? I can see how to update one after I add it, but if try to save changes with blank fields the changes don't apply.
    • 1
    • 1
  • m

    many-dawn-55841

    05/03/2022, 7:20 AM
    Hi There, I am deleting the namespace but it’s get re-created again. There is no resources under that namespace.
    • 1
    • 1
  • i

    incalculable-carpenter-21031

    05/05/2022, 1:47 PM
    I created a cluster using
    docker run -d --restart=unless-stopped -p 443:443 --privileged rancher/rancher
    and it is running fine .
    $docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                  NAMES
    10247a667a0d        rancher/rancher     "entrypoint.sh"     12 months ago       Up 26 minutes       0.0.0.0:443->443/tcp   clever_fermi
    I created a cluster but the dashboard is not accessible from the browser now. What could have gone wrong and how could i troubleshoot ? any pointers is highly appreciated. optionally i don't mind setting up another docker container to import the existing cluster , if possible.
    a
    • 2
    • 6
  • s

    strong-salesmen-29486

    05/11/2022, 2:05 AM
    Question: I am trying to navigate to another cluster using Rancher UI and I either get a fail whale or the browser just hangs there. When it hangs, I am still able to access the shell and the kubectl cli however, I just can't view it from the UI. Also, I am able to perform all kubectl commands from local terminal. Is there anyone that can explain or guide me in the right direction to fix this?
    f
    • 2
    • 1
  • w

    witty-sunset-95652

    05/16/2022, 7:01 PM
    Hi, is it the case that using an external mysql database is no longer the recommended approach for a high availability Rancher installation? https://rancher.com/docs/rancher/v2.6/en/installation/resources/k8s-tutorials/how-ha-works/
    c
    m
    • 3
    • 4
  • b

    boundless-dog-9864

    05/24/2022, 3:45 PM
    Question about rancher RBAC. Do the permissions in the rancher web console differ from those that kubectl gives you? Im trying to create a project role that give read access to everything in a project apart from secrets where I want to be able to list them but NOT get (describe) them. I've added secrets in the api group “” with “list” and “watch” and everything else in the group also has “get”. Kubectl correctly blocks the account from getting the secret but I can still list them. However in the rancher UI I can both list and get them. This is the exact same user without making any rbac permissions.
    f
    • 2
    • 1
  • b

    breezy-france-3792

    05/26/2022, 7:38 AM
    Hey #kubernetes Kubernetes CIS Benchmark requires anonymous auth set to false on api server but adding the new node or upgrade of existing node might fail because of this setting. We have a option to use the discovery file option but that requires the kubconfig to be places on the node which will again be flagged by security team as risk. Did anyone came across this issue in your setup if yes how did you resolve this. I see this setting set to false in other kubenetes engines like RKE(Rancher Kubernetes Engine) Thanks In advance
    f
    • 2
    • 1
  • s

    stale-dinner-99388

    05/27/2022, 12:04 PM
    Hello Everyone. I am trying to embed rancher dashboard into html page using
    iframe
    but i am not able to do B/z of
    X-Frame-Options
    header is there any way to disable
    X-Frame-Options
    header
    m
    • 2
    • 3
  • c

    crooked-cat-21365

    06/08/2022, 11:55 AM
    I have severe problems upgrading my rancher cluster to a new version of rke. Currently it is running rke v1.21.7 and Rancher v2.6.3-patch1. Everytime I try to upgrade the rke cluster it gets bricked somehow, see https://github.com/rancher/rke/issues/2833. This is a production system, i.e. I cannot afford a huge downtime to examine some rke --debug mode, etc. What would you suggest?
    c
    • 2
    • 15
  • b

    bland-school-82008

    06/13/2022, 6:05 AM
    kubectl get po -n connect Unable to connect to the server: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
    r
    • 2
    • 1
  • p

    powerful-autumn-18622

    07/01/2022, 6:53 PM
    hi , can anyone help me with my query, I am trying to import an existing cluster , but cluster-agent fails with crashloopbackoff and looking at logs it says no host found https://rancher, whereas I can ping the https://rancher/ping and get a response , I am trying to import a cluster with version v1.24.2 on rancher v2.4.5, any help is appreciated, thanks !!
    c
    • 2
    • 5
  • m

    most-kite-870

    07/21/2022, 9:06 AM
    Let say that I have a kubernetes cluster setup and running. Do I install the Rancher into the cluster or place the rancher in another cluster to manage the fore mentioned cluster?
    r
    c
    • 3
    • 4
  • b

    busy-crowd-80458

    07/25/2022, 10:55 PM
    okay, on the verge of throwing Longhorn into the dumpster
  • b

    busy-crowd-80458

    07/25/2022, 10:55 PM
    dozens of
    MountVolume.WaitForAttach failed for volume "pvc-15f49a13-3c55-4ca6-a758-cd05696fa6a7" : volume pvc-15f49a13-3c55-4ca6-a758-cd05696fa6a7 has GET error for volume attachment csi-2f9369281bf492a04929add4f5006ca8d15e6137b0ec46d2de90cb164170ee10: volumeattachments.storage.k8s.io "csi-2f9369281bf492a04929add4f5006ca8d15e6137b0ec46d2de90cb164170ee10" is forbidden: User "system:node:emerald01" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope: no relationship found between node 'emerald01' and this object
  • b

    busy-crowd-80458

    07/25/2022, 10:56 PM
    fresh brand new install of Longhorn 1.30 after uninstalling it and wiping everything because it was broken on the earlier install 😞
  • b

    busy-crowd-80458

    07/26/2022, 12:23 AM
    hmm... not sure it's really longhorn's fault. I've got bricks that are themselves hosted on Ceph (Proxmox)
  • c

    careful-advantage-31708

    08/07/2022, 10:02 PM
    hello I want to ask, I tried to use ingress using haproxy on a kubernetes cluster, but it didn't work, instead it redirected to the web ui rancher dashboard. is there any solution for this? I tried to find a way to turn off the default ingress on the cluster but couldn't find a way.
    f
    • 2
    • 3
  • a

    agreeable-oil-87482

    08/10/2022, 9:24 AM
    I'd first find out why your Pods are being evicted
    c
    • 2
    • 4
  • p

    purple-translator-99032

    08/11/2022, 2:00 PM
    Hello. is it possible to deploy RKE2 clusters via the Rancher UI, (v.2.6.6) running on a RKE cluster. Or does rancher has to run on a RKE2 cluster to deploy RKE2 downstream clusters? I only fount this documentation but it doesn't say how it is done in rancher. Also is it possible to update the downstream clusters in rancher to rke2 ?
    a
    • 2
    • 1
  • b

    busy-ability-54059

    08/17/2022, 9:46 AM
    duplicate question https://rancher-users.slack.com/archives/C3ASABBD1/p1660701198552049
  • f

    flaky-shampoo-86024

    08/17/2022, 6:12 PM
    Hi, while importing new cluster into rancher, i got this error
    time="2022-08-17T16:12:46Z" level=error msg="Issuer of last certificate found in chain (CN=Kubernetes Ingress Controller Fake Certificate,O=Acme Co) does not match with CA certificate Issuer (CN=dynamiclistener-ca,O=dynamiclistener-org). Please check if the configured server certificate contains all needed intermediate certificates and make sure they are in the correct order (server certificate first, intermediates after)"
    time="2022-08-17T16:12:46Z" level=fatal msg="Get \"<https://XXXXXXX20bd0d1943a40562b-1514086031.us-east-2.elb.amazonaws.com>\": x509: certificate is valid for ingress.local, not <http://XXXXXXX20bd0d1943a40562b-1514086031.us-east-2.elb.amazonaws.com|XXXXXXX20bd0d1943a40562b-1514086031.us-east-2.elb.amazonaws.com>"
  • s

    sparse-exabyte-74414

    08/17/2022, 6:43 PM
    Hi, i am using github sso in rancher. we are not able to add github teams as cluster member.while adding we are getting all the public github groups in list but not the private groups associated with organisation.
  • c

    careful-optician-75900

    08/18/2022, 6:30 AM
    2022/08/12 20:13:50 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 2022/08/12 20:13:50 [INFO] Watching metadata for management.cattle.io/v3, Kind=Node 2022/08/12 20:13:50 [INFO] Watching metadata for apps/v1, Kind=StatefulSet 2022/08/12 20:13:53 [INFO] Adding peer wss://192.68.101.82/v3/connect, 192.68.101.82 2022/08/12 20:13:53 [ERROR] Failed to connect to peer wss://192.68.101.82/v3/connect [local ID=192.68.102.155]: websocket: bad handshake 2022/08/12 20:38:03 [ERROR] Error during subscribe websocket: close sent 2022/08/12 20:38:03 [ERROR] Error during subscribe websocket: close sent Is anyone faced this kind of websocket close connection in rancher containers ?
  • l

    little-ambulance-5584

    08/22/2022, 10:42 PM
    Is the helm deployment of rancher still pretty well supported? the chart seems very limiting https://github.com/rancher/rancher/tree/release/v2.6/chart
    c
    • 2
    • 23
  • f

    freezing-wolf-83208

    08/23/2022, 5:50 AM
    trying to upgrade kubernetes on my local cluster v2.6.6 - I first downloaded rke 1.3.12, we were using 1.2.2 before and running on k8s 1.19.3. and then on running the rke up command I run into WARN[0196] [etcd] host [161.211.xx.xx] failed to check etcd health: failed to get /health for host [161.211.xx.xx]: Get "https://161.211.xx.xx:2379/health": remote error: tls: bad certificate FATA[0196] [etcd] Failed to bring up Etcd Plane: etcd cluster is unhealthy: hosts [161.211.xx.xx] failed to report healthy. Check etcd container logs on each host for more information The etcd logs show that its not able to trust the CA - kube-ca is being used. tried to rotate the cert, ran into a diff issue - "FATA[0000] Failed to rotate certificates: can't find old certificates" any ideas would be greatly appreciated.
  • f

    flaky-shampoo-86024

    08/24/2022, 7:09 PM
    Hi have setup Rancher Dashboard 2.6.2. The cluster am trying to import, executed Curl YAML generated by Rancher dashboard to import existing cluster. I see pod created in imported cluster is in CrashLoopBackOff state. Error : Error from server : Get "https://workernodip:10250/containerLogs/cattle-system/cattle-cluster-agent-5cxxxxxx/cluster-register": remote error: tls internal error
  • f

    freezing-wolf-83208

    08/25/2022, 4:32 AM
    Upgraded the k8s of the local cluster using RKE but now rancher doesnt start. when I check t -sh-4.2$ kubectl -n cattle-system get pods NAME READY STATUS RESTARTS AGE rancher-5c89f67486-l5hxr 1/1 Running 4 (4d1h ago) 29d rancher-5c89f67486-ssxhm 1/1 Running 4 (4d20h ago) 29d rancher-5c89f67486-zpzwg 0/1 Running 6 (110s ago) 29d rancher-webhook-656ccdb579-mgmjm 1/1 Running 2 (7d ago) 29d -sh-4.2$ kubectl -n cattle-system logs -f rancher-5c89f67486-zpzwg Doing /etc/rancher/ssl 2022/08/25 04:20:08 [INFO] Rancher version v2.6.6 (9b2f2ae0e) is starting 2022/08/25 04:20:08 [INFO] Rancher arguments {ACMEDomains:[] AddLocal:true Embedded:false BindHost: HTTPListenPort:80 HTTPSListenPort:443 K8sMode:auto Debug:false Trace:false NoCACerts:false AuditLogPath:/var/log/auditlog/rancher-api-audit.log AuditLogMaxage:10 AuditLogMaxsize:100 AuditLogMaxbackup:10 AuditLevel:0 Features: ClusterRegistry:} 2022/08/25 04:20:08 [INFO] Listening on /tmp/log.sock 2022/08/25 04:20:08 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/08/25 04:20:10 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/08/25 04:20:12 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/08/25 04:20:14 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials
  • c

    careful-optician-75900

    08/29/2022, 2:27 PM
    Hello, I m using AWS EKS with Classic Load Balancer, Got this kind of issue for web-socket. I really appreciate any help on this.
    websocket: close 1006 (abnormal closure): unexpected EOF
    That Websocket error is from rancher containers.
  • q

    quiet-park-6213

    08/30/2022, 8:34 AM
    please guide how to add gcr private registry in Rancher 2.7 with json file ?
  • p

    purple-translator-99032

    08/31/2022, 3:44 PM
    Hello, i'm facing a problem with a extremly high CPU usage on my Cluster. After upgrading rancher from from 2.6.6. to 2.6.8 and upgrading my downstream cluster from 1.23.6 to version 1.24.2. Even when i drain my node and only the essentials pods are running. Dockerd has a CPU usage of above 1000%. Because on this high CPU usage on all workers, pods are getting killed sometimes. Any suggestions ?
    👀 1
    • 1
    • 2
Powered by Linen
Title
p

purple-translator-99032

08/31/2022, 3:44 PM
Hello, i'm facing a problem with a extremly high CPU usage on my Cluster. After upgrading rancher from from 2.6.6. to 2.6.8 and upgrading my downstream cluster from 1.23.6 to version 1.24.2. Even when i drain my node and only the essentials pods are running. Dockerd has a CPU usage of above 1000%. Because on this high CPU usage on all workers, pods are getting killed sometimes. Any suggestions ?
👀 1
We found the issue. Oracle Linux 8.6 with Kernel version 5.15 will occur this issue on Kubernetes 1.24
Opened Issue here: https://github.com/rancher/rancher/issues/38816
View count: 162