https://rancher.com/ logo
Join Slack
Powered by
# general
  • p
    are k3s and rke2 slowly merging together under the hood? thats the vibe ive been getting for awhile now
    c
    • 2
    • 6
  • k
    Since we've updated Rancher to 2.11, newly created clusters using our terraform modules can't bootstrap any more. The capi-controller-manager is reporting the following error;
    Copy code
    "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: {\"Code\":{\"Code\":\"Forbidden\",\"Status\":403},\"Message\":\"<http://clusters.management.cattle.io|clusters.management.cattle.io> \\\"c-m-xxxxxxxx\\\" is forbidden: User \\\"u-xxxxxxxx\\\" cannot get resource \\\"clusters\\\" in API group \\\"<http://management.cattle.io|management.cattle.io>\\\" at the cluster scope\",\"Cause\":null,\"FieldName\":\"\"}" controller="clustercache" controllerGroup="<http://cluster.x-k8s.io|cluster.x-k8s.io>" controllerKind="Cluster" Cluster="fleet-default/cluster-az1" namespace="fleet-default" name="cluster-az1" reconcileID="dea8fe2f-1f4d-4f1b-a1e4-b7e21e1e1a6e"
    Anyone has ideas what is causing this and how to fix it?
    b
    • 2
    • 5
  • p
    Quick question - I'm setting up Shibboleth authentication in Rancher 2.11.1, and I see that the metadata is only valid for 48 hours at a time - can that validFor time be edited, or is it unalterable?
  • c
    Hi All - Hope all is Well ? I am Mike - I am not new to kubernetes but joined the slack because I am having issues getting Rancher Desktop to form a cluster on my new Apple Silicon M4 laptop - probably not right forum to answer here - but maybe someone can point me to the right forum?
    m
    • 2
    • 1
  • b
    Hi everyone, I'm trying to provision a vSphere RKE2 cluster via Rancher (installed via Helm inside an AKS cluster). The cluster setup gets stuck at: > Configuring bootstrap node(s): waiting for agent to check in and apply initial plan Here’s what I’ve already tried: • Using Ubuntu cloud image (tried both focal and jammy) • VMs receive IPs in vCenter and are accessible via SSH • Curl to Rancher endpoint over HTTPS works from the nodes (valid SSL cert) • No firewall is blocking access between Rancher and the nodes • DNS: Azure Private Resolver pointing to on-prem DNS, which resolves both Rancher and vCenter I attached two pictures that show the issue. Would really appreciate any advice on what else I can check or try. Thanks 🙏 Naor
    c
    p
    • 3
    • 2
  • b
    image.png,image.png
  • a
    I have a question about the aws-cloud-provider in an airgapped environment. We've been running Rancher and RKE2 with the aws out of tree cloud provider in our air-gapped environment just fine. We're having to move to another air-gapped instance of AWS so I'm deploying there now. The cloud provider is erroring with
    Copy code
    Cloud provider could not be initialized: could not init cloud provider "aws": errror finding instance <id>: "error listing AWS instances: ... Post "<https://ec2>.<region>.<http://amazonaws.com|amazonaws.com>" lookup ec2.<region>.<http://amazonaws.com|amazonaws.com> no such host
    I suspect in the previous environment routing was configured for the amazonaws.com domain, but it is not here (I don't have control over that). Is this configurable? On all of the hosts in my 
    /etc/rancher/rke2/cloud.conf
    file I have something like
    Copy code
    [Global]
  Service=ec2
  Region=<region>
  URL=<https://ec2>.<region>.our.domain
  SigningRegion=<region>
    Is there another arg I can add to the aws-cloud-controller-manager values section in my cluster config?
    • 1
    • 1
  • p
    Hi there! I have a question about provisioning clusters to vSphere. Is it possible to tell Rancher to create sata cdrom device for bootstrap instead of an IDE? My goal is to make it work on ESXI-arm fling. Atm I need to power off VMs, change the device type manually, and power them on again. After that, VMs continue deploying with no issues. Or maybe it's a feature request..
  • f
    Hello , K3S users , do you have any problem pulling grafana images from you're deployments ?
  • h
    I want to use RKE2 ( open source) for my dev ,test,prod env . Is this is a good option ? Could you please tell me what all features RKE2 provides ua on top of vanilla k8s . Can I take enterprise support with this open source project as optional ? And who will provide us that support
    c
    b
    • 3
    • 19
  • b
    I'm using Rancher UI and it is installed in a k3s cluster. Something I didn't realize is, I could actually manage k3s upgrades via Rancher UI itself. Silly I suppose, but I found out Rancher does have this capability and had done a manual upgrade of k3s nodes external to rancher at an earlier point in time. I'm running v1.27.16, but the Rancher UI is showing v1.21.9. How can I get this version mismatch fixed?
    • 1
    • 2
  • b
    so, my rancher has 7484 pods, almost all of which are "evicted", how can I clean those up without doing it one at a time? found the answer here; https://gist.github.com/ipedrazas/9c622404fb41f2343a0db85b3821275d
    👍 1
  • a
    I am not able to install RKE2. Following doc: https://docs.rke2.io/install/quickstart seeing below error:
    Copy code
    INFO]  finding release for channel stable
[INFO]  using 1.31 series from channel stable
Updating Subscription Management repositories.
Rancher RKE2 Common (stable)                                                                                                                                                                  26 kB/s | 2.9 kB     00:00
Rancher RKE2 1.31 (stable)                                                                                                                                                                    42 kB/s | 3.5 kB     00:00
Error:
 Problem: package rke2-server-1.31.8~rke2r1-0.el9.x86_64 from rancher-rke2-1.31-stable requires rke2-common = 1.31.8~rke2r1-0.el9, but none of the providers can be installed
  - package rke2-common-1.31.8~rke2r1-0.el9.x86_64 from rancher-rke2-1.31-stable requires rke2-selinux >= 0.12-0, but none of the providers can be installed
  - conflicting requests
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.12-1.el9.noarch from rancher-rke2-common-stable
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.13-1.el9.noarch from rancher-rke2-common-stable
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.14-1.el9.noarch from rancher-rke2-common-stable
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.15-1.el9.noarch from rancher-rke2-common-stable
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.16-1.el9.noarch from rancher-rke2-common-stable
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.17-1.el9.noarch from rancher-rke2-common-stable
  - nothing provides container-selinux >= 3:2.191.0-1 needed by rke2-selinux-0.18-1.el9.noarch from rancher-rke2-common-stable
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
    How do I install it ? Can someone please help. I was able to install it till yesterday.
  • w
    i need help that after uninstalling all docker desktop, and try to rancher desktop the path for docker is not working, i tried using factory reset but still not working, can someone help me
    b
    b
    • 3
    • 2
  • q
    Hi. I am trying to migrate a Rancher single node Docker installation to a Rancher single node Kubernetes (K3S) installation and try to wrap my head around how Racher's data is persisted. For Docker I had a volume attached to my node and bind mounted it into the container. I don't see that the helm chart supports something similar. Does it?
    h
    w
    c
    • 4
    • 7
  • c
    i'm running rancher v2.11.0 which is connect to an ldap server. I'm running into a problem where the tokens that are created for kubeconfig time out, even though when i check the list of api tokens, they are supposed to be valid for another 89+ days (tokens are set to expire after 90 days). I tried upgrading to 2.11.1 but that resulted in a problem where when creating a new cluster, it would show an error that it can't create a namespace, i do believe however that it might have fixed the issue with the tokes (not been able to run long enough). Not sure what to do at this point or where to begin debugging.
  • a
    Hello All We are running rancher on top of rke2 cluster which is managed manually. Then using rancher we are managing 6 different clusters. Yesterday I did a planned upgrade of one clusters which failed and created issues with calico. luckily I had a backup which I used to restore the cluster to a previous state. Today I tried the upgrade again and same thing so I had to restore the cluster from backup. I will have to upgrade this and other clusters eventually to new versions but I cant keep them down in the event of an issue during upgrade. To account for downtime, I want to clone one of my entire cluster and then run the upgrade on it. So how can I clone my cluster?
  • s
    hi guys does anyone knows how to fix this error
    m
    • 2
    • 1
  • a
    is there a way to clear restart counts for kube-system namespace pods for rke2 server? they seem to be running from containerd via rke2-server, but they're running as individual pods and deleting the pod just increments the restart count
    c
    • 2
    • 3
  • c
    Hey folks, I was wondering if anyone else experienced this same rancher bug on version 
    2.10.1
    . It seems that if I navigate to 
    .../v3/projectroletemplatebindings
    and click on a hyperlink to a principal (i.e. 
    groupPrincipalId": "github_<team://XXXXXXX>"
    ) I get a broken URL path 
    .../v3/principals/github_team%253A/XXXXXXX
    that returns a 500 error. But if I go to 
    .../v3/principals/github_team:%2F%2FXXXXXX
    I get the correct information. Any ideas? Im seeing an error in the Rancherv2 terraform module for the 
    rancher2_project_role_template_binding
    resource and I think it is related. Slack Conversation
    c
    • 2
    • 10
  • c
    I want to run a k3s control plan near but not in my datacenters and use my own dedicated servers for the nodes. Having k3s outside of the datacenters will allow me to failover services to the other datacenters without doing multi-cluster. However I also don't want to manage more virtual machines to run k3s. Has anyone tried running k3s control plane on a serverless platform like fly.io or Koyeb or even inside another Kubernetes cluster (is that stupid?)?
    • 1
    • 1
  • c
    I had to recover a managed cluster from an etcd snapshot created by Rancher, but I was rather disappointed that the projects were not recovered. I got a huge list of namespaces "not in a project", but the project itself was gone. Is this as expected? I had thought the projects are listed in the etcd db as well. How can I create a backup including the projects?
  • b
    Copy code
    ~
  • a
    hello, I need a help with the metrics server for my k3s cluster. the issue is I cant get the pods metrics but able to get metrics of the node. i tried everything , editing the yaml file and putting the - --kubelet-insecure-tls but no luck. please help me C:\Users\DELL\Desktop\Coconut-Peat-Supply-chain_core_system> kubectl top pod -n default core-system-b87559449-tw5mq Error from server (NotFound): podmetrics.metrics.k8s.io "default/core-system-b87559449-tw5mq" not found PS C:\Users\DELL\Desktop\Coconut-Peat-Supply-chain_core_system> kubectl top pod -A error: Metrics not available for pod default/core-system-b87559449-tw5mq, age: 7h8m12.9991759s PS C:\Users\DELL\Desktop\Coconut-Peat-Supply-chain_core_system> kubectl top nodes NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% harith-vithanage-main 1204m 10% 2677Mi 34%
  • p
    will rancher be more tightly integrating with kuberc? https://cloudfleet.ai/blog/cloud-native-how-to/2025-05-customizing-kubectl-with-kuberc/ would be nice to not have to update my kube config files every so often by re-downloading the rke2 config
  • a
    Hello everyone. Do I understand correctly that in rke2 there is no mechanism for changing data-dir?
  • v
    }rishabh@RSACHAN-M0PNHJH:~/.kube$ k get pod E0526 103749.695045 3028 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate signed by unknown authority E0526 103749.697242 3028 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate signed by unknown authority E0526 103749.699036 3028 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate signed by unknown authority E0526 103749.700937 3028 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate signed by unknown authority E0526 103749.703790 3028 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate signed by unknown authority Unable to connect to the server: tls: failed to verify certificate: x509: certificate signed by unknown authority getting this error when accessing from wsl ubuntu but fine from windows local
    p
    b
    • 3
    • 2
  • b
    Hi everyone I'm developing a Rancher UI extension and planning to add it as a Top-Level Product. I wanted to ask: • From a top-level product extension, is it possible to access Kubernetes cluster resources (like pods, CRDs, etc.) using Rancher's API ? • If so, is there any documentation or example that shows how to do this properly?
    w
    • 2
    • 1
  • f
    Hello, Since I’ve upgraded to Rancher 2.11.x (tested on 2.11.1 and 2.11.2), I can’t delete namespace anymore. They are stuck in Terminating and when I try to get all resources in the namespace I got following errors.
    Copy code
    $ kubectl get-all -n <stuck-ns>
W0527 11:15:24.709087 2243547 client.go:102] Could not fetch complete list of API resources, results will be incomplete: unable to retrieve the complete list of server APIs: ext.cattle.io/v1: the server is currently unable to handle the request
    h
    c
    • 3
    • 15
  • f
    Hello all good morning nice to meet everyone I am here well at least to this channel anyway