I am having problems with longhorn 1.2.2: I tried setting up a backup target on a custom S3 endpoint. But we're getting an error:

error listing backup volume names: Timeout executing: /var/lib/longhorn/engine-binaries/rancher-mirrored-longhornio-longhorn-engine-v1.2.2/longhorn [backup ls --volume-only <s3://backups@us-east-1/>], output , stderr, , error <nil>

I tried logging in to the container and checking the probelm from its side and this is what we get:

root@longhorn-manager-57lhs:/# /var/lib/longhorn/engine-binaries/rancher-mirrored-longhornio-longhorn-engine-v1.2.2/longhorn backup ls --volume-only <s3://backup/>
ERRO[0020] Fail to list s3: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors  pkg=s3

I am sure the credentials in secret are valid and properly formatted ( I've copied minio configuration and adjusted to our environment ). Can anyone from here help us diagnose the problem further and solve it?

👋 Hi everyone!, am glad to be here

Hi everyone, we upgraded our AKS cluster to the latest version and since then, the cluster remain in waiting state. error message is "This cluster is currently *Waiting*; areas that interact directly with it will not be available until the API is ready. Waiting for API to be available"

Please I need some help on how to fix this issue

Hello, is it an expected behaviour that the old UI is still available in the new versions?

I'm not familiar with the abbreviation "KDM" used in the rancher/rancher milestones (v2.7-KDM-For-2.6.11) can someone point me to an explanation?

Hello, I am just getting started with Rancher and have a setup via docker running to learn the workings of Rancher and need some help with monitoring and managing single nodes/hosts running only docker containers. My test setup has 1. A K3 Cluster - 1 Node (Etc, Server) + 1 Node (Worker) a. I am able to launch PODs and do basic K8 workflows. 2. In addition to the K8 cluster, I also have a single node running containers using docker-compose. I am having a hard time figuring out how I can manage the node and containers via Rancher. Any help would be appreciated.

Hi, We have rancher setup (2.69), and we have created multiple EC2 provider clusters (RKE2). Now, we want to add windows worker nodes in existing ec2 provider clusters. Kindly advise if it is possible. Your support will be highly appreciated. Regards!

I'm having a problem with Prometheus constantly getting OOM killed in rancher 2.6.5. I'd like the drop the 'id' label since it seems to have high cardinality but I'm having trouble figuring out where to put that when configuring the rancher-monitoring app. It's basically this SO question

Hello, I am importing existing EKS Cluster into Rancher-2.7.0. Here is my setup on AWS • Rancher 2.7 is installed on 3 node K3s cluster using helm • EKS Cluster exists in Account-B • IAM user account exists in Account-A • IAM user in Account-A can assume role in Account-B which has Full EKS permissions • I have validated with aws sts get-caller-identity and kubectl commands that IAM user in Account-A is able to get nodes, get pods etc which proves that the basic IAM setup is working as expected. • EKS Cluster version: 1.21. However, when I try to add AWS credential to Rancher before importing the cluster, I get “Authentication test failed, please check your credentials”. Please throw some pointers on how to troubleshoot the error.

Hi All 👋 , Using the Ranger UI I’m trying to create a cluster on Digitalocean using RKE2. I want to install Digitaloceans own cloud provider to enable loadbalancer creation and such, but when I switch the cloud provider setting to

external

the cluster fails to install. Do I need to add a

Cloud Provider Config

in the field that comes up?

Hello. I have 3 master and kube-controller-manager & kube-scheduler cert went old and Rancher UI or “rke2 certificate rotate” does not rotate these ones. Is there something still to test or do I need to generate new one and try with that? Any help appreciated

I'm deploying Stateful sets via API request and ServiceName is always missing even though it's valid in the request. I tested the same data using the UI and it works fine. Anyone run into this problem?

Hello. I have arranged the setup of a local single-node k3s cluster on a WSL2 linux ubuntu ditribution and I'd like to use nerdct to build local images. I tried instaling buildkitd and buildctl from the nerdctl-full package but when I start the daemon with the command sudo $(which buildkitd --containerd-worker-addr /run/k3s/containerd/containerd.sock) I receive this strange warning: "WARN[2023-03-02T192620+01:00] skipping containerd worker, as "not" does not exist buildkitd: no worker found, rebuild the buildkit daemon?" and the daemon doesn't start. Can you help me understand what is missing or if what I'm trying to do is not possible at all? Thanks!

Hi All, I am importing EKS cluster into Rancher-2.7. Downstream EKS version is 1.21. When I register, I see "Waiting for API to be available" on the UI. Where can I get more details? How to troubleshoot this issue?

I am trying to move my rancher install from one cluster to another with rancher-backup. I'm getting errors restoring because it can't create the CRDs. I see that apiextensions.k8s.io/v1beta1 is unavailable on EKS 1.22, but what I can't figure out is why on the old cluster the CRDs are defined with apiextensions.k8s.io/v1 but the json objects in my backup tarball is defined with the v1beta1 version. For example this is how apps.catalog.cattle.io is defined when I inspect it in the rancher UI

apiVersion: <http://apiextensions.k8s.io/v1|apiextensions.k8s.io/v1>
kind: CustomResourceDefinition
metadata:
... trimmed ...
  name: <http://apps.catalog.cattle.io|apps.catalog.cattle.io>

In my backup file I have ./customresourcedefinitions.apiextensions.k8s.io#v1beta1/apps.catalog.cattle.io.json that starts with this:

{"apiVersion":"<http://apiextensions.k8s.io/v1beta1|apiextensions.k8s.io/v1beta1>","kind":"CustomResourceDefinition","metadata":  ...trimmed...

I can't work out why it is defined in my source cluster as

v1

but recorded in the backup set as

v1beta1

.

Hi , We have rancher setup (2.69), and we have created multiple EC2 provider clusters (RKE2). Now, we want to add windows worker nodes in existing ec2 provider clusters. Kindly advise if it is possible. Your support will be highly appreciated. @miniature-megabyte-23048 @rancherteam @everyone Regards!

Hi all, since I moved from Docker Desktop to Rancher, I have a strange problem when building new containers. Sometimes (not always) the build takes forever. I identified this as a network issue. Does anyone have an idea what options I have to debug or solve this? Sorry for the limited information, happy to provide more!

Hi all. We are currently running Rancher 2.6.9 and see that one one of our clusters the cattle-cluster-agent pod is increasing it's memory usage. Over the past 12 hours it increased from 2.53GB to 6.53GB. It also became unresponsive in the rancher UI. Restarting the cattle-cluster-agent pods is the only way to recover it. Is there a known bug around this? or a way to debug this issue?

Hello Ranchers.. strange issue, a simple docker setup which runs a simple sveltejs/Vite app to show that HMR (hot module reloading) works. It works out of the box on Docker but not Rancher desktop. Any hints? docker-composer.yml: https://github.com/fcrozatier/kit_docker/blob/main/docker-compose.yml Dockerfile: https://github.com/fcrozatier/kit_docker/blob/main/Dockerfile

Hello seem the DNS in the upstream cluster is flipping when there are more than one node. Has someone faced this on a cluster with EC2 instances as nodes? Appreciate any help on this issue. https://github.com/rancher/rancher/issues/40762

I just updated kubernetes on a bunch of clusters, now I'm getting "Cluster must have at least one etcd plane host: failed to connect to the following etcd host(s) [172.20.32.83]" They're stuck in "Updating" and the "Updating" badge is red

Hi all, im looking for any best practices and examples how to configure rancher groups/profiles to allow a user for a given tenant to use / create a custom resource whose CRD has been added by an administrator. Is there a way to avoid adding the RBAC permissions for each CR in the profile of the user or group of user? In our cluster, user belongs to "authenticated:user" group. Thanks

Apparently everybody with ClusterMember permission can create his own project without resource limits and become a ProjectOwner. Would it be possible to fix this? How about setting at least some default project constraints for each managed cluster?

I'm still cleaning up the mess in one of my cluster. It looks like there is a node in my etcd that I briefly created and then deleted. Is there a way to forcefully remove a node from etcd? The usual way is not working:

Anyone run into a similar issue?

Internal error occurred: failed calling webhook "rancher.cattle.io.secrets": failed to call webhook: Post "<https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=10s>": context deadline exceeded

Hi, just want to ask if there is any documentations for installing rke2 for rhel 8.6 ?

Hello 👋 I am attempting to make a very restricted

read only

cluster role for team members. The only hangup I have is the monitoring page, only getting

Resources Unavailable

. I do not want to add anyone to the

System

project, and only managed though

RoleTemplate

. Thoughts? Will post yaml to reply.

Hi, Could someone point me to the REST API for creating cluster drivers, Is it /v3/clusters? What params do you need to pass to create a cluster using the REST API? Thanks. I have a tried POST to /v3/clusters, but this only creates a provisioning cluster/management cluster, which is not associated with a driver.

Hi guys, I got some issues on my Rancher Manager. One of my cluster stuck in Updating, Reconciling, configuring bootstrap nodescustom 3668erttyyt6waiting for probes:kube-controller-manager,kube-scheduler. When I see the logs of kube-scheduler, it say “/var/lib/rancher/rke2/server/tls/kube-schedule/kube-schedule.crt no such file or directory”. some glimpses of the issues are following. Please guide. Our business is stuck.