clusters aren’t aware of their names. only rancher knows what you named the cluster.

This isn’t just a rancher thing; there’s no standard resource or field in which to store a cluster’s own name in Kubernetes.

custom scheduler could probably update the pod manifest

i know that the cluster is not aware of his name.. but Rancher is writing the cluster name inside a node annotation:

<http://cluster.x-k8s.io/cluster-name|cluster.x-k8s.io/cluster-name>

im wondering if this can be used somehow

HI, Is there a way containerd can pickup additional mirror configured without actually restarting rke2 service ?

Are there any known issues or challenges when using RKE2 on Ubuntu 22.04 with the Canal network plugin?

Are RKE2 upgrades supposed to be mostly seamless for workloads running on the node, or is a networking interruption expected? I have a handful of single-node “clusters” which are configured not to drain etc, and I was expecting things to mostly just carry on but have been getting “Connection Refused” errors accessing NodePort services for a short time during the RKE2 upgrade process.

On a related note, I had an RKE2 node downgrade from v1.25.10+rke2r1 to v1.25.9+rke2r1 at 2023-06-12T072908Z then upgrade again at 2023-06-12T074402Z when using the System Upgrade Controller; was there some issue with the update channel at that time?

And again related, is there a recommended way to apply an upgrade freeze or only upgrade during a specific time window? I’m thinking about labelling the nodes during the window so that they can be matched in the Plan only during the window.

hi. I am upgrading rke2 and it stucked for cannot connect to containerd. The steps and related output are listed below:

$ curl -sfL <https://get.rke2.io> | INSTALL_RKE2_CHANNEL=stable sh - [INFO]  using v1.25.10+rke2r1 as release
$ systemctl restart rke2-server
---
$ journalctl -u rke2-server -f
Jun 15 11:48:46 kmaster-gpu01 rke2[3072264]: time="2023-06-15T11:48:46+08:00" level=info msg="Waiting for API server to become available"

Jun 15 11:48:46 kmaster-gpu01 rke2[3072264]: time="2023-06-15T11:48:46+08:00" level=info msg="Waiting for cri connection: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /run/k3s/containerd/containerd.sock: connect: connection refused\""

and when I step back to the origin 1.24 version of rke2, it worked.

$ curl -sfL <https://rancher-mirror.rancher.cn/rke2/install.sh> | INSTALL_RKE2_VERSION="v1.24.10+rke2r1" sh -
$ systemctl restart rke2-server # things work fine

is there any guide for upgrading rke2 from 1.24 to 1.25, what have I missed. great thansk.

Hi I am using v1.24.12+rke2r1 for all server+agent. In the environment observed pod

rke2-metrics-server-xxx

ends crashlooping with following error.

panic: failed to create listener: failed to listen on 0.0.0.0:10250: listen tcp 0.0.0.0:10250: bind: address already in use

goroutine 1 [running]:

Looks this 10250 comes from https://github.com/rancher/rke2-charts/blob/main-source/packages/rke2-metrics-serv[…]ed-changes/patch/templates/metrics-server-deployment.yaml.patch

<poof>

Hey! We have setup authorized cluster endpoint on Custom RKE2 cluster but when we try to connect against it with kubectl we get

I0615 16:22:52.043378   49657 versioner.go:58] the server has asked for the client to provide credentials error: You must be logged in to the server (Unauthorized)

And the pod kube-api-auth log things like:

level=error msg="<http://clusterauthtokens.cluster.cattle.io|clusterauthtokens.cluster.cattle.io> \"cattle-system/token-v8rtt\" not found"

error msg="<http://clusterauthtokens.cluster.cattle.io|clusterauthtokens.cluster.cattle.io> \"cattle-system/kubeconfig-u-s34vszhuvdkh6k2\" not found"

I have created a loadbalanced DNS record containing the control plane nodes and added that DNS-name to tls-san config for the cluster. Anyone have any idea what to do?

Hello! We upgraded a three node rke2 cluster from v1.21.9-rke2r1 to v1.24.14-rke2r1 and now have both the kube-proxy static pods and the helm installed kube-proxy pods running. We do not have any HelmChartConfigs as part of this cluster. It is my understanding that the helm chart should have been disabled and I see the following log message in one of the nodes:

rke2[3215805]: time="2023-06-02T11:10:58-05:00" level=info msg="rke2-kube-proxy HelmChartConfig not found, disabling rke2-kube-proxy"

Is there anything else that needs to happen other than manually uninstalling rke2-kube-proxy chart?

can multus be enabled with windows+calico clusters?

Hi @creamy-pencil-82913, Is Calico or Cilium CNI needed for managing multiple rke2 clusters? possibly using also ipv6

using the calico cni (migrated from a manual cilium install) and now pods in the cluster cannot reach the LB ip serviced by the cluster. It appears calico is trying to do something fancy with the packets (as in, I don't see the packet leave the host nic and get snat'd). Anyone ever seen this and know what's going on? how to resolve that?

is there a way to cancel update plan triggered from rancher ui. thanks

If I am installing RKE2 with RPM method, where do I put my config.yaml? It used to be /etc/rancher/config.yaml but with v1.25.10 that config did not get read

A question about rke2 binaries. I found when installing, rke2 script will extract out some bins like

containerd

,

kubelet

in a location of

/var/lib/rancher/rke2/bin

. However, the bootstrap process will also download an image named

rke2-runtime

and extract out the same set of binaries into the location of

/var/lib/rancher/rke2/data/bin

. and the later ones are the running ones. Anybody knows the user of the first ones. They seem to be never used. It’s quiet confusing. Thanks.

Howdy! I’m toying with autoscaling in RKE2. There doesn’t seem to be much in the way of good documentation around how to get it working. I’m using vSphere as my infrastructure provider and have provisioned my RKE2 cluster via Rancher Server. The documentation on Rancher’s website is specifically geared toward EC2 implementations using a “custom” cluster type: https://ranchermanager.docs.rancher.com/v2.7/pages-for-subheaders/install-cluster-autoscaler This documentation for cluster-autoscaler on Github suggests I need to create a config.yaml and start the autoscaler as if I was running it directly from my CLI: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/rancher/README.md This article from earlier this month suggests it’s built-in functionality, but instructions make no sense; as the options the article is talking about do not exist in the Rancher UI (could be typos, but still can’t figure out what the writer is talking about): https://macsources.com/how-to-autoscale-kubernetes-cluster-with-rancher/ And, this is out of date (options it states like nodePoolName are not present in my cluster config): https://vsys.host/how-to/kubernetes-cluster-autoscaling-with-rancher My question… How does cluster autoscaling work in RKE2? Where is complete documentation I can reference to get it working?

how can I force an addon to reinstall?

lol

Hi, how are you ? I have an issue with rke2 😅 (it work as excepted with k3s) • I'm using metalLB to provision IPs (10.0.20.1) of services but I can't acces to it from the LAN (needed to NAT as example 80/443 public traffic to this IPs) • when I do from LAN a curl to 10.0.20.1 I'm able to see ARP packet with tcpdump (

06:46:19.170827 ARP, Request who-has 10.0.20.1 tell 10.0.255.192, length 46

) but the curl timeout / does't reach the service • when I do from any node a curl to 10.0.20.1 I have the excepted result I tried to check iptables rules on the host without luck

I'm at a bit of a loss for a behaviour I'm seeing on a 1.24.13 cluster with cilium, and I'm not quite sure even how to put together a report, because the various tools aren't giving me anything that's suggesting an error RHEL 8.8, RKE2 1.24.13, Cilium, 72 nodes, 6 of them dedicated controllers The behaviour I'm witnessing is: • When the system is stable and pods are running, everything is fine • When a HPA kicks in and starts dynamically scaling replicas up and down, over time we see new pods being unable to connect to remote services • Traffic that stays node-local always works • It's not service specific, and removing network policy has no effect • Restarting a pod can induce the error sometimes • There are no logged cilium errors, but usually restarting cilium fixes it • From what I've been able to determine, it's the SYN/ACK part of a TCP handshake that never arrives back at the originating pod • The node that runs the pod that cannot connect to a remote endpoint, is the one that needs cilium restarted • It's ~ 1210 pods across 72 nodes, most are long lived • There is no host firewall configured • Cilium endpoint health usually reports reachable, as these are long lived pods and paths What it looks like is an incoming firewall is denying the SYN/ACK of a session setup, which leads me to believe Cilium isn't setting something up right

We did have some etcd latency last week, compacting and defragging the etcd instances stopped the etcd warnings, but this doesn't appear to have made a real difference

My metallb is broken, turns out ipvs is broken? https://github.com/rancher/rke2/issues/4120

$ rke2 -v
rke2 version v1.25.9+rke2r1 (842d05e64bcbf78552f1db0b32700b8faea403a0)
go version go1.19.8 X:boringcrypto

I'm on v2.7.4, is that issue above it in 2.7.5?