Hi, can you tell me if rancher v2.12.0 on k8s does not support external mysql now?

👋 Hi, I'm working on certification of Rancher/k3s for the ECK operator for Elastic and I'm running into a bit of an issue with cgroups. It seems to be this user's exact issue reported here: https://github.com/k3s-io/k3s/issues/2500. The environment that we are running within is k3d/k3s and appears to be "hybrid" (link) . This

/sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us

is set to

-1

in our environment, so cpu limits simply aren't supported. We have a nearly identical set of e2e tests that we have been running for a long period of time using

kind

that doesn't present this issue. Are you aware of any known issues with this type of setup? I'm happy to provide additional information as requested. Thanks in advance for the assist!

Hi everyone. SUSE is hiring in the United States. We are seeking qualified candidates in the USA to join our Premium Support organization, which will support SUSE Rancher solutions. If you are interested, please take a look at the job posting below or feel free to reach out to me directly. https://suse.wd3.myworkdayjobs.com/en-US/Jobsatsuse/job/Pennsylvania-United-States-of-America/Premium-Support-Engineer_71007097

Howdy I'm having trouble adding nodes to an existing single node cluster. I set up a single node cluster using

sh ~/k3s server --flannel-backend wireguard-native --node-name someNodeName --cluster-init

and that seems to work, however the server (in

/etc/rancher/k3s/k3s.yaml

) is set to

<https://127.0.0.1:6443>

When I try join another server node to this cluster, using

sh ~/k3s server --flannel-backend wireguard-native --node-name someNodeName --server <https://192.168.88.212:6443>

(where 192.168.88.212 is the IP of the above single node) And I get the following from

journalctl

time="2025-08-21T02:20:23Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token"
time="2025-08-21T02:20:23Z" level=info msg="To join server node to cluster: k3s server -s <https://192.168.88.211:6443> -t ${SERVER_NODE_TOKEN}"
time="2025-08-21T02:20:23Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token"
time="2025-08-21T02:20:23Z" level=info msg="To join agent node to cluster: k3s agent -s <https://192.168.88.211:6443> -t ${AGENT_NODE_TOKEN}"
time="2025-08-21T02:20:23Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
time="2025-08-21T02:20:23Z" level=info msg="Run: k3s kubectl"
time="2025-08-21T02:20:24Z" level=info msg="Password verified locally for node somenodename"
time="2025-08-21T02:20:24Z" level=info msg="certificate CN=somenodename signed by CN=k3s-server-ca@1755699413: notBefore=2025-08-20 14:16:53 +0000 UTC notAfter=2026-08-21 02:20:24 +0000 UTC"
time="2025-08-21T02:20:24Z" level=info msg="certificate CN=system:node:somenodename,O=system:nodes signed by CN=k3s-client-ca@1755699413: notBefore=2025-08-20 14:16:53 +0000 UTC notAfter=2026-08-21 02:20:24 +0000 UTC"
time="2025-08-21T02:20:24Z" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1755699413: notBefore=2025-08-20 14:16:53 +0000 UTC notAfter=2026-08-21 02:20:24 +0000 UTC"
time="2025-08-21T02:20:24Z" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@1755699413: notBefore=2025-08-20 14:16:53 +0000 UTC notAfter=2026-08-21 02:20:24 +0000 UTC"
time="2025-08-21T02:20:24Z" level=info msg="Module overlay was already loaded"
time="2025-08-21T02:20:24Z" level=info msg="Module nf_conntrack was already loaded"
time="2025-08-21T02:20:24Z" level=info msg="Module br_netfilter was already loaded"
time="2025-08-21T02:20:24Z" level=info msg="Module iptable_nat was already loaded"
time="2025-08-21T02:20:24Z" level=info msg="Module iptable_filter was already loaded"
time="2025-08-21T02:20:24Z" level=warning msg="Failed to load kernel module nft-expr-counter with modprobe"
time="2025-08-21T02:20:24Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
time="2025-08-21T02:20:24Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
time="2025-08-21T02:20:25Z" level=info msg="containerd is now running"
time="2025-08-21T02:20:25Z" level=info msg="Creating k3s-cert-monitor event broadcaster"
time="2025-08-21T02:20:25Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=somenodename --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=192.168.88.211 --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
time="2025-08-21T02:20:25Z" level=info msg="Connecting to proxy" url="<wss://127.0.0.1:6443/v1-k3s/connect>"
time="2025-08-21T02:20:25Z" level=info msg="Handling backend connection request [somenodename]"
time="2025-08-21T02:20:25Z" level=info msg="Remotedialer connected to proxy" url="<wss://127.0.0.1:6443/v1-k3s/connect>"
time="2025-08-21T02:20:25Z" level=error msg="Sending HTTP/1.1 503 response to 127.0.0.1:35958: runtime core not ready"
time="2025-08-21T02:20:25Z" level=info msg="Adding member somenodename-c230b040=<https://192.168.88.211:2380> to etcd cluster [lilz-kubernetes-1-db6c7708=<https://192.168.88.212:2380>]"
time="2025-08-21T02:20:25Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=somenodename --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables"
E0821 02:20:25.900707   12934 server.go:1039] "Failed to retrieve node info" err="apiserver not ready"
E0821 02:20:27.001367   12934 server.go:1039] "Failed to retrieve node info" err="apiserver not ready"
time="2025-08-21T02:20:28Z" level=info msg="Failed to test data store connection: failed to get etcd status: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused\""
E0821 02:20:29.390830   12934 server.go:1039] "Failed to retrieve node info" err="apiserver not ready"

Any insights would be greatly appreciated.

hi, I just start new Rancher server with new kube cluster base on 2.12.0. On cluster / tools, the longhorn is not displayed ? I have check on reposiroty configuration. Il use branch release-2.12. And when I look on git repository, on branch release-2.12, I can look longhorn chart. So have you any idea of the hell ?

Hey guys, trying to run rke2 with cilium and I am reading it should be bundled in rke2-images.linux-amd64.tar.gz but it's not in there. Is this on purpose or am I missing something?

Hello 👋 I'm trying to get the rancher desktop on Mac to run a k8s cluster behind a zscaler proxy. It is getting blocked because it cannot download k3s from the registry (logs throw fetchError because unable to get local issuer cert). I've obtained a copy of our custom root CA cert from my local admin, but how do I run the desktop app with these loaded on the application path?

Hey, I used K8S for quite some time and am now doing the switch over to K3S for my home. I am struggling since days with networking issues. At first, MariaDB could not contact a certain domain (context deadline exceeded) Now paperless cannot connect to a postgres database (connection timeout expired) I checked with busybox, that domains resolve. Also tried with <namespace>.svc / .cluster.local and directly with the IP apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-all-ingress namespace: storage spec: podSelector: {} ingress: - {} policyTypes: - Ingress I also created a "all" networkPolicy. But whatever I try and with every container they do not seem to be able to communicate. I disabled firewall completely. kube-system coredns-65577974f-clktm 1/1 Running 0 10m kube-system csi-nfs-controller-5bf646f7cc-c4r46 5/5 Running 0 10m kube-system csi-nfs-node-gtp7q 3/3 Running 0 10m kube-system csi-nfs-node-wrd6v 3/3 Running 0 10m kube-system local-path-provisioner-774c6665dc-vrcqm 1/1 Running 0 10m kube-system metrics-server-6f4c6675d5-6vc9l 1/1 Running 0 10m kube-system svclb-pg-cluster-postgresql-3c437cf4-96h9g 1/1 Running 0 10m kube-system svclb-pg-cluster-postgresql-3c437cf4-dt7gm 1/1 Running 0 10m kube-system svclb-traefik-9a5113ec-68kdc 2/2 Running 0 10m kube-system svclb-traefik-9a5113ec-qh82g 2/2 Running 0 10m kube-system traefik-c98fdf6fb-79gh5 1/1 Running 0 10m With KubeSystem everything runs correctly. I tried deleting every pod and restarting the whole cluster. Any idead? Help would be greatly appreciated. Thank you !!

Addition: I can resolve External IPs into the cluster without problems

I do not see any flannel pod in the list. I suspect, that k3s installed without further networking I installed it with the quickstart option:

curl -sfL <https://get.k3s.io> | sh -

No. Flannel does not run as a pod. flannel and kube-router network policy controller run in the main k3s process, same as etcd, kubelet, apiserver, and all the other core components.

you can see this if you look at the logs.

hey guys, I'll really appreciate help for this! I'm installing a CloudNative Postgres Operator using ansible, and after installing it, i also try to create a cluster, but then i receive this error for example on my RKE2 cluster: Failed to create object: b''{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: failed calling webhook \\"mcluster.cnpg.io\\": failed to call webhook: Post \\"https://cnpg-webhook-service.cnpg-system.svc:443/mutate-postgresql-cnpg-io-v1-cluster?timeout=10s\\": proxy error from 127.0.0.1:9345 while dialing <serverip>:9443, code 502: 502 Bad Gateway","reason":"InternalError","details":{"causes":[{"message":"failed calling webhook \\"mcluster.cnpg.io\\": failed to call webhook: Post \\"https://cnpg-webhook-service.cnpg-system.svc:443/mutate-postgresql-cnpg-io-v1-cluster?timeout=10s\\": proxy error from 127.0.0.1:9345 while dialing <serverip>:9443, code 502: 502 Bad Gateway"}]},"code":500}\n''' reason: Internal Server Error: now thing is - at the beginning i took my RKE2 nodes' rke2.yaml and changed the "server:" value of it under the "clusters:", and changed it from 127.0.0.1 to my <serverip> value, and restarted the rke2 service and it made it work! however now when i try to do so, the "server:" value keeps getting reverted back to 127.0.0.1 so I figured it may not be the best approach to fix it. I will appreciate any help to what I can do here and why it may happen, thanks alot!

Hi. I have a single node RKE2 cluster. I would like to have a static container running on the master node, that - for example - provides an emergency web ui where I can run a script (... rke2 server --cluster-reset) on the server by pressing a button. Calling shell scripts instead of using a web ui is not possible for my use case. I don't want to use RKE2 for that (also no static pods), because RKE2 could break and "rke2 server --cluster-reset" is stopping all containers and also containerd and therefore RKE2. Is it even possible to do that? e.g. Podman and RKE2 running in parallel? Or a simpler method?

It looks like this Bug on GitHub might be missing a few labels and isn't on the radar for the dev team. Is anything missing I can add or edit for it? (If someone knows where in the codebase this is I don't mind submitting a PR, I'm just not a front end dev so quality may vary lol)

Does anyone know approximately when release v2.12.0 will be declared stable?

Hi, i'm looking to move a couple of imported RKE2 clusters from one Rancher to another - are there any concerns with this? Perhaps some guide or docs around it?

Hi team, I was wondering if anyone else has experienced a similar issue where users are creating API tokens and the enabled field on the tokens is being switched to false. It's causing users to have to re-create tokens constantly. For reference we are on rancher 2.9.x and use LDAP for auth. Appreciate any suggestions on how to move forward, thanks!

Hi eveyone, I have an upstream Rancher cluster configured with a single domain (e.g., rancher.com) and a downstream cluster (cluster01) where all subdomains (*.rancher.com) are routed to the LoadBalancer on cluster01. I am planning to create a second downstream cluster under the same upstream Rancher. I’m looking for documentation or guidance on setting up this second cluster so that services and load balancing work correctly. Specifically: Do I need to create separate domains for each downstream cluster? How can I configure routing so that user URLs reach the correct cluster? For example: foo.cluster01.rancher.com → cluster01 foo.cluster02.rancher.com → cluster02 I am new to Rancher and Kubernetes, so I apologize for any gaps in my knowledge. I am happy to provide any additional information if needed — just let me know what would be helpful. Thank you!

Hi there! Just filed a bugreport in 2.12.0 witch feels pretty severe: https://github.com/rancher/rancher/issues/51685 Have confirmed with nutanix prepresentatives what this is reproduceable using the Nutanix Node driver 3.6.0 and the Nutanix UI Extension 1.1.0 and Rancher 2.12.0. Worked like a charm in 2.11.x More info at github issue tracking:

Hi, can anyone help me how to safely fix this error on one of the three prod rancher nodes? ,"msg":"request sent was ignored by remote peer due to cluster ID mismatch", However my rancher cluster is up and functioning . Not sure if the quorum is 2 for 3 nodes deployment.

I currently have all my clusters updating simultaneously on Rancher 2.12.0, all seemingly stuck with the message "KDM release data is empty for v1.33.3+rke2r1" (or slight variation in release), all with one controller stating "Reconciling. Waiting for plan to be applied". Should I be worried?

Running RKE2 with a bridge as the primary interface for multis. I see the docs want firewalld disabled but I don't want non cluster members to have access to the etcd port, etc. Any solutions here?

Unable to Record UI Events with k6 Extension / Studio #778 Hi team, We are using the k6 extension / k6 browser studio for recording, but we're currently unable to capture UI interactions during the browser recording. The tool is only capturing network requests, but not frontend events like clicks, text inputs, or other user interactions. Our goal is to record actual UI events for test generation. Could you please guide us on how to enable or achieve this? Below is the issue link i had raised with k6 studio https://github.com/grafana/k6-studio/issues/778

Have anyone here ever tried to setup/install SUSE private registry in air gapped mode ? I see very less documentation on suse private registry.

Howdy again! This time to announce that Rancher v2.12.1 it out! https://github.com/rancher/rancher/releases/tag/v2.12.1

what would cause rancher to stop displaying the pods of a statefulset? if I inspect the network requests, I can see that the api is returning the pods

Hi, has anyone experienced the login loop on SLES 15 SP6? On terminal you can access it but on gui only the root user who is able to login .

Hey, i tired to upgrade my longhorn install and along the way thing went wrong. Rancher now thinks it should upgrade from 1.7 to 1.9 while i have 1.6.4 installed and running. How can i get rancher to recognise whats the actual version installed and offer the other upgrade versions (from 1.6.4 to 1.7.3)? I'm kinda lost and google is not helping 😞