Has anyone tried to change the template of an RKE2 cluster agent-pool on vSphere? It attempts to upgrade and create a node, but just stops there. Doesn’t register the node or do anything. I’m able to manually delete a node and it’ll auto-gen a new one using the old template, but it doesn’t affect the update process. Update process is broken.

I am trying to get an RKE2 cluster up and running, but I get into trouble when I try to install

cert-manager

, or more specifically when I define

ClusterIssuer

which in turn trigger the webhook

cert-manager-webhook

. As it is the API server that want to connect to the webhook, it needs two things: • Be able to resolve the DNS address of the URL https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s - for which it needs configuration and reachability to the CoreDNS service. • Then it needs to be able to reach the service endpoint as well. Neither of these two works. What puzzles me is that the API server runs as a host-networked pod, and therefore lacks the DNS config needed, it just has config that points to the external name server (e.g. as if the ClusterFirstWithHostNet setting was not set on the pod). Secondly I do not known how to debug the issue with not being able to reach the service (must be some kind of iptables misconfiguration?). All help / ideas are welcome! Thanks!

I have 2 RKE2 v1.25.9 clusters, both are new clusters created from RancherUI (2.7.3) Both have 6 nodes (3xcontrol plane/etcd and 3 workers) Both are RHEL 8.7... On both clusters, after cluster creation, I went into Edit YAML and set

cloud-provider-name: external

However, when I do

kubectl get nodes -o wide

, on one cluster I see both Internal-IP and External-IP populated (in this case both are same which is expected). But on second cluster, I have Internal-IP populated but for External-IP it shows <none>... I am trying to figure out why is this different?

This message was deleted.

Hello Team, I am trying to setup RKE on 4 CentOS Linux release 7.9.2009 (Core) VMs with 4 Cores and 8GB RAM. Configuration is 1 master node and 3 worker nodes. rke up ran fine without errors but I have issues with coredns not running which I think is caused by calico-kube-controllers in crash loop back off. The error from calico is that it cannot reach https://10.43.0.1:443/apis with a

no route to host

error - can curl that ip on all my nodes (including master). I thought it might have been SELinux so I disabled that but still the same issue. I've attached my cluster.yaml and some logs.

Hello, we are using RKE1 with external etcd server. How can I install rke2 also using the external etcd server. In the docs cannot find any information? thanks

@acoustic-motherboard-98931 has left the channel

I can't really find information regarding this, but what is the correct way to reboot a node running rke2-server? It seems the containers are not stopped when stopping rke2-server service, so the reboot takes ages because of service timeouts...

Whenever i stop rke2-server to upgrade, rke2 still runs stuff in the background, so much that i kill it (due to the size of my control plane nodes). Am i missing something or should systemctl stop rke2-server actually stop everything?

Okay i just read the question above me with the hint of

rke2-killall.sh

Now, why is it like that though?

anyone running or toyed with running an rke2 on prem metal cluster with an auto-scaler to something like azure?

hi!! did anyone have this problem?

"couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request".

i haven t done anything, it s the default metrics server. i only installed the cluster and after deploying some apps this error started to occure after every kubectl command. also in api-service

<http://v1beta1.metrics.k8s.io|v1beta1.metrics.k8s.io> , not local but in kubesystem/rke2-metrics-server and false(failedDiscoveryCheck)

. 😬

Hi. I'm testing RKE2 (Rancher v.2.7.3) on Rocky v9.1. A cluster was created on RKE2 webui. I checked "Control Plane" and copied the registration string. I paste it on another Rocky VM (same version) and wait for provision. RKE web interface show: "Read: False - [Disconnected] Cluster agent is not connected". Rocky VM is SELinux and firewalld disabled. CP VM and RKE2 webui are on the same network. Any tip?

[root@k8s-rancher-rocky-cp1 ~]# journalctl -u rancher-system-agent
May 15 08:24:32 k8s-rancher-rocky-cp1 systemd[1]: Started Rancher System Agent.
May 15 08:24:32 k8s-rancher-rocky-cp1 rancher-system-agent[5319]: time="2023-05-15T08:24:32-03:00" level=info msg="Rancher System Agent version v0.3.2 (afbc4aa) is starting"
May 15 08:24:32 k8s-rancher-rocky-cp1 rancher-system-agent[5319]: time="2023-05-15T08:24:32-03:00" level=info msg="Using directory /var/lib/rancher/agent/work for work"
May 15 08:24:32 k8s-rancher-rocky-cp1 rancher-system-agent[5319]: time="2023-05-15T08:24:32-03:00" level=info msg="Starting remote watch of plans"
May 15 08:24:32 k8s-rancher-rocky-cp1 rancher-system-agent[5319]: E0515 08:24:32.466285    5319 memcache.go:206] couldn't get resource list for <http://management.cattle.io/v3|management.cattle.io/v3>:
May 15 08:24:32 k8s-rancher-rocky-cp1 rancher-system-agent[5319]: time="2023-05-15T08:24:32-03:00" level=info msg="Starting /v1, Kind=Secret controller"

Hi all, I am new to this channel and I am delighted to be here.

My org running is running rancher 2.6.3 and RKE2 1.21.5+rke2r1, we are planning on doing the upgrade to rancher 2.6.5 and rke2 to latest, would love to know if there is a documentation on how to do it. I have experience doing on RKE so one of the important question is are the commands and process same?

hi, I have 3 master node + 6 worker node as a cluster. I need to release 3 master node and convert existing 3 nodes in worker to master. To do it I have killed 2agent node and drained 2 master. Then started rke2-server services in the old agent nodes.

Hi, Can someone please help with docker-machine command for rke2? Example docker machine command for rke1 docker-machine -D create -d oci --engine-install-url https://releases.rancher.com/install-docker/20.10.sh

hello!! 🙂 can someone please give me a rke2 metrics-server yaml?

I had a windows server 2019 worker node on RKE2 cluster. Wanted to remove this windows node, so I ran

rke2-uninstall.ps1

However this did not clean up things under

c:\var\lib\

Is there a different (proper) way to do uninstall on Windows nodes?

Does anybody knows how to run rke2 without rancher-ui?

how can I force rke2 to attempt to install coredns? not even sure why it's removed in the first place but then it failed a deploy job with

+ helm_v3 install --set-string global.clusterCIDR=10.42.0.0/16 --set-string global.clusterCIDRv4=10.42.0.0/16 --set-string global.clusterDNS=10.43.0.10 --set-string global.clusterDomain=cluster.local --set-string global.rke2DataDir=/var/lib/rancher/rke2 --set-string global.serviceCIDR=10.43.0.0/16 rke2-coredns /tmp/rke2-coredns.tgz --values /config/values-10_HelmChartConfig.yaml
Error: INSTALLATION FAILED: cannot re-use a name that is still in use

Hi, my org is running RKE2 on v1.21.5+rke2r1, so I have couple of questions please: 1. from v1.21.5 can we do v1.21.14 rite? 2. from v1.21.14 can we directly upgrade to v1.22.17+rke2r1 3. the upgrade will be done on both control plane and worder nodes?

Hi, using Rancher 2.6.9 created on-prem RKE2 k8s cluster v1.24.10 with a mix of Windows and Linux worker nodes. After sometime we started seeing disk space usage was close to 80%. Looking through the Kubernetes documentation I was able to find some info regarding Garbage Collection. Especially •

HighThresholdPercent

•

LowThresholdPercent

These values are set to 85% and 80% percent accordingly. Any suggestion where can I change these values?

Has anyone tried adding multiple network interfaces (multiple vlans) to their nodes with vSphere? I’m noticing it’s adding them to the VM in lexicographic type of order and not the order you add them in from the GUI. ie. you add two networks,

299 - core

and

295 - services

, but 295 ends up being the primary nic on the VM and 299 secondary. This is on Rancher 2.7.0.

I am trying to create a cluster via rancher ui rke2. But it's in Reconciling state from more than 12 hours. Any idea how to debug whats the issue? Provisioning logs:- There are no provisioning log entries.

does rke2 + calico + windows support standard k8s network policies?

Hi folks. I deployed a downstream cluster(rke2 1.25.7) following the rke2 hardening guide https://ranchermanager.docs.rancher.com/pages-for-subheaders/rke2-hardening-guide. The management cluster is also rke2 1.25.7 and rancher version is 2.7.3.

spec:
  defaultPodSecurityAdmissionConfigurationTemplateName: rancher-restricted
  rkeConfig:
    machineSelectorConfig:
      - config:
          profile: cis-1.23
          protect-kernel-defaults: true

Then I tried CIS scan using

rke2-cis-1.23-profile-hardened

profile following this guide https://ranchermanager.docs.rancher.com/pages-for-subheaders/cis-scan-guides and got this result. Is this expected result? PS: CIS benchmark app version is 4.0.0

I filed a github issue also with more verbose words https://github.com/rancher/cis-operator/issues/196

Hello! Following my initial message on #general, I'm redirecting my RKE2 specific questions here in a thread

is it possible to config api-server

dnsPolicy

, currently it is set to

ClusterFirst

and since api-server is static pod using hostnetwork, the

dnsPolicy

will fallback to

default

behaviour https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/