I need to know if RKE2 support CephFS like it does with RBD.

how to configure rke2 integrated coredns to forward other upstream server. thank you

Hi everyone, I have a haproxy that load balance requests to my kubernetes cluster, in order to keep the IP address of the client I used the

send-proxy

param in haproxy, and I used the

use-proxy-protocol

config for my rke2-nginx-ingress-controller. Unfortunately when I do this my services under nodeport are no longer working, and I also want to keep the ip for the nodeport in order to apply network policies based on client ip, any idea how to fix maybe the proxy or my nodeport so it works ?

Prolly already asked but how do I safely restart my control plane and worker nodes? https://www.suse.com/support/kb/doc/?id=000020031 is what I looking at and wondering if this is the same method

how to correctly change rke2 helmchart manifest on a HA cluster. I means a have 3 master/server node, and if I want to change some settings for rke2 default integrated ingress-nginx. How should I shutdown some master nodes and modify the helmchartconfig yaml files in what kind of the order? thanks

Hey, what's the intended way to run the containernetworking DHCP plugin daemon when using RKE2?

I was able to get multus CNI working by modifying the RKE2 server config.yaml like so:

cni:
- multus
- canal

But the only ways I could figure out to get the IPAM DHCP plugin working were by either: • manually installing the systemd units from https://github.com/containernetworking/plugins/tree/main/plugins/ipam/dhcp/systemd -- which is less-than-ideal for doing it at scale • installing one of the dhcp-daemonset manifests you can find in various projects, e.g. https://github.com/k8snetworkplumbingwg/reference-deployment/tree/master/multus-dhcp -- which does fully solve the issue for me, but seems like RKE2 should be able to do this itself?

Getting the containernetworking dhcp plugin daemon running seems to be a common point of confusion, and even if it doesn't make sense for RKE2 to supply its own copy of this, it'd probably be good to at least document how to get it to work in the RKE2 install directions (in with the other network options)

Hi folks, I am trying to setup HA RKE2 cluster (advertise-address is load-balancer address) using Cilium CNI, without kube-proxy component from stable channel on Ubuntu 22.04.2. Internet access via HTTP-proxy. First master node created. Problem: all pods started, but CoreDNS failing probes. Can you give me any hint to find the problem?

Events:                                                                                                                                                                                                                                                       
  Type     Reason     Age                     From     Message                                                                                                                                                                                                
  ----     ------     ----                    ----     -------                                                                                                                                                                                                
  Warning  Unhealthy  22m (x1150 over 19h)    kubelet  Liveness probe failed: Get "<http://10.42.0.160:8080/health>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)                                                                
  Warning  BackOff    7m27s (x3234 over 19h)  kubelet  Back-off restarting failed container                                                                                                                                                                   
  Warning  Unhealthy  2m27s (x3001 over 19h)  kubelet  Readiness probe failed: Get "<http://10.42.0.160:8181/ready>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Configuration:

write-kubeconfig-mode: "0644"
advertise-address:
  - 192.168.0.12
cni:
  - cilium
disable-kube-proxy: true
disable:
  - rke2-canal
  - rke2-kube-proxy
# Make a etcd snapshot every 6 hours
etcd-snapshot-schedule-cron: " */6 * * *"
# Keep 56 etcd snapshorts (equals to 2 weeks with 6 a day)
etcd-snapshot-retention: 56
node-taint:
  - "CriticalAddonsOnly=true:NoExecute"
tls-san:
  - 192.168.0.12

Hi, I would like to use metallb on RKE2 provisioned cluster. I have to add this configuration to

kube-proxy

:

kubeproxy:
  extra_args:
  ipvs-strict-arp: 'true'
  proxy-mode: ipvs

But looks like there is no config map for kube-proxy. When I try to edit

cluster.yaml

to add custom config to kubeproxy I'm not sure where I should add this cfg. Could you help me, please? 🙂

Hi everyone, I have a service that I want to expose and I do it with NodePort. I want to restrict the IP address of who can access to the service, but the IP that I get is 10.something, so I can't restrict with network policy using my IP. I have read that in order to get the client IP using nodeport, we have to set

externalTrafficPolicy

to Local in the Nodeport definition, unfortunately when I do this I can't access my service even without NetworkPolicy. Does someone have any idea on how to fix that ? Thank you !

Hello, I hope for you a happy day if you reached here! I'm trying to install RKE2 on a machine that have 2 network interfaces 10.x.x.x and 172.x.x.x and when I finish the installation ETCD advertise it's URL through the 10.x.x.x interface and i want to change it to the 172.x.x.x interface. Is there a way to do so? P.S: the list of things i tried already: 1- https://docs.rke2.io/reference/server_config 2- exec into ETCD pod and change it manually https://manpages.org/etcd Any recommendation?

Hi, Is it possible to install RKE2 cluster with Kube-vip from within rancher instead installing RKE2 with kube-vip prior rancher is installed on the nodes? If so, would you happen to have a video and steps? Thank you.

@acceptable-air-49600 has left the channel

Hi, Is it possible to install RKE2 cluster with Kube-vip from within rancher instead installing RKE2 with kube-vip prior rancher is installed on the nodes? If so, would you happen to have a video and steps? Thank you. Posted in #rke2

Hello. Are these docs current? The "mirrors" section doesn't appear to reflect reality. Specifically, the "endpoint" field seems to be completely disregarded by containerd, despite it being added to the generated toml file, and it always uses the registry component of the image id as the hostname. The "configs" section works just fine. Did "mirrors" get deprecated or something?

hi guys, how to override registry endpoint in rke2. I do create a

registries.yaml

and try to put some things like below to make the kubernetes registry point to own harbor proxy cache, but it doesn’t work and the containerd still tries to pull image from the original one. What’s wrong with my configuration? thanks.

Good morning. Does anyone have any recommendations for tools to copy images from a private registry to the RKE2 containerd backend? I tried skopeo but it doesn’t look like containerd is supported https://github.com/containers/skopeo/issues/1683 I know ctr can import images, but I was looking for something that can be ran from one VM hosting the private registry to connect to vms/nodes in a separate cluster remotely. Thanks in advance

my gpu operator on rke2 can find only 6 gpus on worker nodes, however the nodes have 8 gpus each. Is anyone happened to the same issue? thank you

No stupid questions policy: We had someone attempt to jump straight from 1.22 to 1.24 instead of 1.22->1.23->1.24, and expect that this might be causing some issues. (installed latest rke2-stable on new nodes, then joined the cluster, then updated rke2 on the older nodes to match). Is this actually a problem? can the cluster be downgraded to 1.23 then back up to 1.24? There is minimal guidance on how to handle a skipped kubernetes version in general. A little bit of unknown territory here

Is there a possibility to upgrade rke2 with no downtime in a multi-node HA?

Hi! Thats my case: https://github.com/rancher/rke2/issues/3710 I would like to know, where exactly should I add this:

kube-proxy-arg:
  - proxy-mode=ipvs
  - ipvs-strict-arp=true

in

cluster.yaml

. I mean in which section of yaml

Hey what are the differences between the normal kubernetes and rke?

I ran

rke2 server --server <https://172.16.1.11:6443>

and the master didn't join the cluster, what could be the issue?

Good morning all! I’m really struggling with this for the past couple of weeks and would love any insight. I’m deploying RKE2 on AWS using AMIs that I automate the build and deployment of with packer. I have complete automation and minimization of the configuration of the RHEL 8.5 OS from iso minimal install, setting up the

etcd

user, etc…. I’m deploying in a 3-master HA configuration. What I am seeing is sometimes (not all) one of the master nodes (could be A, B or C) will hang. I am unable to SSH in, I’m unable to run any commands against the node. The node never recovers. I have to login to AWS console and manually stop and start the node, at which point it joins and I can add workers. I am trying to ensure that I am compliant with the supported runtime and requirements exactly as they should be as this will be a large production cluster and need to ensure it is exactly as needed. Anyone else had this issue? Anyone else seen this before? My process is: 1) I start Master A, wait 5m 2) Add Join Token to B/C 3) Start Master B, wait 5m 4) Start Master C, wait 5m… The entire process is automated with ansible. As I stated, this doesn’t happen every time. One thing I was doing is SSH into Master A and run

watch -n1 "kubectl get nodes"

so I’m not sure if querying the API server while things are getting going would have this effect. I also provided all of the logs to AWS support for them to review OOM errors, but nothing looked out of the ordinary. RKE2 Version:

1.26.0+rke2r2

OS:

RHEL 8.5 - minimal install

Should I maybe add more time (more than 5m) between the start of each Master node?

~a try before I give up and just spin up openshift instead. when I perform an rke up I am getting: INFO[0000] [dialer] Setup tunnel for host [192.168.2.252] WARN[0000] Failed to set up SSH tunneling for host [192.168.2.252]: Can't retrieve Docker Info: error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info": Unable to access node with address [192.168.2.252:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found cluster.yaml and other details to follow~

~tux@rancher:~> cat cluster.yml # If you intened to deploy Kubernetes in an air-gapped environment, # please consult the documentation on how to configure custom RKE images. nodes: - address: 192.168.2.252 port: "22" internal_address: "" role: - controlplane - worker - etcd hostname_override: "" user: tux docker_socket: /var/run/docker.sock ssh_key: /home/tux/.ssh/id_rsa ssh_key_path: "/home/tux/.ssh/id_rsa" ssh_cert: "" ssh_cert_path: "" labels: {} taints: [] services: etcd: image: "" extra_args: {} extra_binds: [] extra_env: [] win_extra_args: {} win_extra_binds: [] win_extra_env: [] external_urls: [] ca_cert: "" cert: "" key: "" path: "" uid: 0 gid: 0 snapshot: null retention: "" creation: "" backup_config: null kube-api: image: "" extra_args: {} extra_binds: [] extra_env: [] win_extra_args: {} win_extra_binds: [] win_extra_env: [] service_cluster_ip_range: 10.43.0.0/16 service_node_port_range: "" pod_security_policy: false always_pull_images: false secrets_encryption_config: null audit_log: null admission_configuration: null event_rate_limit: null kube-controller: image: "" extra_args: {} extra_binds: [] extra_env: [] win_extra_args: {} win_extra_binds: [] win_extra_env: [] cluster_cidr: 10.42.0.0/16 service_cluster_ip_range: 10.43.0.0/16 scheduler: image: "" extra_args: {} extra_binds: [] extra_env: [] win_extra_args: {} win_extra_binds: [] win_extra_env: [] kubelet: image: "" extra_args: {} extra_binds: [] extra_env: [] win_extra_args: {} win_extra_binds: [] win_extra_env: [] cluster_domain: cluster.local infra_container_image: "" cluster_dns_server: 10.43.0.10 fail_swap_on: false generate_serving_certificate: false kubeproxy: image: "" extra_args: {} extra_binds: [] extra_env: [] win_extra_args: {} win_extra_binds: [] win_extra_env: [] network: plugin: canal options: {} mtu: 0 node_selector: {} update_strategy: null tolerations: [] authentication: strategy: x509 sans: [] webhook: null addons: "" addons_include: [] system_images: etcd: rancher/coreos-etcd:v3.4.13-rancher1 alpine: rancher/rke-tools:v0.1.69 nginx_proxy: rancher/rke-tools:v0.1.69 cert_downloader: rancher/rke-tools:v0.1.69 kubernetes_services_sidecar: rancher/rke-tools:v0.1.69 kubedns: rancher/k8s-dns-kube-dns:1.15.10 dnsmasq: rancher/k8s-dns-dnsmasq-nanny:1.15.10 kubedns_sidecar: rancher/k8s-dns-sidecar:1.15.10 kubedns_autoscaler: rancher/cluster-proportional-autoscaler:1.8.1 coredns: rancher/coredns-coredns:1.7.0 coredns_autoscaler: rancher/cluster-proportional-autoscaler:1.8.1 nodelocal: rancher/k8s-dns-node-cache:1.15.13 kubernetes: rancher/hyperkube:v1.19.7-rancher1 flannel: rancher/coreos-flannel:v0.13.0-rancher1 flannel_cni: rancher/flannel-cni:v0.3.0-rancher6 calico_node: rancher/calico-node:v3.16.5 calico_cni: rancher/calico-cni:v3.16.5 calico_controllers: rancher/calico-kube-controllers:v3.16.5 calico_ctl: rancher/calico-ctl:v3.16.5 calico_flexvol: rancher/calico-pod2daemon-flexvol:v3.16.5 canal_node: rancher/calico-node:v3.16.5 canal_cni: rancher/calico-cni:v3.16.5 canal_controllers: rancher/calico-kube-controllers:v3.16.5 canal_flannel: rancher/coreos-flannel:v0.13.0-rancher1 canal_flexvol: rancher/calico-pod2daemon-flexvol:v3.16.5 weave_node: weaveworks/weave-kube:2.7.0 weave_cni: weaveworks/weave-npc:2.7.0 pod_infra_container: rancher/pause:3.2 ingress: rancher/nginx-ingress-controller:nginx-0.35.0-rancher2 ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.5-rancher1 metrics_server: rancher/metrics-server:v0.3.6 windows_pod_infra_container: rancher/kubelet-pause:v0.1.4 aci_cni_deploy_container: noiro/cnideploy:5.1.1.0.1ae238a aci_host_container: noiro/aci-containers-host:5.1.1.0.1ae238a aci_opflex_container: noiro/opflex:5.1.1.0.1ae238a aci_mcast_container: noiro/opflex:5.1.1.0.1ae238a aci_ovs_container: noiro/openvswitch:5.1.1.0.1ae238a aci_controller_container: noiro/aci-containers-controller:5.1.1.0.1ae238a aci_gbp_server_container: noiro/gbp-server:5.1.1.0.1ae238a aci_opflex_server_container: noiro/opflex-server:5.1.1.0.1ae238a ssh_key_path: /home/tux/.ssh/id_rsa ssh_cert_path: "" ssh_agent_auth: false authorization: mode: rbac options: {} ignore_docker_version: null kubernetes_version: "" private_registries: [] ingress: provider: "" options: {} node_selector: {} extra_args: {} dns_policy: "" extra_envs: [] extra_volumes: [] extra_volume_mounts: [] update_strategy: null http_port: 0 https_port: 0 network_mode: "" tolerations: [] default_backend: null cluster_name: "" cloud_provider: name: "" prefix_path: "" win_prefix_path: "" addon_job_timeout: 0 bastion_host: address: "" port: "" user: "" ssh_key: "" ssh_key_path: "" ssh_cert: "" ssh_cert_path: "" monitoring: provider: "" options: {} node_selector: {} update_strategy: null replicas: null tolerations: [] restore: restore: false snapshot_name: "" rotate_encryption_key: false dns: null~

You’re in #rke2 but appear to be talking about RKE1?