rancher-users #rke2

busy-crowd-80458

03/23/2023, 7:17 AM

Copy code

root@awx1:~# /var/lib/rancher/rke2/bin/kubectl get nodes
NAME   STATUS   ROLES                              AGE     VERSION
awx1   Ready    control-plane,etcd,master,worker   4h38m   v1.24.11+rke2r1
awx2   Ready    control-plane,etcd,master,worker   4h36m   v1.24.11+rke2r1
awx3   Ready    control-plane,etcd,master,worker   4h36m   v1.24.11+rke2r1

busy-crowd-80458

03/23/2023, 7:18 AM

but we notice that rke2-agent is not running, which we think is the worker role side of things

busy-crowd-80458

03/23/2023, 7:18 AM

is rke2-server alone sufficient, for nodes that are running multirole like this?

busy-crowd-80458

03/23/2023, 7:23 AM

how do I ensure a node truly is a worker?

busy-crowd-80458

03/23/2023, 7:28 AM

nvm

busy-crowd-80458

03/23/2023, 7:28 AM

Copy code

This output indicates that each replica is running on a different node, and thus the worker role is functioning on all three nodes.

Regarding the rke2-agent, it might not be running on all nodes because the RKE2 architecture combines the control plane and worker roles in a single binary (rke2-server). In an RKE2 cluster, you typically have one or more nodes running rke2-server and other nodes running rke2-agent. However, since your nodes are running both roles, the rke2-server binary takes care of both control plane and worker functionality, and there's no need for the rke2-agent to be running separately.

future-magician-11278

03/23/2023, 2:11 PM

Does azure dev ops repo connect through rke2? Sorry I’m still learning this process

stale-painting-80203

03/24/2023, 10:57 PM

Just tried to create a downstream RKE2 cluster and see the following pods go into a CrashLoop. The cluster shows as active in rancher, so whats the impact of the cluster functioning correctly and how do I fix the crash?

Copy code

kube-system           helm-install-rke2-ingress-nginx-tp5xd                   0/1     CrashLoopBackOff   45 (58s ago)    
kube-system           helm-install-rke2-metrics-server-rq8mc                  0/1     CrashLoopBackOff   45 (104s ago)

Both seem to have the same error:

Copy code

+ helm_v3 install --set-string global.clusterCIDR=10.42.0.0/16 --set-string global.clusterCIDRv4=10.42.0.0/16 --set-string global.clusterDNS=10.43.0.10 --set-string global.clusterDomain=cluster.local --set-string global.rke2DataDir=/var/lib/rancher/rke2 --set-string global.serviceCIDR=10.43.0.0/16 rke2-ingress-nginx /tmp/rke2-ingress-nginx.tgz
Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "<https://10.43.0.1:443/version>": dial tcp 10.43.0.1:443: i/o timeout
+ exit

+ helm_v3 install --set-string global.clusterCIDR=10.42.0.0/16 --set-string global.clusterCIDRv4=10.42.0.0/16 --set-string global.clusterDNS=10.43.0.10 --set-string global.clusterDomain=cluster.local --set-string global.rke2DataDir=/var/lib/rancher/rke2 --set-string global.serviceCIDR=10.43.0.0/16 rke2-metrics-server /tmp/rke2-metrics-server.tgz
Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "<https://10.43.0.1:443/version>": dial tcp 10.43.0.1:443: i/o timeout

hallowed-breakfast-56871

03/26/2023, 9:20 PM

Hey team - Is it possible to use 2 private registries within my registries.yaml? So far I can not make this work (both are to dockerhub)

refined-analyst-8898

03/27/2023, 12:33 PM

I've migrated a workload to RKE2, a distro that's still new to me. The application requires ingress SSL-passthrough (as opposed to termination), which is not enabled by default in RKE2's tailoring of the

ingress-nginx

chart. That's expected, but I'm hesitant to customize it because I don't have a complete mental map of the life cycle, especially upgrades. I've patched the daemonset to add the optional arg. It'd be helpful to hear from someone more familiar with RKE2 that this is entirely expected or a patently bad idea!

hundreds-evening-84071

03/27/2023, 2:09 PM

Hey folks, Question on cleaning up old unused images. Working on RKE2 v1.23.16+rke2r1 I came across this for k3s: https://github.com/k3s-io/k3s/issues/1900

sudo k3s crictl
images to see what images have been pulled locally

sudo k3s crictl rmi --prune
to delete any images no currently used by a running container

Is there something similar for RKE2?

hundreds-evening-84071

03/27/2023, 8:11 PM

Windows cluster question... I have RKE2 (v1.23.16+rke2r1) cluster with 2 nodes. • A single Linux node with Control Plane / Etcd / Worker • A single Windows Server 2019 Standard - Worker Question is about

rke2-coredns-rke2-coredns

There are 2 pods for rke2-coredns and first is running on Linux node second remains in pending state, Is this one of those things that will only run on Linux?

refined-analyst-8898

03/27/2023, 9:00 PM

I have an RKE2 cluster created in Rancher Manager 2.7.1. After the nodes were running, I adjusted the cluster config. This triggered a rolling replacement, which was cool automation to watch. When the node replacements finally finished, there were still numerous pods stuck in a transitional state, as viewed in Rancher Manager. I can see them with

kubectl

too. The cluster seems to be functioning despite all the ruckus in the pod list. Is it normal for you to clean up a bunch of stuck pods after a cluster configuration change?

worried-ram-81084

03/28/2023, 11:16 AM

I get this error:

Copy code

Mar 28 11:10:06 soss-m1 rke2[1006]: time="2023-03-28T11:10:06Z" level=info msg="Failed to test data store connection: this server is a not a member of the etcd cluster. Found [soss-m1-a184a80d=<https://172.16.1.11:2380> soss-m3-bd71347c=<https://172.16.1.13:2380>], expect: soss-m1-a184a80d=<https://10.1.1.11:2380>"

how do i change the expected IP address?

ancient-army-24563

03/29/2023, 6:58 AM

ancient-army-24563

03/29/2023, 6:58 AM

i want to edit the nginx configmap in RKE2 cluster deployed through rancher UI

ancient-army-24563

03/29/2023, 7:00 AM

like this mentioned here https://rke.docs.rancher.com/config-options/add-ons/ingress-controllers#configuring-network-options

ancient-army-24563

03/29/2023, 7:01 AM

does it need to be done from rancher UI or directly on the configmap manifest file

ancient-army-24563

03/29/2023, 7:05 AM

Copy code

ingress:
  provider: nginx
  options:
    map-hash-bucket-size: "128"
    ssl-protocols: SSLv2
  extra_args:
    enable-ssl-passthrough: ""

ancient-army-24563

03/29/2023, 10:18 AM

mutliple registries

ancient-army-24563

03/29/2023, 10:33 AM

how to add multiple endpoints in registries.yaml file where one endpoint need authentication , other one need to bypass authentication

handsome-tiger-45123

03/29/2023, 1:28 PM

Hello, I'd like to install rke2 using ansible. How can I create a token to be used by ansible playbooks? Does it have to follow some rules, or a simple string is enough?

little-doctor-70130

03/30/2023, 12:48 PM

Apologies for the total noob question but I am new to RKE2 and trying to figure out how to upgrade it properly. I inherited a v1.21.3 cluster which I've tried upgrading to v1.21.14 manually per the instructions here: https://docs.rke2.io/upgrade/manual_upgrade . Details were sparse so I stopped the rke2-server service on a master node and then installed v1.21.14 via the installation scripts. I then restarted the rke2-server service and the node appears functional, with working pods. However, a

kubectl get nodes

says it the master node is still v1.21.3. Checking

rke2 --version

says it is v1.21.14 - what am I missing?

colossal-television-75726

03/30/2023, 3:12 PM

Hi I have a problem with the Flannel VXLAN interface on RKE2 and I'm not quite sure where to put this: I can't restrict Flannel to listen on a specific address other than 0.0.0.0. I use Canal as CNI and configured it over a HelmChartConfiguration to use a specific interface. Although the interface itself is configured correctly, the service listens on 0.0.0.0:8472 (according to netstat -tulpn). Does anyone know how to fix this? Or do you know on which GitHub repository I should open an Issue? Thanks in advance!

steep-london-53093

03/30/2023, 3:54 PM

Hello, after installing new rke2 cluster I can get pod logs scheduled on any node of the cluster via any master node API. After restarting some master or worker nodes I try to get logs again and at this time via some master node API I can’t get pod logs from some nodes with error:

Copy code

<https://192.168.0.15:10250/containerLogs/kube-system/kube-proxy-k8s-master-03/kube-proxy>": proxy error from 127.0.0.1:9345 while dialing 192.168.0.15:10250, code 503: 503 Service Unavailable

Can you give me any hint to find the root cause of this problem? Thank you in advance!

polite-translator-35958

03/30/2023, 6:06 PM

I asked a question about rpm.rancher.io over in #general (https://rancher-users.slack.com/archives/C3ASABBD1/p1680194007334849) but figured I’d ask here as well. I want to instal rke2 via yum. If I follow docs.rke2.io/install/methods#enterprise-linux-8 I can install

yum install rke2-server

but I’d really like to add rpm.rancher.io to my company’s Artifactory server as a remote so that we can cache the rpms locally. I can’t seem to browse rpm.rancher.io trees and I can’t seem to configure an Artifactory Remote for it. Anyone ever done this? Anyone ever mirrored one of the rpm.rancher.io repos?

crooked-cat-21365

03/31/2023, 11:27 AM

What is the recommended setup to run my worker nodes with a second network bridge connected via a dedicated firewall to the internet? The "usual kubernetes traffic" should stay on the internal network all nodes are connected to. The control plane nodes are not supposed to be connected to the external network. I read through the Multus guides on github. If I got this correctly I have to create a NetworkAttachmentDefinition on each worker node with type "macvlan" and mode "bridge" and a set of undocumented (as it seems) parameters, install lets say another Ingress Controller and tell it to use the other network using an annotation. Is this correct?

hundreds-airport-66196

03/31/2023, 3:23 PM

Hi Rancher team, How do I rotate a certificate for v1.21.5+rke2r1? This one, "rke2 certificate rotate" is for v1.21.8+rke2r1 only.

magnificent-vr-88571

03/31/2023, 4:33 PM

hi all, I am trying to run debug containers https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/#ephemeral-container however it fails with following error

Copy code

Defaulting debug container name to debugger-bzglt.
error: ephemeral containers are disabled for this cluster (error from server: "the server could not find the requested resource").

Any documentation to enable it?

stale-painting-80203

03/31/2023, 6:59 PM

Anyone know how I can resolve a x509 error when pulling an image into a RKE2 cluster from a private container registry? I tried adding additionalCA to rancher as per https://ranchermanager.docs.rancher.com/v2.6/getting-started/installation-and-upgrade/installation-references/helm-chart-options#addition[…]sted-cas

Copy code

Warning  Failed     20m (x3 over 20m)  kubelet            Failed to pull image "harbor10165.senode.dev/sgs/webapp:2.0": rpc error: code = Unknown desc = failed to pull and unpack image "harbor10165.senode.dev/sgs/webapp:2.0": failed to resolve reference "harbor10165.senode.dev/sgs/webapp:2.0": failed to do request: Head "<https://harbor10165.senode.dev/v2/sgs/webapp/manifests/2.0>": x509: certificate signed by unknown authority