rancher-users #rke2

future-grass-75901

04/29/2022, 2:26 PM

Hello Fellow Rancher Users ... currently I am investigating EGress solutions and wanted to test Cilium ... as described on https://docs.cilium.io/en/latest/gettingstarted/k8s-install-helm/#install-cilium it should be possible via RKE and RKE2 (but apparently ist natively supported by RKE2 and recommended using the latter) ... but since RKE2 is viewed as an Technical Preview in Rancher 2.6.4 I wanted to use RKE since I have some expirience there ... but it showed some strange behaviour that the cilium health checks were not properly running and it was not very stabel with all that pod-starting-before-CNI-taint-stuff ... also first I setup a single node cluster just for testing which worked eventually after specing the VM to at least 4CPU/8GRam and restarting the pods and cilium daemonset ... anyhow after wanting to grow it to 3Nodes things got strange again so I wanted to try the native support from RKE2 ... first with a single node setup - But again the same issues coredns/cattle-cluster-agent pods crashlooping and cilium health check not properly working another round of rollout restart ds cilium && restarting unmanaged pods and some time did the trick ... for me this looked like the DNS pods started bevore the CNI therefore they had no network and the cattle-clsuter-agent could not resolve and connect to rancher ... today I updated the hypercube version from v1.21.10 > v1.21.12 and it seems to work ... but all this is quite strange and I would expect that Rancher would launch the nodes with this ominous taint cilium is recommending > https://docs.cilium.io/en/latest/gettingstarted/taints/ ... TLDR: is there some good tutorial on how to setup an k8s cluster via the rancher UI with the help of RKE2 :?

acceptable-gigabyte-29847

04/29/2022, 3:09 PM

Hello hello, what version of RKE2 should I use to not get any “-buildxxxxx” images? I’m working in an air gapped environment and I’m trying to figure out which images I need. However I see a lot of “-buildxxxx” images…

bland-jackal-22983

05/01/2022, 12:52 AM

another question, regarding kubelet extra blinds: previously in

rke

, we need to use

services_kubelet->extra-binds

to add a mapping in

cluster.yaml

like:

/opt/rke/var/lib/kubelet/plugins:/var/lib/kubelet/plugins

specifically i am trying to do this just wondering how can we add the extra binds if it's

rke2

, do i need to? i can't find corresponding server/agent config flag for this in the rke2 doc

mysterious-parrot-80714

05/01/2022, 6:16 PM

and pushing the default policies don't help 😕

future-truck-59205

05/02/2022, 8:19 AM

Hi everyone, I've been trying to configure Azure credentials in Rancher but every time I try it gives me "Authentication test failed, please check your credentials". Has anyone else also faced the same problem and how did you get rid of it? I've created the correct steps as given on docs ie: https://rancher.com/docs/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/azure/

acceptable-gigabyte-29847

05/02/2022, 10:17 AM

Any ideas how I can provide my own CA to the rancher-agent for communication to the rancher server? I can’t seem to find anything in the docs.

✅ 1

wonderful-helicopter-16401

05/03/2022, 4:23 AM

Hi, I am doing my first install of Rancher on RKE2 (2.6) and am having some issues with accessing

/var/lib/rancher/rke2/*server/manifests*

to configure nginx as a daemonset as outlined in these docs. It appears the permissions for the server
directory are not set properly on any of the nodes in this cluster. I'm seeing the steps were tested in 2.5.6. So, is this a bug or intended? Also, what permissions should I set on that directory to complete the final step in those docs?

numerous-country-20400

05/03/2022, 11:21 AM

How do i retrieve the kube-proxy network pool under rke2? Asking to resolve an issue with calico at https://github.com/projectcalico/calico/issues/5952#issuecomment-1115971887

faint-airport-83518

05/04/2022, 2:39 PM

is there any way to use the automated upgrade controller in an airgap?

magnificent-caravan-81252

05/04/2022, 3:47 PM

Hey friends! Can someone help clear up some confusion for me about the installation docs? As part of the process of installing a Rancher cluster using RKE2, it says that you need to setup “two” listeners on port 9345 and port 6443. However, the link shown for setting up the LB in a previous step doesn’t mention anything other than port 80 and 443. Am I to understand that I should add 9345 and 6443 as listeners onto the LB? I’m a little nervous about the security implications.

narrow-noon-75604

05/05/2022, 11:06 AM

Hi, I am trying to add the agents (worker) nodes to the rke2 server, but the "rke2-agent" service is stuck in "activating" state with an error msg "failed to get CA certs: ". I am unable to find any recommendations for the issue. So please help me in fixing this issue.

creamy-scooter-43304

05/05/2022, 1:37 PM

Could You please share with some example for rke2 config file ?

acceptable-gigabyte-29847

05/05/2022, 3:21 PM

Hello, is it possible to deploy a Rancher Server on top of a cluster (RKE2) deployed by another Rancher Server?

curved-caravan-26314

05/07/2022, 10:19 PM

Hello all, is it difficult to turn a rke2 from single server to HA? I can set 3 replicas on the first server and then add the other to control planes later on?

ripe-queen-73614

05/08/2022, 9:23 AM

Please can someone help me? Thank you

curved-caravan-26314

05/08/2022, 6:04 PM

Does anyone know if rke2 would be able to leverage an external etcd mysql database like K3S?

hallowed-breakfast-56871

05/08/2022, 11:12 PM

Hey folks - Anyone got filebeat working with collecting pod logs with RKE2? I see logs are placed / setup differently on RKE2 so this means filebeat gets confused with this logs belongs to which pod. https://www.elastic.co/guide/en/beats/filebeat/current/running-on-kubernetes.html

curved-caravan-26314

05/08/2022, 11:25 PM

Im on a new stable install of rancher and I am getting this error and i have no backup to get a new secret. what do i do? Ive been through the troubleshooting page and everything else checks out.

Copy code

Events:
  Type     Reason       Age                From               Message
  ----     ------       ----               ----               -------
  Normal   Scheduled    45s                default-scheduler  Successfully assigned cattle-system/rancher-webhook-6958cfcddf-jzqd7 to k42
  Warning  FailedMount  45s (x2 over 45s)  kubelet            MountVolume.SetUp failed for volume "tls" : secret "rancher-webhook-tls" not found
  Normal   Pulled       41s                kubelet            Container image "rancher/rancher-webhook:v0.2.5" already present on machine
  Normal   Created      40s                kubelet            Created container rancher-webhook
  Normal   Started      39s                kubelet            Started container rancher-webhook

sparse-continent-53433

05/09/2022, 2:00 PM

Are there any directions for fixing a host that has expired certificates? I'm receiving this error on a few of my nodes:

Copy code

level=error msg="CA cert validation failed: Get \"<https://127.0.0.1:6444/cacerts>\": x509: certificate has expired or is not yet valid

hundreds-airport-66196

05/09/2022, 9:03 PM

Our rke2 is up and running in prod. Now, it has having conflict with network subnet 10.42.0.0/16 as this subnet is also use by the cluster load. Is there a documentation on how to change the cidr settings? thanks

busy-orange-63415

05/09/2022, 11:53 PM

Hi, can someone help me? https://rancher-users.slack.com/archives/C3ASABBD1/p1652140228814169

silly-jordan-81965

05/10/2022, 11:31 AM

What do i need to change to get rke2 to give me logs in json format instead of klog?

faint-ram-5049

05/11/2022, 7:05 PM

Does anyone know if there is documentation showing the maximum number of worker nodes RKE2 can maintain?

victorious-analyst-3332

05/12/2022, 12:59 AM

Would anyone be able to tell me if it is possible to install arbitrary CRDs as a part of the RKE2 install process? We’re currently installing calico via Helm chart to stay more recent on RKE installs, but it appears that we can do most of that via RKE2 Helm Integration. In our case, we need to apply the calico CRDs and create a number of custom resources for things like BGP peering and felixConfig before the operator deploy (options not present in the operator), so I was hoping there might be a method for also applying the custom config. I did see the

/var/lib/rancher/rke2/server/manifests

approach, but don’t fully understand the ramifications of the

AddOn custom resources

detail mentioned there. Thanks a lot for any help you can give.

silly-jordan-81965

05/13/2022, 7:45 AM

Im deploying a rancher2_cluster_v2 cluster and want to add cloud provider config. But receives the following error when doing that: on main.tf line 109, in resource “rancher2_cluster_v2” “k8s-sandbox-sc-mr”: │ 109: cloud_provider { │ │ Blocks of type “cloud_provider” are not expected here. My terraform code looks like this:

Copy code

rke_config {
    cloud_provider {
      name = "openstack"
      openstack_cloud_provider {
        global {
          auth_url = var.openstack_auth_url
          username = var.openstack_username
          password = var.openstack_password
          tenant_id = var.openstack_tenant_id
        }
      }
    }
    machine_global_config = <<EOF

Anyone that can point me in the right direction?

rapid-helmet-86074

05/13/2022, 1:32 PM

I noticed today that Rancher 2.6.5 was released (https://forums.rancher.com/t/rancher-release-v2-6-5/37891/3) and in it RKE2 deployment goes from tech preview to general availability, but that's only with Kubernetes v1.22+. Can any Rancher folks let me know if there're plans to extend that to earlier Kubernetes versions? We've got a reason that we were hoping to use v1.20 at the moment, so I'm wondering if my best answer is wait or push a completely separate team to upgrading Kubernetes versions.

refined-magician-25478

05/13/2022, 3:48 PM

Does anyone know if you can use the Ranchers helm crd with a local helm chart, not in a chart repository?

hundreds-airport-66196

05/13/2022, 10:08 PM

How to upgrade rke2 on an airgap installation?

broad-farmer-70498

05/16/2022, 8:07 PM

anyone around been able to get host-process (windows) containers working with rke2? I'm not having much luck..

bland-jackal-22983

05/17/2022, 8:55 AM

hi, i couldn't figure out why my

rke2-server

installation would fail when i set the

node-label

values? in the

/etc/rancher/rke2/config.yaml

i have:

Copy code

node-label:
- <http://node-role.kubernetes.io/master|node-role.kubernetes.io/master>
- <http://openebs.io/engine=mayastor|openebs.io/engine=mayastor>

now if i install rke2 with everything the same as before but with this new

node-label

, i would have this error:

Copy code

May 17 08:48:26 698F191 rke2[121465]: time="2022-05-17T08:48:26Z" level=error msg="Kubelet exited: exit status 1"
May 17 08:48:28 698F191 rke2[121465]: {"level":"warn","ts":"2022-05-17T08:48:28.923Z","logger":"etcd-client","caller":"v3@v3.5.3-k3s1/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"<etcd-endpoints://0xc000955340/127.0.0.1:2379>","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""}
May 17 08:48:28 698F191 rke2[121465]: time="2022-05-17T08:48:28Z" level=info msg="Failed to test data store connection: context deadline exceeded"
May 17 08:48:30 698F191 rke2[121465]: {"level":"warn","ts":"2022-05-17T08:48:30.005Z","logger":"etcd-client","caller":"v3@v3.5.3-k3s1/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"<etcd-endpoints://0xc000955340/127.0.0.1:2379>","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""}
May 17 08:48:30 698F191 rke2[121465]: time="2022-05-17T08:48:30Z" level=error msg="Failed to check local etcd status for learner management: context deadline exceeded"
May 17 08:48:30 698F191 rke2[121465]: time="2022-05-17T08:48:30Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:9345/v1-rke2/readyz>: 500 Internal Server Error"

if i don't have this

node-label

section, the installation goes through. os:

ubuntu 20..04

rke2 version:

v1.22.9