Can I run rke up --dind Then deploy my application / k8s resources Then save that docker image as one big image and then deploy it into another machine , but this time using docker run ... ? The expected result is a running cluster and my deployed application inside the docker image.

I'm needing to pass a flag to cri-dockerd in rke, and i'm not seeing how this is possible. (https://github.com/Mirantis/cri-dockerd/pull/100) the only option I see in rke is to turn it on.. does anyone know if it's possible to configure cri-dockerd before, or after the fact?

Hello, we have connected Rancher 2.6.9 with the Harvester cluster v1.1.0. While trying to deploy RKE1 cluster it's waiting with the message "Waiting for ssh connection..." and after a while, this node is replaced with another and the cycle repeats. The issue looks similar to: https://github.com/harvester/harvester/issues/1519. Is there a fix/workaround?

Hi, I ran into an issue where creating a new cluster was failing with the following error:

2022/11/17 22:09:38 [ERROR] cluster [c-jpmvb] provisioning: [[network] Host [10.10.50.53] is not able to connect to the following ports: [10.10.50.44:2379]. Please check network policies and firewall rules]

I previously created a cluster a few months ago with the same node template and RKE template and did not run into the issue. The main difference is that when https://releases.rancher.com/install-docker/20.10.sh ran months ago in the previous cluster it installed Docker engine version

20.10.12

whereas it installed version

20.10.17

in the new cluster (assuming the script has been updated since). I was able to workaround the issue by adding

iptables=false

to the Docker engine options in my node template but based on the Docker documentation, it sounds like disabling that is not recommended. Anyone know how I can get it working without disabling

iptables

?

I am trying to create and rke cluster inside a dind docker image , I have reached a point where all cluster Components are created , but the network plugin fails And when i look at the network plugin pod logs , i see a time out when it tries to reach the node ip Any suggestions , is it even possible to create a full working Rke cluster inside a dind docker image

has anyone used rke on Ubuntu 22.04 and got k8s working correctly? I cannot access any services--even NodePort or LoadBalancer--from a host (such as my laptop) outside of the cluster. I don't think it's an rke thing at all, just curious.

Is there any way to change kubelet configuration, on fly, on only one node? I see there were two Issues reported (feature request) on this subject but both were closed without solution. Wonder if anyone find the way?

Not sure if this is the proper place to ask this question. I'm seeing a problem with two cattle-node-agents connecting/running. It appears to only be two. I'm also seeing an issue with two downstream clusters where cluster agent is not connected. This is my scenario and how I sort of broke it... • dev cluster was fine and working, we setup the new rancher control plane with HA, and imported the dev cluster from an older control plane. We never got around to importing the production cluster to this new control plane and put this on pause. • I went ahead sometime later to try and import / register the prod cluster with this new control plane but in doing so it moved all of my dev namespaces and resources under the production cluster name, not what I was expecting. This part was my fault because I think I still had the dev config file set for my kubeconfig ◦ I attempted to re-trace my steps backwards and re-register the dev cluster and in doing so I managed to disconnect the cluster agent... I think this is what had happened. This is where things got worse. I set my kubeconfig file back to the production config file attempted to re-register the production downstream cluster to see the dev resources and caused the production cluster agent to disconnect. Wondering if anyone has encountered this issue before and steps to rectify. At this point in time I can only talk to my production namespaces. It appears the production cattle-node-agents are trying to resolve the old control plan address. I tried manually editing the server url but just get errors when I try to apply the new yaml. Both cattle-node-agents are pointing to the old control plane url for the server.

👋 I think this is the right place to ask about this, but let me know if it is not, basically I have seen a bunch of issues like these: 29449, 38359, 19916, 19597, and a few more similar with the error:

cannot proceed with upgrade of controlplane since 1 host(s) cannot be reached prior to upgrade

The cluster was provisioned with Terraform and RKE, the problem seems to have been triggered by a downscale at the same time that the cluster was about or getting upgraded, I have tried clearing the node from the CRDs, update the state manually, recreate a node, but nothing worked after that, nodes can’t register, and the upgrade is also stuck, any ideas to what can I do to keep debugging this? is there a way to deregister and register the cluster again in rancher to see if that helps?

Hi All, I'm a pretty new Rancher user and I have a question about an on-prem cluster I have. My cluster was deployed with Terraform and is managed by rancher 2.5. Some manual operations were done along the way and now the RKE config of my cluster is different from its real state. In other words rancherKubernetesEngineConfig != appliedSpec I have 3 nodes that are part of the cluster but are not in the rancherKubernetesEngineConfig. I'd like to go back to managing my cluster with terraform but for that I need to "re-sync" rancherKubernetesEngineConfig and appliedSpec. Is there a way to do that?

trying to run dual-stack rke2 with ipv6 support

cluster-cidr: 10.42.0.0/16,2001:cafe:42:0::/56
service-cidr: 10.43.0.0/16,2001:cafe:42:1::/112

any suggestions?

E1214 11:56:17.881602       1 main.go:330] Error registering network: failed to configure interface flannel-v6.1: failed to ensure v6 address of interface flannel-v6.1: failed to add v6 IP address 2001:cafe:42::/56 to flannel-v6.1: permission denied

I am trying to run the skaffold quick start example from https://skaffold.dev/docs/quickstart/ on an on-prem RKE1 Rancher cluster. When running `skaffold dev`an error occurs. Deployed is RK1, Rancher version 2.5.11 and Kubernetes version 1.20.15. The error is in the attached skaffold-error.txt There is an on-prem Nexus Container Registry in place. skaffold pushes the image successfully, but the image cannot be pulled. Probably somewhere in the rancher the registry URL needs to be declared similary to the registries.yaml in k3s/k3d. Where should that be configured?

I get this error 1 in 3 times when trying any kubectl command, using a KubeConfig from rancher ui for a "downstream" rke cluster:

Error from server (InternalError): an error on the server ("unable to create impersonator account: error setting up impersonation for user user-6j58n: failed to get secret for service account: cattle-impersonation-system/cattle-impersonation-user-6j58n, error: timed out waiting for the condition") has prevented the request from succeeding (get nodes)

There's 3 nodes with "all roles", plus 1 worker. "current-context" in KubeConfig points to fqdn for rancher mgmt cluster. If I change this to point to one of the rke cluster nodes, it always works, for all 3 nodes. In "cattle-impersonation-system" ns on rke cluster exists secret "cattle-impersonation-user-6j58n-token", but not without "-token". Help? 🙂

Hello, I’m using rancher v2.7.0 and I’ve created a cluster with rke. During the process, I’ve configured 3 node pools but after the creation of the machines, the hostname of all machines are changed to respect

<the name of the node pool>-<machine number>

. However for the cloud provider, we must not to override the hostname, is there a way to disable the hostname override when using node pools ? I know

hostname_override

in the yaml, but I don’t see how to use it with node pools. Thank you 🙂

Is there any recommendation about how to Handle admission deadlock on RKE when using the Rancher provided OPA? I was not able to find anything on Rancher documentation.

Hi all, I'm in some really serious trouble here and I hope someone can help; I just upgraded from rke v1.3.2 to rke v1.3.17 by updating the binary and doing an

rke up

. All nodes upgraded fine, but now the etcd containers are continuously panicking on all nodes;

{"level":"panic","ts":"2023-01-04T11:13:47.673Z","caller":"rafthttp/transport.go:346","msg":"unexpected removal of unknown remote peer","remote-peer-id":"d88f54ed22afab7e","stacktrace":"<http://go.etcd.io/etcd/server/v3/etcdserver/api/rafthttp.(*Transport).removePeer\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/api/rafthttp/transport.go:346\ngo.etcd.io/etcd/server/v3/etcdserver/api/rafthttp.(*Transport).RemovePeer\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/api/rafthttp/transport.go:329\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyConfChange\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:2301\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).apply\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:2133\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyEntries\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:1357\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyAll\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:1179\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).run.func8\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:1111\ngo.etcd.io/etcd/pkg/v3/schedule.(*fifo).run\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/pkg/schedule/schedule.go:157|go.etcd.io/etcd/server/v3/etcdserver/api/rafthttp.(*Transport).removePeer\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/api/rafthttp/transport.go:346\ngo.etcd.io/etcd/server/v3/etcdserver/api/rafthttp.(*Transport).RemovePeer\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/api/rafthttp/transport.go:329\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyConfChange\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:2301\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).apply\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:2133\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyEntries\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:1357\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyAll\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:1179\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).run.func8\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdserver/server.go:1111\ngo.etcd.io/etcd/pkg/v3/schedule.(*fifo).run\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/pkg/schedule/schedule.go:157>"}
panic: unexpected removal of unknown remote peer

This is happening on all 3 etcd nodes. I found related information on the etcd github, but there everyone talks about removing nodes and adding specific startup options. Since this is running in docker and I have no idea where the config is, I am unclear on how to proceed here. If someone could please help me out here; we now have a field cluster that is unreachable and I have no idea what to do...

How do I allow for a company internal container registry? This error occurs when deploying: Pod web-86b444ddc6-cm4vq Failed to pull image "nexus.p2.mycompany.net:8023/mycompany/skaffold-buildpacks-node:v2.0.2-76-g420b7901a-dirty@sha256:5efb8734b9e1dabe00e6bd3541ec7b74a41f89472a365396be2e62c96a6569fb": rpc error: code = Unknown desc = Error response from daemon: Get https://nexus.p2.mycompany.net:8023/v2/: http: server gave HTTP response to HTTPS client

Hello all, I'm looking to install rancher for desktop into a vps and I wanted to know if some of you have already did it and have some lessons learns and best practice on how to make it, error to avoid, minimal specification for CPU, Ram, hard disk etc... what kind of storage do you used for volumes etc... thanks

Is there a difference between rke1 and rke2 from a paid support prespective. does rancher treat both product lines the same? Will they provide long term support for rke1 ?

Hi guys, I am using RKE 1.3.11 and I need to move my cluster nodes interfaces IPs from 172.16.30.x/24 to 172.16.40.x/24. They are VMs so I could change easily their IPs but I guess I can not just happily change the IP's in the cluster.yaml file and expect it works next time I run 'rke up'. The first thing I think will fail is the cluster conectivity because the IP in the api certificate will not match, but probably there will be way more things to take in care. Can anyone throw some light on how to change the IPs of the cluster nodes? Any help or clus is welcome. Thank you in advance

Hello, I got some issue when trying to spin up a cluster with 2 VM VM1 => control plane etc worker node VM2 => worker node The VM2 cannot be found FATA[0370] [ "node01.example.com" not found] Log in kubelet I0112 094040.188538 3988 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: Get "https://127.0.0.1:6443/apis/storage.k8s.io/v1/csinodes/node01.example.com": dial tcp 127.0.0.16443 connect: connection refused E0112 094040.243255 3988 kubelet.go:2424] "Error getting node" err="node \"node01.example.com\" not found" What I don't understand is the call to localhost 6443 in the VM2 kubelet Should it not be the address of a control plane node (here VM1) ?

Ok got the solution, its been 1 year RKE is not working with arm64 (nginx-proxy container)

How much it is the rke paid support for a freelancer? I have tried to contact suse but their partners are taking forever to contact me back. maybe I am not their sales target.

Hello, We recently upgrade one node in cluster from debian 10 to debian 11 and any pod in this node dont have network connection right now. K8s version 1.21.1 Any ideas?

is there a way to recover from accidentally deploying a different network plugin and then switching back?

ist there a solution to provision nodes in a private network,

rke

has the bastion option, but it doesn't seem to work from Rancher with RKE Provisioning.

Hi, can anyone tell me, how I can update rke-tools from v0.187 to v0.188 in a rancher-setup created via docker-compose?

@victorious-analyst-3332 has left the channel

[Repost from K3s]

Do you have ay pointers on how to proceed with debugging ? A few days ago we upgraded to K8S 1.24 and also added a few nodes. Everything was fine until a physical reboot. For the control plane we have 3 nodes with controlplane and etcd roles. Two of them are operating without errors and one of them with exactly the same error as the workers