I am getting following in

rancher-vsphere-cpi-cloud-controller-manager

RKE2 v1.25 and also in v1.26

E0713 17:24:46.617584       1 node_controller.go:229] error syncing 'node1': failed to get provider ID for node node1 at cloudprovider: failed to get instance ID from cloud provider: VM Guest hostname is empty, requeuing

Is this a permissions issue in vsphere?

Hi, I just get managed to deploy a vSphere Cluster with VM-Templates, cloud-init, vApp-Options and so on with the standard cluster->create->vSphere - wizard. Becaus there are many parameters to enter, I like to create a cluster-template. For RKE2 there is the new helm-concept: https://github.com/rancher/cluster-template-examples/tree/main. Do I have to register this templates under Repositories for the local-cluster? This seams to work with the example github-repo. Or can I deploy a template direct with helm from a local helm-chart? I did not get this to work. I like to avoid the deploiment with a repo. Rancher: 2.7 on a k3s cluster.

Hi All, I am having a bit of an issue with getting something working, I've spun up an Ubuntu VM and installed rancher with docker (using the quick start command).

I can then go to the cluster management page and 'create' a cluster on vsphere, I put in the right creds, then in the options I choose RKE2 as the provider, mostly leave the options unchanged apart from selecting a network and a VM template to use (Ubuntu, spins up with dynamic IP)

a VM spins up in vpshere named after the node pool name I entered, but it gets stuck on 'configuring bootstrap nodes <stuff> waiting for agent to check in and apply initial plan

Have tried opening ports with UFW (a few rancher ports I read about, plus 443, on both the vm hosting the dockerised rancher and the node pool it spins up, no luck)

The thought occurs it could be some host.docker.internal stuff maybe? (pls feel free to recommend troubleshooting steps if you agree?) or that the node it spins up just isn't doing anything rancher-y at all (for example, I cant see any containers on that machine with dockerps, and don't see any mention of rancher or RKE in the syslog/auth log etc)

hi all, i am going through the process of upgrading my clusters from 1.24 to 1.25.11

hi all, i am going through the process of upgrading my clusters from 1.24 to 1.25.11 as i upgraded from Rancher 2.73 -2.75. Its a 3 node etc plane , and 2 worker nodes. The cluster are managed using the Vmware cluster management/provider. It went through the process of upgrading them to 1.25.11 through the UI and it has gonee through the steps of updating them labelling them with the taints ode.kubernetes.io/network-unavailable=:NoSchedule and the plan.upgrade.cattle.io/system-agent-upgrader=7385bc6684d78a6b8f192e81ba787f77cc494f9d44f90c44c4e4aca0 . However what I am finding is that there are status in the pools to say that it is waiting for probes: Calico, waiting for probes kube-apiserver on two different nodes and one node node is in an unknown state, I have checked the nodes and they have upgraded to 1.25.11. but the status on them has not changed. Would be grateful for any assistance here.

Does anyone have a working example of using a Cluster Template with vSphere to provision machines with vappProperties specified in

values.yaml

? No matter what format I’ve tried, I get a Helm error when trying to deploy the chart/template:

2023-07-21T14:47:52.234573800-04:00 Error: INSTALLATION FAILED: YAML parse error on cluster-template/templates/nodeconfig-vsphere.yaml: error converting YAML to JSON: yaml: line 45: did not find expected ',' or ']'

Without specifying vappProperty, everything else works properly – the VM is provisioned from the right template, in the right cluster, using the right storage pool, etc. and placed in the right folder. However, since I’m attempting to parse network configuration from a vSphere Network Protocol Profile provided via

vappTransport: "com.vmware.guestInfo"

, the machine never progresses. (A similar configuration using an RKE1 node template works fine.)

Guys i wanted to have the vsphere out-of-tree storage approach by installing CSI and CPI , I am trying as per the documentation, to choose the cloud provider as vsphere in the cluster section . I am provisionig a 3 master node and 3 worker node cluster . But when the provisionig the starts the node gets stuck at the "waiting for agent" step. I even tried doing ssh into the initial master on which it shows this waiting message steps used to reproduce the problem - 1. journalctl -u rancher-system-agent.service -f 2. systemctl status rke2-server what i have found it this

```Main PID: 1772 (rke2)

Tasks: 139

Memory: 1.8G

CPU: 1min 24.636s

CGroup: /system.slice/rke2-server.service

├─1772 "/usr/local/bin/rke2 server"

├─1797 containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/rke2/agent/containerd

├─1811 kubelet --volume-plugin-dir=/var/lib/kubelet/volumeplugins --file-check-frequency=5s --sync-frequency=30s --cloud-provider=external --cloud-config= --address=0.0.0.0 --anonymous-auth=false --authent>

├─1898 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id b92974ab30b1148fab78c2624c31eb44b50ab1d5b926c8cd844647349c135107 -address /run/k3s/containerd>

├─1984 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id 814f7b7f1f860789efd453ac8bfa793b48b1a561f5704d44c9092a66fb999e62 -address /run/k3s/containerd>

├─2078 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id 8314cd7f26eb53a3e865d92b1c0c66d79c5391de4556261420c3fd76bcc02207 -address /run/k3s/containerd>

├─2079 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id 5caf19098e89f5466118df43fe5a5d90181f3c1ca641a1217fa897a9442f4528 -address /run/k3s/containerd>

├─2236 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id 3667f8df7e783aa37fc5ff06356e21912b7555b44c257b221acd4aa719410c1b -address /run/k3s/containerd>

├─3314 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id c70ca041b43f8f8fdecae901beb37d40c5370a229888cb9dbe0ed06a3dbc94fb -address /run/k3s/containerd>

├─3779 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id 3ebc43b47311aa483cbe3857f48633306292e36d2901d9a92e7c36b9cff04ba6 -address /run/k3s/containerd>

└─4001 /var/lib/rancher/rke2/data/v1.26.6-rke2r1-8e8a9f4b16b3/bin/containerd-shim-runc-v2 -namespace k8s.io -id 0cfc1fd44cd126b40f9508f9c8e790f2620323a47053f1ed77396026d94af02b -address /run/k3s/containerd>

Jul 27 115816 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T115816Z" level=info msg="Active TLS secret kube-system/rke2-serving (ver=380) (count 10): map[listener.cattle.io/cn-10.10.8.60:10.10.8.60 li>

Jul 27 115816 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T115816Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --c>```

Jul 27 11:58:16 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:16Z" level=error msg="error syncing 'kube-system/rke2-calico': handler helm-controller-chart-registration: <http://helmcharts.helm.cattle.io|helmcharts.helm.cattle.io> \"r>

Jul 27 11:58:18 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:18Z" level=error msg="error syncing 'kube-system/rke2-coredns': handler helm-controller-chart-registration: <http://helmcharts.helm.cattle.io|helmcharts.helm.cattle.io> \">

Jul 27 11:58:19 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:19Z" level=error msg="error syncing 'kube-system/rke2-ingress-nginx': handler helm-controller-chart-registration: helmcharts.helm.cattle>

Jul 27 11:58:20 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:20Z" level=error msg="error syncing 'kube-system/rke2-metrics-server': handler helm-controller-chart-registration: helmcharts.helm.cattl>

Jul 27 11:58:21 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:21Z" level=error msg="error syncing 'kube-system/rke2-snapshot-controller-crd': handler helm-controller-chart-registration: helmcharts.h>

Jul 27 11:58:21 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:21Z" level=error msg="error syncing 'kube-system/rke2-snapshot-controller': handler helm-controller-chart-registration: helmcharts.helm.>

Jul 27 11:58:21 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T11:58:21Z" level=error msg="error syncing 'kube-system/rke2-snapshot-validation-webhook': handler helm-controller-chart-registration: helmchar>

```Jul 27 115859 vspherec1-pool1-dcfe19da-dtrkj rke2[1772]: time="2023-07-27T115859Z" level=info msg="Tunnel authorizer set Kubelet Port 10250"

lines 1-31/31 (END)```

the highlighted lines it says that these are not found requeing. How should i proceed with this and get the cpi and csi working .

When spinning up a nodepool/cluster in vsphere using the in-rancher provisioner, the nodes are getting stuck at waiting to check in and apply initial plan - What are some troubleshooting steps I could do to get some logs from the nodepool machines? RKE2 on Ubuntu 2204.

Running Rancher on-premise on VMware. One issue I am seeing from an operations stand point is determining actual resource utilization. Our environment has (3) Rancher nodes, (3) Controllers, and (3) Workers all which we have running 6cpu and 12GB of RAM. When looking at vROPs it shows I can reduce CPU by 2 and all the nodes should be fine running on 4cpu. We have memory to spare so could increase which we are looking at doing so the nodes will have 20-24GB of RAM. Then you look at Rancher interface and you see the reservation for the Cores are 62% but Used is around 6%. I have tried using vROPs recommendation and reducing CPU to 4 but then our Platform Team is unable to deploy all the other various tools like Longhorn, Sysdig, ArgoCD, Harbor, etc, etc. Is there some means to help bridge the gap with what vROPs shows as being used and what Rancher is showing as used to help understand why vROPs says we can reduce CPU and Rancher says we are good or need more cores? Our Platform Team has used Goldilock to help ensure everything is right-sized but trying to be as efficient as we can with our hardware in running K8s on-premise. Any advice or recommendations are greatly appreciated. Anyone else have a good balance running on-premise K8s with Rancher and VMware?

Hi all. Does rancher 2.69 rke2 support vsphere 8?

Hi.. we have setup rke2 with vsphere cloud provider.. the rancher-vsphere-cpi-cloud-controller-manager pods came up without any issue.. but the vsphere-csi-controller pods are in crashloop error with the error --> {"level":"info","time":"2023-08-01T144951.230009171Z","caller":"vanilla/controller.go:1805","msg":"CreateVolume: called with args {Name:pvc-0dafdc33-2b02-44b4-ba0e-820d7e8f5b43 CapacityRangerequired bytes8589934592 VolumeCapabilities[mount<fs_type:\"ext4\" > access_mode&lt;modeMULTI_NODE_MULTI_WRITER > ] Parameters:map[] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}","TraceId":"62157d51-da75-4a54-9b31-6a102a392bc9"} panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x1b22a25] goroutine 249 [running]: sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/vanilla.(*controller).createFileVolume(0xc00007c370, {0x26af658, 0xc0000dbe90}, 0xc000446ee0) /build/pkg/csi/service/vanilla/controller.go:1736 +0xd05

Hi All, I am facing this error while trying to add new nodes to my Vsphere Rancher Cluster. Please advise what could be done to fix this.

Rkecontrolplane was already initialized but no etcd machines exist that have plans, indicating the etcd plane has been entirely replaced. Restoration from etcd snapshot is required

Hi all, I am trying to deploy an RKE2 cluster on vSphere via the Rancher UI. I use the below details. • Rancher Helm installation 2.7.5-stable • vSphere Client version 7.0.3.00700 As a basic step, I tried to deploy a basic RKE2 cluster with 1 node using the cloud-provider vSphere, Canal as my CNI and version 1.26.6+rke2r1 As for the VM template I used a basic 22.04 Ubuntu image and I installed all the needed dependencies based on the following guide: https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/vsphere/create-a-vm-template It's worth mentioning that ufw was disabled for testing purposes. Whatever I tried, the cluster is not getting created by instead is stuck in "waiting for agent to check in and apply initial plan" state. The logs from the Rancher pod can be found below.

2023/08/03 06:50:39 [ERROR] [rkecluster] fleet-default/rke2-canal: error getting CAPI cluster no matching controller owner ref
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler rke-cluster: no matching controller owner ref, requeuing
2023/08/03 06:50:39 [ERROR] [rkecluster] fleet-default/rke2-canal: error getting CAPI cluster no matching controller owner ref
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler rke-cluster: no matching controller owner ref, requeuing
2023/08/03 06:50:39 [ERROR] [planner] rkecluster fleet-default/rke2-canal: error encountered during plan processing was no matching controller owner ref
2023/08/03 06:50:39 [ERROR] [rkecluster] fleet-default/rke2-canal: error getting CAPI cluster no matching controller owner ref
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler rke-cluster: no matching controller owner ref, requeuing
2023/08/03 06:50:39 [ERROR] [rkecluster] fleet-default/rke2-canal: error getting CAPI cluster no matching controller owner ref
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler rke-cluster: no matching controller owner ref, requeuing
2023/08/03 06:50:39 [ERROR] [rkecluster] fleet-default/rke2-canal: error getting CAPI cluster no matching controller owner ref
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler rke-cluster: no matching controller owner ref, requeuing
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler planner: no matching controller owner ref, requeuing
2023/08/03 06:50:39 [ERROR] [planner] rkecluster fleet-default/rke2-canal: error encountered during plan processing was no matching controller owner ref
2023/08/03 06:50:39 [ERROR] [rkecluster] fleet-default/rke2-canal: error getting CAPI cluster no matching controller owner ref
2023/08/03 06:50:39 [ERROR] error syncing 'fleet-default/rke2-canal': handler rke-cluster: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [planner] rkecluster fleet-default/rke2-canal: error encountered during plan processing was no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal': handler planner: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [planner] rkecluster fleet-default/rke2-canal: error encountered during plan processing was no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal': handler planner: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [planner] rkecluster fleet-default/rke2-canal: error encountered during plan processing was no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal': handler planner: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for infrastructure ready
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for infrastructure ready
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for infrastructure ready
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for infrastructure ready
2023/08/03 06:50:40 [ERROR] [CAPI] Reconciler error: Operation cannot be fulfilled on <http://machinedeployments.cluster.x-k8s.io|machinedeployments.cluster.x-k8s.io> "rke2-canal-pool1": the object has been modified; please apply your changes to the latest version and try again
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for infrastructure ready
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for at least one control plane, etcd, and worker node to be registered
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for at least one control plane, etcd, and worker node to be registered
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for at least one control plane, etcd, and worker node to be registered
2023/08/03 06:50:40 [ERROR] [rkebootstrap] fleet-default/rke2-canal-bootstrap-template-jw4bv: error getting machine by owner reference no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-bootstrap-template-jw4bv': handler rke-bootstrap: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [rkebootstrap] fleet-default/rke2-canal-bootstrap-template-jw4bv: error getting machine by owner reference no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-bootstrap-template-jw4bv': handler rke-bootstrap: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: error getting machine by owner reference: no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt': handler machine-provision: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: error getting machine by owner reference: no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt': handler machine-provision: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [rkebootstrap] fleet-default/rke2-canal-bootstrap-template-jw4bv: error getting machine by owner reference no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-bootstrap-template-jw4bv': handler rke-bootstrap: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: error getting machine by owner reference: no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt': handler machine-provision: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [rkebootstrap] fleet-default/rke2-canal-bootstrap-template-jw4bv: error getting machine by owner reference no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-bootstrap-template-jw4bv': handler rke-bootstrap: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: error getting machine by owner reference: no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt': handler machine-provision: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for at least one control plane, etcd, and worker node to be registered
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for at least one control plane, etcd, and worker node to be registered
2023/08/03 06:50:40 [ERROR] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: error getting machine by owner reference: no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt': handler machine-provision: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [rkebootstrap] fleet-default/rke2-canal-bootstrap-template-jw4bv: error getting machine by owner reference no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-bootstrap-template-jw4bv': handler rke-bootstrap: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [ERROR] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: error getting machine by owner reference: no matching controller owner ref
2023/08/03 06:50:40 [ERROR] error syncing 'fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt': handler machine-provision: no matching controller owner ref, requeuing
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] EnsureSecretForServiceAccount: waiting for secret [rke2-canal-bootstrap-template-jw4bv-machine-bootstrapz5jv] to be populated with token
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:40 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:41 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:41 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:41 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:41 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:49 [INFO] [planner] rkecluster fleet-default/rke2-canal: waiting: waiting for viable init node
2023/08/03 06:50:50 [INFO] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: reconciling machine job
2023/08/03 06:50:50 [INFO] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: reconciling machine job
2023/08/03 06:50:50 [INFO] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: reconciling machine job
2023/08/03 06:50:50 [INFO] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: reconciling machine job
2023/08/03 06:50:50 [INFO] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: reconciling machine job
2023/08/03 06:50:50 [INFO] [machineprovision] fleet-default/rke2-canal-pool1-bb5a7e33-qb9zt: reconciling machine job

Any ideas or pointers on how to proceed? I experience the same output via a Terraform plan and via the use of the latest rancher2 Terraform provider.

I'm stuck with "Configuring bootstrap node(s) argh-pool1-7cbd9f6556-gfnqh: waiting for cluster agent to connect" and not sure where to proceed. If anyone could point me in the right direction, it would be appreciated.

Hi All, What would be the best way to apply load based autoscaling on Rancher Cluster running on Vsphere?

well you can use the metallb

@tall-traffic-61849 has left the channel

I'm trying to provision a k8s cluster on vSphere 6.7u3 using a node template. I must be missing something but shouldn't I be seeing the menu item to use or create a template when creating a cluster like in this image? Rancher v2.7.5 on RKE2 v1.25.12+rke2r1. Target: vSphere 6.7.0.54000 (6.7 U3s). Thanks.

Hi all,

Hi all, We have deployed Rancher v2.7.5 on RKE2 v1.25.12+rke2r1 and we try to spin up RKE2 v1.26.7+rke2r1 clusters on vSphere. The clusters seem to be operational however, in the Rancher UI they remain in a "Provisioning" state. The error is the following

Error while applying agent YAML, it will be retried automatically: exit status 1, Error from server (NotFound): the server could not find the requested resource

. Any ideas how to tshoot this issue further?

2023/08/24 15:41:38 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-n8z49bk5: ClusterUnavailable 503: cluster not found, requeuing
2023/08/24 15:41:51 [ERROR] Error during subscribe websocket: close sent
2023/08/24 15:42:08 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-n8z49bk5: ClusterUnavailable 503: cluster not found, requeuing
2023/08/24 15:42:38 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-n8z49bk5: ClusterUnavailable 503: cluster not found, requeuing
2023/08/24 15:43:04 [INFO] EnsureSecretForServiceAccount: waiting for secret [cluster-d9309f-bootstrap-template-v6f6n-machine-plan-tokengxw5k] to be populated with token

Hi All, I updated the Vsphere access credentials but after updating the Cloud credentials, the nodes are stuck in "Failed to delete" upon scaling in and out. Please advise what could be the issue here. Failed deleting server [fleet-default/rancher-prod-2-worker-node-1f0d750b-dbhrj] of kind (VmwarevsphereMachine) for machine rancher-prod-2-worker-node-6fd99d547c-f66mx in infrastructure provider: DeleteError: About to remove rancher-prod-2-worker-node-1f0d750b-dbhrj WARNING: This action will delete both local reference and remote instance. Error removing host "rancher-prod-2-worker-node-1f0d750b-dbhrj": ServerFaultCode: Cannot complete login due to an incorrect user name or password.

Hi All, hoping someone can point me in the correct direction for an issue I am having with my vsphere CSI. The CSI controller doesn't seem to be able to create volumes any more. Not sure when things broke as this was working a while back, but now I am seeing this error in the logs, "ServerFaultCode {{<nil> None of the datastore in VC is file service enabled.}}". Not sure what that means, am I missing some setting on vsphere? Any assistance would be appreciated.

Hi all , I needed help with the CSI . I am able to install the CSI ( installed with the manifest file as the helm chart that rancher provides was not adding the resizer container ) on the existing cluster using the default rke2 default cpi . Volumes are getting created , I am able to see the PVs in the CNS section of the vcenter . But the thing is when i am trying to check the disk properties , it is showing the disk mode as "dependent" . As shown in the pic below . Is it possible for me to change this mode for the pv created by CSI and can have the disk mode as "independent persistent" ? Any guidance will be appreciated . Thankyou

Hi, I have a rancher install which I used to provision an rke2 cluster on vsphere, I deployed the premade monitoring app (which installed grafana, prometheus, alertmananager AFAIK) and I can see the pods running on the cluster. What I'm not sure of is how to actually access grafana? the last time I deployed monitoring on rancher it created grafana icons allover you could just click on. Whatever I've done here doesn't seem to have done that.