Rancher-Server v2.7 is installed on a docker. I downloaded a Ubuntu 20.04 cloud vm template from https://cloud-images.ubuntu.com/focal/current/ and created a template on vsphere. Using the Rancher Server UI, created a cluster on vsphere env. Added the add-on config in the UI as well for CPI and CSI. Nodes gets created but it gets stuck in

Waiting for cluster agent to connect

. Any suggestions on what is missing?

seeing this error on

vsphere-cpi-cloud-controller-manager

on the node

2023-01-24T11:33:34.685454772Z stderr F I0124 11:33:34.685331       1 requestheader_controller.go:244] Loaded a new request header values for RequestHeaderAuthRequestController
2023-01-24T11:33:34.685694867Z stderr F W0124 11:33:34.685637       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
2023-01-24T11:33:34.686149571Z stderr F I0124 11:33:34.686093       1 main.go:160] vsphere-cloud-controller-manager version: v1.24.2
2023-01-24T11:33:34.68642421Z stderr F E0124 11:33:34.686367       1 config_yaml.go:136] vsphere.conf does not have the VirtualCenter IP address specified
2023-01-24T11:33:34.686430714Z stderr F E0124 11:33:34.686386       1 config_yaml.go:214] validateConfig failed: vsphere.conf does not have the VirtualCenter IP address specified
2023-01-24T11:33:34.686434755Z stderr F W0124 11:33:34.686392       1 config.go:69] ReadCPIConfigYAML failed: vsphere.conf does not have the VirtualCenter IP address specified
2023-01-24T11:33:34.686468138Z stderr F E0124 11:33:34.686414       1 config.go:73] ReadConfigINI failed: 3:1: expected section header
2023-01-24T11:33:34.686472835Z stderr F F0124 11:33:34.686427       1 main.go:265] Cloud provider could not be initialized: could not init cloud provider "vsphere": 3:1: expected section header
2023-01-24T11:33:34.686850604Z stderr F goroutine 1 [running]:
2023-01-24T11:33:34.686856726Z stderr F <http://k8s.io/klog/v2.stacks(0x1)|k8s.io/klog/v2.stacks(0x1)>
2023-01-24T11:33:34.686861036Z stderr F         /go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:860 +0x8a
2023-01-24T11:33:34.686864956Z stderr F <http://k8s.io/klog/v2.(*loggingT).output(0x3d99f80|k8s.io/klog/v2.(*loggingT).output(0x3d99f80>, 0x3, 0x0, 0xc0000e7f80, 0x1, {0x2f81483, 0x1}, 0xc000500000, 0x0)
2023-01-24T11:33:34.686868724Z stderr F         /go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:825 +0x686
2023-01-24T11:33:34.686872582Z stderr F <http://k8s.io/klog/v2.(*loggingT).printfDepth(0x3d99f80|k8s.io/klog/v2.(*loggingT).printfDepth(0x3d99f80>, 0xb52d0108, 0x0, {0x0, 0x0}, 0x0, {0x2579198, 0x2b}, {0xc000346a10, 0x1, ...})
2023-01-24T11:33:34.686877114Z stderr F         /go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:630 +0x1f2
2023-01-24T11:33:34.686881311Z stderr F <http://k8s.io/klog/v2.(*loggingT).printf(...)|k8s.io/klog/v2.(*loggingT).printf(...)>
2023-01-24T11:33:34.686884951Z stderr F         /go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:612
2023-01-24T11:33:34.686888634Z stderr F <http://k8s.io/klog/v2.Fatalf(...)|k8s.io/klog/v2.Fatalf(...)>
2023-01-24T11:33:34.686892695Z stderr F         /go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:1516

check if You have ip address assigned to this server, it looks like networking failure

vm*

the node and the rancher server host are reachable

root@direct-sheepdog-ctl-plane-60d5f517-b2l66:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:50:56:b5:6c:73 brd ff:ff:ff:ff:ff:ff
    inet 10.221.42.187/24 brd 10.221.42.255 scope global dynamic ens192
       valid_lft 34451sec preferred_lft 34451sec
    inet6 fe80::250:56ff:feb5:6c73/64 scope link
       valid_lft forever preferred_lft forever
root@direct-sheepdog-ctl-plane-60d5f517-b2l66:~# ping 10.221.41.63
PING 10.221.41.63 (10.221.41.63) 56(84) bytes of data.
64 bytes from 10.221.41.63: icmp_seq=1 ttl=63 time=0.559 ms
64 bytes from 10.221.41.63: icmp_seq=2 ttl=63 time=0.765 ms
64 bytes from 10.221.41.63: icmp_seq=3 ttl=63 time=0.521 ms
64 bytes from 10.221.41.63: icmp_seq=4 ttl=63 time=0.841 ms
^C
--- 10.221.41.63 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3066ms
rtt min/avg/max/mdev = 0.521/0.671/0.841/0.134 ms

this is ip of rancher service? or master node?

What ingess controller You are using? if any

For Rancher Server? Nothing. It is on docker.

~# docker ps
CONTAINER ID   IMAGE                    COMMAND           CREATED       STATUS       PORTS                                                                      NAMES
6791f8a001c8   rancher/rancher:latest   "entrypoint.sh"   13 days ago   Up 13 days   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   intelligent_goodall

i'm working on rancher on another kubernetes stack, so i'm not sure how could i help You