Hello, i am new to the community. Please if you can help me where to look. Install rancher 2.7.1 on ks3 1.24 on vsphere 7.0u3. Problem is that i am unable to create RKE cluster. DHCP is not enabled. The only message that i got timeout ssh key. In logs of vcentar api i see that user is logged in but dont see any other action like create vm from template. I try different templates and different variations and everytime message is the same. I stuck with this error for 10 days..

So you see the user logged into vcenter that you're using in your cloud credential in Rancher but no vm cloning activity? • Have you checked the vcenter server vpxd logs for that user? • Have you tried logging into vcenter with the these user credentials to verify it has access to the vm templates etc?

in rancher logs

In the local

Rancher

cluster, look for a

job

in the fleet-default namespace that resembles your node name. You'll get more specific provisioning logs from there

i can not find any job in fleet-default namespace. root@enes-rancher:~# kubectl get jobs --all-namespaces NAMESPACE NAME COMPLETIONS DURATION AGE kube-system helm-install-traefik 1/1 26s 4d23h kube-system helm-install-traefik-crd 1/1 23s 4d23h Pool: vafnode VMware vSphere – No Location / 2.15 G, 2 Core (template_no_dhcp) 0 of 3 Saving m-4kqxv — - — All 2.2 mins Error with pre-create check: "unexpected EOF":Timeout waiting for ssh key Saving m-p5j6b — - — All 2.2 mins Error with pre-create check: "unexpected EOF":Timeout waiting for ssh key Saving m-z5nlf — - — All 2.2 mins Error with pre-create check: "unexpected EOF":Timeout waiting for ssh key Condition Status Updated Message BackingNamespaceCreated True 2.5 hours ago — Connected False 2.5 hours ago — Created True 2.5 hours ago — CreatorMadeOwner True 2.5 hours ago — DefaultProjectCreated True 2.5 hours ago — InitialRolesPopulated True 2.5 hours ago — NoDiskPressure True 2.5 hours ago — NoMemoryPressure True 2.5 hours ago — Pending True - — Reconciling True 2.5 hours ago [Reconciling] RKECluster True 2.5 hours ago — RKESecretsMigrated True 2.5 hours ago — SecretsMigrated True 2.5 hours ago — ServiceAccountSecretsMigrated True 2.5 hours ago — Stalled False 2.5 hours ago — SystemProjectCreated True 2.5 hours ago — Waiting Unknown 2.5 hours ago Waiting for API to be available

It'll be removed after a while, I would try and provision a new cluster and monitor that ns for the job

When i try to create rke2 cluster i see the job, with creation of rke1 cluster noting happen This is the error messages from that process events Active 4m46s Warning BackoffLimitExceeded job/vknode-pool1-aad64d9d-6qcwz-machine-provision job-controller Job has reached the specified backoff limit rancher logs 2023/02/08 13:02:12 [ERROR] error syncing 'fleet-default/vknode-bootstrap-template-g79kh': handler rke-machine: failed to delete fleet-default/vknode-bootstrap-template-g79kh-machine-plan /v1, Kind=Secret for rke-machine fleet-default/vknode-bootstrap-template-g79kh: secrets "vknode-bootstrap-template-g79kh-machine-plan" not found, failed to delete fleet-default/vknode-bootstrap-template-g79kh-machine-bootstrap /v1, Kind=ServiceAccount for rke-machine fleet-default/vknode-bootstrap-template-g79kh: serviceaccounts "vknode-bootstrap-template-g79kh-machine-bootstrap" not found, failed to delete fleet-default/vknode-bootstrap-template-g79kh-machine-plan rbac.authorization.k8s.io/v1, Kind=RoleBinding for rke-machine fleet-default/vknode-bootstrap-template-g79kh: rolebindings.rbac.authorization.k8s.io "vknode-bootstrap-template-g79kh-machine-plan" not found, requeuing 2023/02/08 13:02:12 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:12 [ERROR] [CAPI] Reconciler error: machines.cluster.x-k8s.io "vknode-pool1-85c45798dc-k2np7" not found 2023/02/08 13:02:13 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:13 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:14 [ERROR] error syncing 'fleet-default/vknode-pool1-aad64d9d-2kdhg': handler machine-provision: no machine owner ref, requeuing 2023/02/08 13:02:14 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:14 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:14 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:14 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-p99n6 2023/02/08 13:02:15 [ERROR] error syncing 'local/crtb-9w4q8': handler mgmt-auth-crtb-controller: couldn't create role cluster-owner: namespaces "p-p99n6" not found, requeuing 2023/02/08 13:02:15 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:16 [INFO] EnsureSecretForServiceAccount: waiting for secret [vknode-bootstrap-template-8sbhm-machine-bootstrap-token-j9f6v] to be populated with token 2023/02/08 13:02:16 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:16 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:17 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:17 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:17 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:18 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:20 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:36 [INFO] [MachineProvision] Failed to create infrastructure fleet-default/vknode-pool1-aad64d9d-2kdhg for machine vknode-pool1-85c45798dc-vjr7t, deleting and recreating... 2023/02/08 13:02:36 [INFO] [MachineProvision] Failed to create infrastructure fleet-default/vknode-pool1-aad64d9d-2kdhg for machine vknode-pool1-85c45798dc-vjr7t, deleting and recreating... 2023/02/08 13:02:36 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:36 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:36 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:37 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:37 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:37 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:38 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:44 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:45 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:45 [ERROR] error syncing 'fleet-default/vknode-bootstrap-template-8sbhm': handler rke-machine: failed to delete fleet-default/vknode-bootstrap-template-8sbhm-machine-plan /v1, Kind=Secret for rke-machine fleet-default/vknode-bootstrap-template-8sbhm: secrets "vknode-bootstrap-template-8sbhm-machine-plan" not found, failed to delete fleet-default/vknode-bootstrap-template-8sbhm-machine-bootstrap /v1, Kind=ServiceAccount for rke-machine fleet-default/vknode-bootstrap-template-8sbhm: serviceaccounts "vknode-bootstrap-template-8sbhm-machine-bootstrap" not found, failed to delete fleet-default/vknode-bootstrap-template-8sbhm-machine-plan /v1, Kind=ServiceAccount for rke-machine fleet-default/vknode-bootstrap-template-8sbhm: serviceaccounts "vknode-bootstrap-template-8sbhm-machine-plan" not found, requeuing 2023/02/08 13:02:45 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:46 [ERROR] [CAPI] Reconciler error: machines.cluster.x-k8s.io "vknode-pool1-85c45798dc-vjr7t" not found 2023/02/08 13:02:46 [ERROR] error syncing 'fleet-default/vknode-pool1-aad64d9d-nskmb': handler machine-provision: no machine owner ref, requeuing 2023/02/08 13:02:46 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:46 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:46 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:47 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:48 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node 2023/02/08 13:02:48 [INFO] [planner] rkecluster fleet-default/vknode: waiting: waiting for viable init node

What are the logs from vknode-pool1-aad64d9d-6qcwz-machine-provision?

Hello, i create another cluster the same error Running pre-create checks... (vk8node-pool1-9412b1b0-qbd7p) Connecting to vSphere for pre-create checks... Error with pre-create check: "unexpected EOF" The default lines below are for a sh/bash shell, you can specify the shell you're using, with the --shell flag.

I'd check the VPXD logs next, looks like your vcenter server is returning something invalid

Hello, i inspect logs and didnt find anything strange.. just to mention that network in vcentar is using distributed switch and we have two clusters separated..

Interesting. You shouldn't need to, vds should work out of the box

Thank you very much David