https://rancher.com/ logo
s

steep-zoo-77219

03/08/2023, 6:25 PM
Rancher v2.5.12 - all of a sudden getting “Rancher Error in driver during machine creation” when scaling with vsphere.. anyone had anything similar? it worked a few days ago, and all of a sudden -stopped?
a

agreeable-oil-87482

03/08/2023, 7:05 PM
Can you post the rancher logs?
s

steep-zoo-77219

03/08/2023, 7:28 PM
There is really not much to go on; but here in debug mode:
Copy code
2023/03/08 10:17:15 [INFO] [node-controller-rancher-machine] (k8s-em-wkr1) Adding network: /DC/network/VLAN123
2023/03/08 10:17:15 [INFO] [node-controller-rancher-machine] (k8s-em-wkr1) Resizing disk disk-1000-0 up from 52428800Kb to 53248000Kb
2023/03/08 10:17:16 [INFO] [node-controller-rancher-machine] (k8s-em-wkr1) Setting disk.enableUUID to TRUE
2023/03/08 10:17:16 [INFO] [node-controller-rancher-machine] (k8s-em-wkr1) Creating cloud-init.iso
2023/03/08 10:17:16 [INFO] [node-controller-rancher-machine] (k8s-em-wkr1) Uploading cloud-init.iso
2023/03/08 10:17:16 [DEBUG] Triggering auth refresh on u-adn6tvozol
2023/03/08 10:17:16 [DEBUG] Skipping refresh for u-adn6tvozol due to max-age
2023/03/08 10:17:16 [DEBUG] Triggering auth refresh on u-adn6tvozol
2023/03/08 10:17:16 [DEBUG] Skipping refresh for u-adn6tvozol due to max-age
2023/03/08 10:17:18 [INFO] [node-controller-rancher-machine] The default lines below are for a sh/bash shell, you can specify the shell you're using, with the --shell flag.
2023/03/08 10:17:18 [DEBUG] [GenericEncryptedStore]: set secret called for mc-m-q8ghk
2023/03/08 10:17:18 [DEBUG] [GenericEncryptedStore]: updating secret mc-m-q8ghk
2023/03/08 10:17:18 [INFO] [node-controller-rancher-machine]
2023/03/08 10:17:21 [INFO] Generating and uploading node config k8s-em-wkr1
2023/03/08 10:17:21 [DEBUG] getNodeTemplate parsed [cattle-global-nt:nt-dfxst] to ns: [cattle-global-nt] and n: [nt-dfxst]
2023/03/08 10:17:21 [DEBUG] Cleaning up [/opt/jail/c-rsw6m/management-state/node/nodes/k8s-em-wkr1]
2023/03/08 10:17:21 [ERROR] error syncing 'c-rsw6m/m-q8ghk': handler node-controller: Error creating machine: Error in driver during machine creation: 500 Internal Server Error, requeuing
I had it on trace mode; but got spammed out with SOAP calls 😄
a

agreeable-oil-87482

03/08/2023, 7:29 PM
Can you enable trace logging? And is this an rke1 cluster?
s

steep-zoo-77219

03/08/2023, 7:29 PM
RKE1, correct.
I have trace logs - what is it you need to see? there is a lot of info there..
a

agreeable-oil-87482

03/08/2023, 7:30 PM
Last soap call before it logs the error
s

steep-zoo-77219

03/09/2023, 9:38 AM
Hi David, our server dept found something in the vSphere logs, that pointed to some crypto permission that was missing. When re-enabled, this worked again. Thanks for the support
13 Views