Running into two problems on Harvester. We have Harvester running in the lab connected to Rancher 2.6.9. It works fine. We started out with early versions of both and have upgraded to Harvester 1.1.0 and Rancher 2.6.9. Everything works fine. We’ve since tried to add a new Harvester cluster in our DataCenter, while Harvester can be accessed through rancher, we can’t spin up new clusters. We’ve tried reinstalling Harvester, and various other tests. We also tried installing a fresh Rancher 2.6.6, and then connecting the new Harvester cluster to it and it worked. We then upgraded to 2.6.9 and it also worked. We even tried upgrading to 2.7.0 and it also worked. We then installed a fresh 2.6.9 and it did not work. So there is something from 2.6.6 that is left behind in the install that allows Harvester to work. Any help/guidance is greatly appreciated. We have also commented on the following bug in Rancher as it describes the full effect. https://github.com/rancher/rancher/issues/38863

do you see any errors when new clusters cant be spun up? any error in the rancher logs

This is the log from the pod in fleet-default that is failing:

Downloading driver from <https://<domain>/assets/docker-machine-driver-harvester>

Mon, Nov 14 2022 3:14:36 pm

Doing /etc/rancher/ssl

Mon, Nov 14 2022 3:14:39 pm

docker-machine-driver-harvester

Mon, Nov 14 2022 3:14:39 pm

docker-machine-driver-harvester: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped

Mon, Nov 14 2022 3:14:39 pm

Trying to access option which does not exist

Mon, Nov 14 2022 3:14:39 pm

THIS ***WILL*** CAUSE UNEXPECTED BEHAVIOR

Mon, Nov 14 2022 3:14:39 pm

Type assertion did not go smoothly to string for key

Mon, Nov 14 2022 3:14:39 pm

Running pre-create checks...

Mon, Nov 14 2022 3:14:41 pm

Error with pre-create check: “Get \“<https😕/&lt;domain&gt;/k8s/clusters/c-m-92n86znn/apis/harvesterhci.io/v1beta1/settings/server-version&gt;\“: x509: certificate signed by unknown authority”

Mon, Nov 14 2022 3:14:42 pm

The default lines below are for a sh/bash shell, you can specify the shell you’re using, with the --shell flag.

Mon, Nov 14 2022 3:14:42 pm