rancher-users #vsphere

Channels

vsphere

stale-magazine-62677

02/06/2023, 1:02 PM

Hello vsphere users, I am trying to use the vSphere cloud provider for use with the vSAN CSI Storage driver The cluster deploys fine, but fail when starting "rancher-vsphere-cpi-cloud-controller-manager" and continuously crash-loops. The pod crashes with the following error message:

I0206 12:33:14.147902       1 serving.go:348] Generated self-signed cert in-memory
I0206 12:33:14.943719       1 requestheader_controller.go:244] Loaded a new request header values for RequestHeaderAuthRequestController
W0206 12:33:14.944385       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0206 12:33:14.945319       1 main.go:160] vsphere-cloud-controller-manager version: v1.24.2
E0206 12:33:14.946287       1 config_yaml.go:136] vsphere.conf does not have the VirtualCenter IP address specified
E0206 12:33:14.946305       1 config_yaml.go:214] validateConfig failed: vsphere.conf does not have the VirtualCenter IP address specified
W0206 12:33:14.946423       1 config.go:69] ReadCPIConfigYAML failed: vsphere.conf does not have the VirtualCenter IP address specified
E0206 12:33:14.946558       1 config.go:73] ReadConfigINI failed: 3:1: expected section header
F0206 12:33:14.946582       1 main.go:265] Cloud provider could not be initialized: could not init cloud provider "vsphere": 3:1: expected section header

The vsphere-cloud-config ConfigMap looks empty

apiVersion: v1
data:
  vsphere.yaml: # Global properties in this section will be used for all specified
    vCenters unless overriden in VirtualCenter section.\n\nglobal:\n  secretName:
    \"vsphere-cpi-creds\"\n  secretNamespace: \"kube-system\"\n  port: 443\n  insecureFlag:
    true\n\nvcenter:\n  \"\":\n    server: \"\"\n    user: \n    password: \n    datacenters:\n
    \     - \"\"\n
kind: ConfigMap
metadata:
  annotations:
    <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: rancher-vsphere-cpi
    <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: kube-system
....

The vsphere-config-secret is populated with the correct configuration But the vsphere-cpi-creds looks empty:

apiVersion: v1
data:
  .password: ""
  .username: ""
kind: Secret
metadata:
  annotations:
    <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: rancher-vsphere-cpi
    <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: kube-system

Has anyone here run in to a similar issue?

hundreds-twilight-31144

02/08/2023, 6:54 AM

Hello, i am new to the community. Please if you can help me where to look. Install rancher 2.7.1 on ks3 1.24 on vsphere 7.0u3. Problem is that i am unable to create RKE cluster. DHCP is not enabled. The only message that i got timeout ssh key. In logs of vcentar api i see that user is logged in but dont see any other action like create vm from template. I try different templates and different variations and everytime message is the same. I stuck with this error for 10 days..

brash-jewelry-40212

02/15/2023, 7:28 PM

Hello, i am new to rancher and vsphere. I am trying to setup the cluster on vsphere. I can see that nginx ingress controller was created on cluster with with cluster-ip and ingress class name is nginx. How to use this nginx ingress class to create the ingress rules for the cluster As per my requirement i want to create ingress controller as load balancer to expose the cluster-ip applications services. Any suggestions on how to deal this situation on vsphere environment ?

brash-jewelry-40212

02/16/2023, 7:32 AM

Hello , I am trying setup the storage class in vsphere cluster created by rancher. I have installed CPI chart with vpshere credentials and CSI already installed. Created storage class with vmwareVpsherevolume provisioner. When i am trying to create pvc , its failing and showing error that f*ailed to provision the volume with storageclass "vspehre-test": cloud provider not initiated properly.* How to troubleshoot this issues ? Please help.

brash-jewelry-40212

02/16/2023, 1:12 PM

Hello all, I am using rancher 2.7.1 and created vsphere cluster k8s with v1.24.9 . When i am trying to install CSI as per official documents, from Apps--> charts , i could not find that chart(vSphere CSI). How to get the vSphere CSI chart ? Please help.

brash-jewelry-40212

02/16/2023, 5:23 PM

Hello Everyone, I have k8s cluster running on vsphere. To expose the services on cluster, i want to create a HA-proxy in-front of my k8s worker nodes which can be auto-healed by rancher. challenge: How to configure HA-proxy to send the traffic to healthy and dynamically changed nodes in self healed node-pool ? Has anyone here implemented this kind of setup ? Please let me know your suggestions / point to the right documentation ? Thanks in advance.

gray-psychiatrist-7159

02/17/2023, 10:00 AM

@brash-jewelry-40212 Hi. I'm also trying to deploy downstream K8S clusters with Rancher 2.7.1 (K3s with 1 replica for Rancher Manager) in VMWare vSphere cluster.

gray-psychiatrist-7159

02/17/2023, 10:01 AM

I'm actualy not in advanced steps as you seems to bee, but I have some working stuff in my Terraform code

gray-psychiatrist-7159

02/17/2023, 10:05 AM

I'm actually able to create 3 master and 1 worker nodes in vSphere with Terraform, but the cluster is not able to initialize because I have not yet implemented

rancher2_cluster_sync

and terraform get timeouts

gray-psychiatrist-7159

02/17/2023, 10:07 AM

But I would be curious to discuss with you to know how you went about your CSI setup and stuff like that

gray-psychiatrist-7159

02/17/2023, 10:10 AM

Did you have some issues about deleting rke2 clusters in Rancher manager when Terraform deployments failed ?

microscopic-telephone-96049

02/22/2023, 4:31 PM

Hi! We want to upgrade our vmware cluster from 7U3 to 8. Is vsphere-csi-plugin version 2.5.1 (This version is in the current latest rancher catalog) compatible with vsphere 8? We didn't dare until now, because the official recommendation is vsphere-csi-plugin 2.7 for vmware 8. The affected clusters are RKE1 and the kubernetes version is 1.23.14, and the Rancher version is 2.7.1. Thanks!

dry-policeman-72946

03/02/2023, 4:45 PM

I face the same issue as https://rancher-users.slack.com/archives/CBEB5A0H1/p1676553176768709 Did anybody else notice that the vSphere-CSI chart has disappeared from the Apps list? And now the existing installations can't be upgraded.

millions-ocean-48249

03/03/2023, 4:42 PM

Hi everyone! I am running into a bit of an odd issue this morning trying to get a hybrid (windows/linux) cluster working with the vsphere cloud provider. I deployed the rke2 cluster as "custom", selected the vsphere cloud provider, filled in the blanks, added the extra manifests as described here and everything seems to have started up okay, except my cloud-controller-manager is reporting issues for both of my windows nodes and cant initialize them with the following error:

millions-ocean-48249

03/03/2023, 4:42 PM

I0303 16:39:27.155021 1 node_controller.go:391] Initializing node sre-b89d40bd1vq with cloud provider
I0303 16:39:27.155095 1 search.go:76] WhichVCandDCByNodeID nodeID: sre-b89d40bd1vq
I0303 16:39:27.181513 1 search.go:208] Found node sre-b89d40bd1vq as vm=VirtualMachine:vm-18616 in vc=x.x.x.x and datacenter=dc1
I0303 16:39:27.181675 1 search.go:210] Hostname: SRE-B89D40BD1VQ, UUID: 4211dcfa-7a40-fae2-bd0c-d694c118c50b
I0303 16:39:27.181764 1 nodemanager.go:127] Discovered VM using FQDN or short-hand name
E0303 16:39:27.187892 1 node_controller.go:213] error syncing 'sre-b89d40bd1vq': failed to get provider ID for node sre-b89d40bd1vq at cloudprovider: failed to get instance ID from cloud provider: VM GuestNicInfo is empty, requeuing

millions-ocean-48249

03/03/2023, 4:43 PM

I was able to see VMTools showing the ip address in the vsphere ui before I added it to the cluster, but after adding it and letting it register it disappeared...

millions-ocean-48249

03/03/2023, 4:44 PM

$ govc <http://vm.info|vm.info> -json vm-name | jq .VirtualMachines[0].Guest | grep Net
  "Net": null,

millions-ocean-48249

03/03/2023, 4:49 PM

linux rke2 version: v1.24.9+rke2r2 windows rke2 version: v1.24.9 rancher version: 2.7.1 vsphere version: 7.0.3 linux nodes: ubuntu 22.04 (open-vm-tools) windows nodes: windows server core 2019 ltsc (vmtools version:12320)

millions-ocean-48249

03/03/2023, 10:43 PM

Ive had zero luck with this. I tried to provision some storage on a pod running on the initialized linux nodes and I cant complete this either, likely because the windows nodes arent getting initialized properly from above...

ProvisioningFailed  4s (x2 over 10s)  persistentvolume-controller  Failed to provision volume with StorageClass "vsphere-gold": cloud provider not initialized properly

steep-zoo-77219

03/08/2023, 6:25 PM

Rancher v2.5.12 - all of a sudden getting “Rancher Error in driver during machine creation” when scaling with vsphere.. anyone had anything similar? it worked a few days ago, and all of a sudden -stopped?

hundreds-evening-84071

03/08/2023, 6:44 PM

Hello all, On RKE2 cluster... working on vSphere CSI Configuration I created conf file as shown in example here (for CSI vSphere Config Secret): But it is not working. I get error "no Virtual Center hosts defined" makes no sense because there is an entry for VirtualCenter. Should I be following a different example to create the CSI vSphere Config Secret?

hundreds-evening-84071

03/08/2023, 8:16 PM

Is there any advantages/disadvantages of setting vSphere as Cloud Provider during cluster creation (in Rancher UI)? Or should I always set to External and install vSphere CPI/CSI from Apps?

salmon-noon-33588

03/09/2023, 5:35 PM

Hey uh...we're trying to upgrade to the newer rancher-vsphere-cpi chart and it's crashing because there seems to be a broken comment in the configMap:

vsphere.yaml: >
    # Global properties in this section will be used for all specified vCenters
    unless overridden in VirtualCenter section.

hundreds-evening-84071

03/10/2023, 4:58 PM

Is there a flag or setting that I missed where if a PV is deleted that should also cleanup in vsphere? Currently (for me) when PV is deleted the objects are not removed in vsphere...

limited-elephant-16797

03/14/2023, 9:59 AM

Hi All, I'm attempting to set up an on-premises K8S cluster with Rancher on Vsphere. I'm using a Docker container to access the Rancher Management UI, and my concern is how I'll access the configured management UI again if the Docker container dies for any reason. Do you also recommend this method for production workloads?

hundreds-evening-84071

03/17/2023, 2:06 PM

Hello folks, is there a similar document for Rancher v2.7.x? https://ranchermanager.docs.rancher.com/v2.5/how-to-guides/new-user-guides/kubernetes-cluste[…]nodes-in-an-infra-provider/vsphere/create-credentials Or same one is still applicable from 2.5 version?

white-xylophone-9961

03/17/2023, 2:12 PM

@white-xylophone-9961 has left the channel

prehistoric-advantage-39331

03/17/2023, 2:41 PM

Has anyone seen issues with Ubuntu 22.04 nodes in rancher vsphere clusters? I’m running into ssh errors when I try to deploy 22.04 nodes, but it works fine for 20.04.

bored-architect-10999

03/17/2023, 2:58 PM

We ran into issues with 22.04 but they were not ssh related. Our problem appears to be related to https://github.com/rancher/rancher/issues/38816 We have had to stay on 20.04 ubuntu for our clusters

prehistoric-advantage-39331

03/17/2023, 3:04 PM

Yeah, we never got that far. The nodes never deploy at all due to ssh failures.