This message was deleted.
# rke2
a
This message was deleted.
a
Is
ens160
the correct interface name for your VM?
I'd also check to see if the deployed VM has its hostname changed - a quick way to determine if cloud init has ran properly
s
Yes , coz i tried applying vapps variable directly on my template and used this cloud-config the vm gets created but when i am specifying the vapp values in rancher and then creating a node it shows doesnt assign any ip ,netmask or anything on the node
So should i run a simple hostname change command from cloud config ?
a
No. The cloud config rancher generates will cause cloud init to change the hostname of the node
If it hasn't. Cloud init hasnt run
s
Yes that is getting generated rancher assigns a hostname to the node , like i said vapps if attached to vmtemplate is working but when i pass those values from rancher it goes blank
Copy code
vmtoolsd --cmd 'info-get guestinfo.ovfEnv' > /tmp/ovfenv
This command then doesnt shows anything then
a
You don't need to attach the vapps config to the template
In fact, I wouldn't
s
Well that is a workaround till the time i am able to get it working from rancher .
a
Have you tried it without setting vapp options in the template?
s
Yes i have so i basically gave all access on the vapps to the service account . Enabled vapps as per this —> https://blog.connley.net/2023/01/16/easy-static-ip-configuration-for-rancher-nodes/ Then provided values from ranchee but its not working , using ubuntu 22.04 template for this
a
Does the deployed VM have vapp options assigned?
s
Yes they have , i validated that by creating a vm manually using vapp and adding those attributes there
Copy code
com.vmware.guestinfo    IPv4    fixedAllocated

guestinfo.interface.0.ip.0.address  ip:atl2_dvp_devtestk8_922

guestinfo.interface.0.ip.0.netmask  ${netmask:atl2_dvp_devtestk8_922}

guestinfo.interface.0.route.0.gateway   ${gateway:atl2_dvp_devtestk8_922}

guestinfo.dns.servers   ${dns:atl2_dvp_devtestk8_922}
That worked manually
Everything got assigned as expected
a
I mean, does the VM deployed by the rancher provisioning process get vapp options?
s
Yes thats enabled on the template level now
a
Remove the vapp bindings from the template. Use that template in rancher and determine if it inherits vapp settings
s
Tried that as well but then it doesnt provide anything to the vm. Basically gives an error saying ip not found expected x.x.x.x/24 got only /24 or sometimes shows CA error ( that rancher usually shows when ip is not assigned to the node) https://blog.connley.net/2023/01/16/easy-static-ip-configuration-for-rancher-nodes/ is the process of enabling vapps correct on this that i am following ?
a
Sounds like the vapps options didn't get parsed. Check the VM that's deployed and the vapp options are populated
IE from vcenter
s
Vapp options if passed from rancher are not getting populated on the nodes on vcenter
a
Then it's likely an issue in the cluster config
Vcenter bails on setting vapp options if you get a reference wrong
s
Copy code
vmtoolsd --cmd 'info-get guestinfo.ovfEnv'

And thats why this command shows no values tried running it on vm
I am using these
Copy code
com.vmware.guestinfo    IPv4    fixedAllocated
Copy code
guestinfo.interface.0.ip.0.address  ip:atl2_dvp_devtestk8_922
Copy code
guestinfo.interface.0.ip.0.netmask  ${netmask:atl2_dvp_devtestk8_922}
Copy code
guestinfo.interface.0.route.0.gateway   ${gateway:atl2_dvp_devtestk8_922}
Copy code
guestinfo.dns.servers   ${dns:atl2_dvp_devtestk8_922}
And they work if provided directly on vm on vcenter but doesnt work from rancher
If its issue with cluster config what do you suggest ? I can share the terraform code reference that i basically used to set up my cluster
Copy code
Fetching the vsphere credential metadata from the rancher

data "rancher2_cloud_credential" "auth" {
  name = "vsphereprgx"
}


# creating a cluster using the node pools

resource "rancher2_cluster_v2" "foo-rke2" {
  name                                     = var.cluster_name
  kubernetes_version                       = var.kubernetes_version # which kubernetes version is to be deployed
  enable_network_policy                    = var.enable_network_policy
  default_cluster_role_for_project_members = var.default_cluster_role_for_project_members
  rke_config {

    # creating the machine global config

    machine_global_config = <<EOF
      cni: ${var.cni}
      disable-kube-proxy: false
      etcd-expose-metrics: false
      kubelet-arg:
max-pods=250    
    EOF

    # creating dynamic machine pools

    dynamic "machine_pools" {
      for_each = var.node
      content {
        cloud_credential_secret_name = data.rancher2_cloud_credential.auth.id
        control_plane_role           = machine_pools.key == "ctl_plane" ? true : false
        etcd_role                    = machine_pools.key == "ctl_plane" ? true : false
        name                         = machine_pools.value.name
        quantity                     = machine_pools.value.quantity
        worker_role                  = machine_pools.key != "ctl_plane" ? true : false

        machine_config {
          kind = rancher2_machine_config_v2.machineconfig[machine_pools.key].kind
          name = replace(rancher2_machine_config_v2.machineconfig[machine_pools.key].name, "_", "-")
        }
      } # End of dynamic for_each content
    }   # End of machine_pools
  }
}

# creating a machine config to be used while creating the cluster

resource "rancher2_machine_config_v2" "machineconfig" {
  for_each      = var.node
  generate_name = "${each.value.name}-config"

  vsphere_config {
    cfgparam      = ["disk.enableUUID=TRUE"] # Disk UUID is Required for vSphere Storage Provider ( mandatory for the cluster to work )
    clone_from    = var.vsphere_env.cloud_image_name
    cloud_config  = file("/root/terraform/cloud.yaml", )
    cpu_count     = each.value.cpu
    creation_type = "template"
    datacenter    = var.vsphere_env.datacenter
    datastore     = var.vsphere_env.datastore
    disk_size     = each.value.disk_size
    memory_size   = each.value.ram
    network       = var.vsphere_env.vm_network
    folder        = var.vsphere_env.folder
    pool          = var.vsphere_env.pool
  }
}
a
Use this as a reference. Make sure you use the correct case ie guestInfo Vs guestinfo
For the transport
Try outside of terraform first
s
Oh okay let me try with with this guestInfo in transport , will update here
I have tried it but the rancher dashboard is showing node is waiting for cluster agent to apply initial plan and when checking from the vaphere no vapps options got applied
a
And vcenter doesn't show the IP of the node?
s
Yes there is no ip i even tried what randel has written as comment under your vapps blog regarding the usage of guestInfo instead of guestinfo . But that didnt worked either
a
If the vapp config isn't present in vcenter when looking at the VMs properties then there's likely an issue with the config set in the cluster config.
s
What do you suggest where should i look at
a
Double check you're referencing the right port group and it has the network profile assigned etc.
s
yes David , it has actually because it is working when applying those vapps options manually , i am gonna try it from scratch now validating everything up at every step
Also , i am not performing these steps
Copy code
Edit: This post has been updated to reflect changes in newer versions of Rancher.

Note: As mentioned by Jonathan in the comments, disabling cloud-init's initial network configuration is recommended. To do this, create a file:

/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg

To contain:

network: {config: disabled}

In your VM template.

How networking configuration is applied to k8s nodes (or VM's in general) in on-premises environments is usually achieved by one of two ways – DHCP or static. For some, DHCP is not a popular option and static addresses can be time-consuming to manage, particularly when there's no IPAM feature in Rancher. In this blog post I go through how to leverage vSphere Network Protocol Profiles in conjunction with Rancher and Cloud-Init to reliably, and predictably apply static IP addresses to deployed nodes.
but i am removing the
50-cloud-init.yaml
before creating a vm template , evven then should i add this 99-disable-network-config.cfg
i made it work finally
you were right so basically it was not able to assign the ip and other things because of the cloud-config
Copy code
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
i created this file in the template and ran this script as well --> https://github.com/David-VTUK/Rancher-Packer/blob/d2822931a12b1967f4c3362072c5f386fb4eb57f/vSphere/ubuntu_2204/script.sh#L22
Thankyou so much for the guidance 🙌
218 Views