https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# rke2
a
This message was deleted.
c
says the bootstrap node isn’t ready yet.
Check the rancher-system-agent logs on that node, make sure it’s all reporting OK. Check that a Machine exists for this node on the Rancher cluster. Check that cattle-cluster-agent is checking in with Rancher.
look at the capi controller manager logs on the rancher cluster to see what it says.
g
Hey Brandon thanks for your response. I had time to circle back to this.
Copy code
k get machines -A
NAMESPACE       NAME                      CLUSTER   NODENAME                  PROVIDERID                       PHASE          AGE     VERSION
fleet-default   test-pool1-skvmw-2n4kq    test                                                                 Provisioning   2m32s   
fleet-default   test-pool2-42ms5-r4wnm    test                                                                 Provisioning   2m32s
I did see that the machines are not actually provisioned The machine plan for the worker node actually doesn't have the secret populated
Copy code
k get secrets -A | grep pool2

fleet-default                              test-pool2-42ms5-r4wnm-machine-bootstrap                   rke.cattle.io/bootstrap               1      4m5s
fleet-default                              test-pool2-42ms5-r4wnm-machine-bootstrap-token-2dlj9       kubernetes.io/service-account-token   3      4m6s
fleet-default                              test-pool2-42ms5-r4wnm-machine-driver-secret               Opaque                                1      3m56s
fleet-default                              test-pool2-42ms5-r4wnm-machine-plan                        rke.cattle.io/machine-plan            0      4m6s
fleet-default                              test-pool2-42ms5-r4wnm-machine-plan-token-p8fxz            kubernetes.io/service-account-token   3      2m48s
fleet-default                              test-pool2-42ms5-r4wnm-machine-state                       rke.cattle.io/machine-state           1      3m56s
fleet-default                              vultr-pool2-4ncj6-t26m5-machine-bootstrap                  rke.cattle.io/bootstrap               1      4d3h
fleet-default                              vultr-pool2-4ncj6-t26m5-machine-bootstrap-token-wqm65      kubernetes.io/service-account-token   3      4d3h
fleet-default                              vultr-pool2-4ncj6-t26m5-machine-plan                       rke.cattle.io/machine-plan            13     4d3h
fleet-default                              vultr-pool2-4ncj6-t26m5-machine-plan-token-tg6b7           kubernetes.io/service-account-token   3      4d3h
fleet-default                              vultr-pool2-4ncj6-t26m5-machine-state                      rke.cattle.io/machine-state           1      4d3h
Copy code
root@test-pool2-42ms5-r4wnm:~# journalctl -u rancher-system-agent.service 

Jun 23 18:18:49 test-pool2-42ms5-r4wnm systemd[1]: Started Rancher System Agent.
Jun 23 18:18:49 test-pool2-42ms5-r4wnm rancher-system-agent[1435]: time="2025-06-23T18:18:49Z" level=info msg="Rancher System Agent version v0.3.11 (b8c28d0) is starting"
Jun 23 18:18:49 test-pool2-42ms5-r4wnm rancher-system-agent[1435]: time="2025-06-23T18:18:49Z" level=info msg="Using directory /var/lib/rancher/agent/work for work"
Jun 23 18:18:49 test-pool2-42ms5-r4wnm rancher-system-agent[1435]: time="2025-06-23T18:18:49Z" level=info msg="Starting remote watch of plans"
Jun 23 18:18:50 test-pool2-42ms5-r4wnm rancher-system-agent[1435]: time="2025-06-23T18:18:50Z" level=info msg="Starting /v1, Kind=Secret controller"
hmm the 
capi-controller-manger
is waiting for the infrastructure...
Copy code
k logs pod/capi-controller-manager-bd4c76b4c-nbjpk -n cattle-provisioning-capi-system

I0623 18:28:11.846181       1 machine_controller_phases.go:306] "Waiting for infrastructure provider to create machine infrastructure and report status.ready" controller="machine" controllerGroup="cluster.x-k8s.io" controllerKind="Machine" Machine="fleet-default/test-pool1-skvmw-2n4kq" namespace="fleet-default" name="test-pool1-skvmw-2n4kq" reconcileID="6fef1bf4-e2f9-425f-bac2-26f628acc678" MachineSet="fleet-default/test-pool1-skvmw" MachineDeployment="fleet-default/test-pool1" Cluster="fleet-default/test" VultrMachine="fleet-default/test-pool1-skvmw-2n4kq"
I0623 18:28:11.846207       1 machine_controller_noderef.go:60] "Waiting for infrastructure provider to report spec.providerID" controller="machine" controllerGroup="cluster.x-k8s.io" controllerKind="Machine" Machine="fleet-default/test-pool1-skvmw-2n4kq" namespace="fleet-default" name="test-pool1-skvmw-2n4kq" reconcileID="6fef1bf4-e2f9-425f-bac2-26f628acc678" MachineSet="fleet-default/test-pool1-skvmw" MachineDeployment="fleet-default/test-pool1" Cluster="fleet-default/test" VultrMachine="fleet-default/test-pool1-skvmw-2n4kq"
If i look at the 
vultrmachine
config for pool2 it looks to be populated accordingly
Copy code
kubectl get vultrmachine  vultr-pool2-4ncj6-t26m5 -o yaml  -n fleet-default
apiVersion: <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>
kind: VultrMachine
metadata:
  annotations:
    <http://cluster.x-k8s.io/cloned-from-groupkind|cluster.x-k8s.io/cloned-from-groupkind>: <http://VultrMachineTemplate.rke-machine.cattle.io|VultrMachineTemplate.rke-machine.cattle.io>
    <http://cluster.x-k8s.io/cloned-from-name|cluster.x-k8s.io/cloned-from-name>: vultr-pool2-a206879f
  creationTimestamp: "2025-06-19T14:41:52Z"
  finalizers:
  - <http://wrangler.cattle.io/machine-provision-remove|wrangler.cattle.io/machine-provision-remove>
  generation: 2
  labels:
    <http://cattle.io/os|cattle.io/os>: linux
    <http://cluster.x-k8s.io/cluster-name|cluster.x-k8s.io/cluster-name>: vultr
    <http://cluster.x-k8s.io/deployment-name|cluster.x-k8s.io/deployment-name>: vultr-pool2
    <http://cluster.x-k8s.io/set-name|cluster.x-k8s.io/set-name>: vultr-pool2-4ncj6
    machine-template-hash: 722474703-4ncj6
    <http://rke.cattle.io/capi-machine-name|rke.cattle.io/capi-machine-name>: vultr-pool2-4ncj6-t26m5
    <http://rke.cattle.io/cluster-name|rke.cattle.io/cluster-name>: vultr
    <http://rke.cattle.io/rke-machine-pool-name|rke.cattle.io/rke-machine-pool-name>: pool2
    <http://rke.cattle.io/worker-role|rke.cattle.io/worker-role>: "true"
  name: vultr-pool2-4ncj6-t26m5
  namespace: fleet-default
  ownerReferences:
  - apiVersion: <http://cluster.x-k8s.io/v1beta1|cluster.x-k8s.io/v1beta1>
    blockOwnerDeletion: true
    controller: true
    kind: Machine
    name: vultr-pool2-4ncj6-t26m5
    uid: 641f44f7-28d2-4faf-af5b-7837a59e0ceb
  resourceVersion: "490184690"
  uid: 500ccf3c-ab2a-4669-ac44-b996049e53b4
spec:
  apiKey: ""
  appId: "0"
  cloudInitUserData: ""
  common:
    cloudCredentialSecretName: cattle-global-data:cc-nkhlb
  ddosProtection: false
  enableVpc: false
  enabledIpv6: false
  firewallGroupId: ""
  floatingIpv4Id: ""
  imageId: ""
  ipxeChainUrl: ""
  isoId: ""
  osId: "1743"
  providerID: <rke2://vultr-pool2-4ncj6-t26m5>
  region: ewr
  sendActivationEmail: false
  snapshotId: ""
  sshKeyIds: []
  startupScriptId: ""
  tags: []
  vpcIds: []
  vpsBackups: false
  vpsPlan: voc-c-2c-4gb-75s-amd
status:
  addresses:
  - address: 144.202.3.168
    type: InternalIP
  - address: vultr-pool2-4ncj6-t26m5
    type: Hostname
  cloudCredentialSecretName: cattle-global-data:cc-nkhlb
  conditions:
  - message: ""
    reason: ""
    status: "True"
    type: CreateJob
  - message: ""
    reason: ""
    status: "True"
    type: Ready
  driverHash: 5b37756e1f55ddfff19b9d4fe3531003fe91f6d75c56c3bb6a9854bd1ea36b5f
  driverUrl: <https://rancher.vultr.dev/assets/docker-machine-driver-vultr>
  jobName: vultr-pool2-4ncj6-t26m5-machine-provision
  ready: true
c
is providerID getting set on the node in the downstream cluster?
that is the responsibility of the cloud controller manager on the downstream cluster
it looks like you’re just using the built-in rke2 cloud provider though, since I see 
<rke2://vultr-pool2-4ncj6-t26m5>
does that match the actual cloud provider ID used by vultr? I suspect not.
g
I was deploying the CCM after provisioning, which was my mistake. I wasn't aware that the default CCM would take over if not provided on creation. All good now! Thanks for your help Brandon!
Copy code
k get no -A
NAME                          STATUS   ROLES                  AGE     VERSION
test-controller-7cgmf-v494j   Ready    control-plane,master   3m51s   v1.31.9+rke2r1
test-etcd-72tnk-tklhn         Ready    etcd                   3m46s   v1.31.9+rke2r1
test-worker-dnrkw-fm894       Ready    worker                 75s     v1.31.9+rke2r1
test-worker-dnrkw-gmlg2       Ready    worker                 75s     v1.31.9+rke2r1
19 Views
Open in Slack
PreviousNext
Powered by