Has anyone tried cluster api with --disable-agent?...
# k3s
v
Has anyone tried cluster api with --disable-agent? I think the Control Plane health checks would fail as it looks for ready status rather than /healthz
c
which cluster api? If whatever it is relies on there being a node for the control-plane, then yes, that would be broken
v
I used this before with kubeadm - https://cluster-api.sigs.k8s.io/ to vend out bare-metal clusters and while kubeadm was fine the control plane was visible to the customer and we were never truly "managed" I like the k3s --disable agent solution as it's mostly hidden from the customer sort of like the hyperscalers.
Copy code
kubectl get nodes
NAME           STATUS   ROLES    AGE    VERSION
test-node-01   Ready    <none>   4d8h   v1.33.4+k3s1
The end goal is to vend out baremetal clusters to customers so looking at solutions
c
k3s should provide a transparent control-plane if you use a single-server setup with --disable-agent, but I’ve never tested it with CAPI. We’re working on something similar for RKE2 as well.
v
I did the embedded etcd setup for 3 CP. I disabled most things because we have argocd that deploys coredns/cilium cni outside the cluster. The demo cluster seems to be functioning but haven't had time to send heavy load.
Copy code
write-kubeconfig-mode: "0644"
cluster-init: true
disable-cloud-controller: true
disable-kube-proxy: true
disable-helm-controller: true
disable-network-policy: true
disable-agent: true
etcd-disable-snapshots: true
flannel-backend: "none"
cluster-cidr: "192.168.0.0/16"
egress-selector-mode: "cluster"
tls-san:
  - "<redacted>"
disable:
  - "servicelb"
  - "coredns"
  - "traefik"
  - "local-storage"
  - "metrics-server"
  - "runtimes"
  - "ccm"
I was looking at this custom cloud provider today and seeing if Rancher UI could manage it. Is that what you're referring too? https://rke.docs.rancher.com/config-options/cloud-providers/custom
We use proxmox for control plane but the workers are bare metal with maas