Hello! Has anyone else experienced issues deployin...
# harvester
s
Hello! Has anyone else experienced issues deploying the Rancher vCluster Add-on with Harvester 1.6.0? We have tested deploying this every version of Harvester since 1.3.x without issue. Now, when simply running the Add-on, it's failing with the following error. These are internal pod CIDR ranges not the external IP CIDR ranges of our environment. Any ideas? Thanks!
Copy code
2025-09-10 20:50:12	INFO	vcluster/main.go:44	start controllers: sync kubernetes service: sync kubernetes service: Service "kubernetes" is invalid: spec.clusterIPs: Invalid value: []string{"10.53.27.235"}: failed to allocate IP 10.53.27.235: the provided IP (10.53.27.235) is not in the valid range. The range of valid IPs is 10.96.0.0/12	{"component": "vcluster"}
t
Did you change the default CIDR?
s
No. That's what is so confusing. We ended up doing a clean install of Harvester 1.6.0 in lab. Then deployed the Rancher vCluster Add-on with same default params.
t
and same error?
s
That is where we first saw the error.
Our previous cluster failed to upgrade from 1.5.1 -> 1.6.0 so we rebuilt. Harvester is running fine and we are able to deploy VMs, but the Rancher Add-on fails. No idea what would be different about this other than Harvester versions.
t
let me take a look at it.
s
Hey thanks! We are also going to run a clean Harvester install on a single device and try to compare deltas. We'll let you know what we find as well.
t
perfect.
s
I uninstalled the Helm chart and reinstalled on broken cluster and noticed an earlier error. It seems it's failing to detect the service CIDR range for some reason.
Copy code
2025-09-11 14:50:29	INFO	setup/initialize.go:87	failed to detect service CIDR, will fallback to 10.96.0.0/12, however this is probably wrong, please make sure the host cluster service cidr and virtual cluster service cidr match. Error details: failed to find IPv4 service CIDR: couldn't find host cluster Service CIDR ("Service "test-service-k8bhj" is invalid: spec.clusterIPs: Invalid value: []string{"4.4.4.4"}: failed to allocate IP 4.4.4.4: the provided network does not match the current range") ; or IPv6 service CIDR: couldn't find host cluster Service CIDR ("Service "test-service-j6frg" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")	{"component": "vcluster"}
2025-09-11 14:50:29	INFO	k3s/k3s.go:52	Starting k3s	{"component": "vcluster", "args": "/binaries/k3s server --write-kubeconfig=/data/k3s-config/kube-config.yaml --data-dir=/data --disable=traefik,servicelb,metrics-server,local-storage,coredns --disable-network-policy --disable-agent --disable-cloud-controller --egress-selector-mode=disabled --flannel-backend=none --kube-apiserver-arg=bind-address=127.0.0.1 --disable-scheduler --kube-controller-manager-arg=controllers=*,-nodeipam,-nodelifecycle,-persistentvolume-binder,-attachdetach,-persistentvolume-expander,-cloud-node-lifecycle,-ttl --kube-apiserver-arg=endpoint-reconciler-type=none --service-cidr 10.96.0.0/12 --token redacted"}
So, we straight up installed 1.6.0 from USB on one of our beefy HP Z2 machines and enabled the Rancher Add-on and we are seeing the exact same errors we are seeing on our lab Harvester cluster. I believe this is an issue with 1.6.0 and the Rancher Add-on.
Curious if you're seeing the same.
We're going to flash a USB drive with 1.5.1 and repeat on same device. If that works out of the box, I'm going to say there is an issue with 1.6.0 and Rancher vCluster Add-on support.
t
I am able to recreate this issue. digging a little deeper.
1
s
Thanks! The sanity check is definitely appreciated. I'm inclined to open a GitHub issue. I could not find any open issues on this. We think the version of the vCluster Helm chart (v0.19) is older and potentially breaks with the change from RKE2 1.32 -> 1.33 that comes with Harvester 1.6.0.
t
open the issue. I think it is the subnet that is given to the vlcuster from helm. I am looking at the source now.
1
s
The code attempted to create a test Service and when it FAILED - it parsed the error message.
🤦‍♂️
error message in 1.32
Copy code
The Service "test-service" is invalid: spec.clusterIPs: Invalid value: []string{"4.4.4.4"}: failed to allocate IP 4.4.4.4: the provided IP (4.4.4.4) is not in the valid range. The range of valid IPs is 10.53.0.0/16
error message in 1.33
Copy code
The Service "test-service" is invalid: spec.clusterIPs: Invalid value: []string{"4.4.4.4"}: failed to allocate IP 4.4.4.4: the provided network does not match the current range
So, it seems that the change from RKE2 1.32 -> 1.33 did change the error message returned when the test service could not be created. And this was REQUIRED for vCluster to set the correct CIDR...
t
I have it running in 1.32.8
s
Yep - it will work on 1.32
t
I am testing in 1.33..4 now
s
The Rancher vCluster Add-on ("experimental" though it is) could not have been tested prior to rolling out Harvester 1.6.0
t
I was able to install upstream vcluster to 1.33.4 on rke2 with no issues.
w
what version of the vcluster helm chart are you using?
t
helm upgrade --install my-vcluster vcluster --repo <https://charts.loft.sh> --namespace team-x --create-namespace --repository-config=''
s
The Rancher vCluster Add-on uses v0.19
Yeah. That's the issue. And the Helm chart has been changed a good bit from v0.19 to latest.
Thanks for testing. This confirms what we have been seeing.
t
I have a hack we can try….
curl -sL <https://raw.githubusercontent.com/harvester/experimental-addons/main/rancher-vcluster/rancher-vcluster.yaml> | sed 's/"v0.19.0"/"v0.28.0"/g' | kubectl apply -f -
testing it now.
s
Found a GitHub issue for this opened 3 days ago in the experimental-addons repo. Updating with our info now.
Actually there is an MR FROM JULY!
t
guess we wait.. lol
s
This gives us enough to build a workaround for deployment now. Thanks for all the help/testing today!
🎉 1
a
I guess we need to update the chart values, seeing as v0.20.0 and newer give errors:
Copy code
vcluster:
- (root): Additional property hostname is not allowed
- (root): Additional property init is not allowed
- (root): Additional property syncer is not allowed
- (root): Additional property vcluster is not allowed
- (root): Additional property rancherVersion is not allowed
- (root): Additional property bootstrapPassword is not allowed
- sync: Additional property ingresses is not allowed
Did you find the correct way to modify the values yet?
Even using the values from the pull request you linked, can't make the addon install because it re-adds the faulty values 😞
t
Can confirm the errors.
a
Ended up removing the addon entirely and recreating it with the PR's manifest. Means I have to recreate downstream clusters though...
s
Hey! Since I came across this thread when troubleshooting, I figured I'd reply here with a temporary fix whilst waiting on that PR to merge. All I had to do to bypass the mandated fields was instantiate the add-on as my own "new" add-on. I'm not 100% on how it all works behind-the-scenes, but it seems like those fields are hard-coded in Harvester to be presented when that add-on is both added and enabled. Add-on documentation didn't really tell me where those values were ever coming from, so again I just assume it's tied to the vcluster name currently.
Copy code
apiVersion: v1
kind: Namespace
metadata:
  name: temp-vlcuster-fix
---
apiVersion: <http://harvesterhci.io/v1beta1|harvesterhci.io/v1beta1>
kind: Addon
metadata:
  name: temp-vlcuster-fix
  namespace: temp-vlcuster-fix
  labels:
    <http://addon.harvesterhci.io/experimental|addon.harvesterhci.io/experimental>: "true"
spec:
  enabled: false
  repo: <https://charts.loft.sh>
  version: v0.28.0
  chart: vcluster
  valuesContent: |-
    serviceCIDR: 10.53.0.0/16
    controlPlane:
      distro:
        k3s:
          resources:
            limits:
              memory: 16096Mi
              cpu: 8000m
          enabled: true
          imagePullPolicy: IfNotPresent
          image:
            tag: v1.33.4-k3s1
            repository: rancher/k3s
    sync:
      toHost:
        ingresses:
          enabled: true
    experimental:
      deploy:
        vcluster:
          manifests: |-
            apiVersion: v1
            kind: Namespace
            metadata:
              name: cattle-system
            ---
            apiVersion: v1
            kind: Namespace
            metadata:
              name: cert-manager
              labels:
                <http://certmanager.k8s.io/disable-validation|certmanager.k8s.io/disable-validation>: "true"
          helm:
            - chart:
                name: cert-manager
                repo: <https://charts.jetstack.io>
                version: v1.8.0
              release:
                name: cert-manager
                namespace: cert-manager
              values: |-
                installCRDs: true

            - chart:
                name: rancher
                repo: <https://releases.rancher.com/server-charts/latest>
                version: v2.12.0
              release:
                name: rancher
                namespace: cattle-system
              values: |-
                hostname: rancher.<your-vip-goes-here>.home.arpa
                replicas: 1
                bootstrapPassword: password1234
                rancherImage: rancher/rancher
                ingress:
                  tls:
                    source: rancher
                global:
                  cattle:
                    psp:
                      enabled: "false"
                extraEnv:
                  - name: CATTLE_AGENT_IMAGE
                    value: rancher/rancher-agent:v2.12.0
👍 2