This message was deleted.

Hopefully that makes sense too

did you add the harvester cloud provider?

Thanks for the response, im running the Harvester Cloud provider

So Rancher Vcluster deploying vms?

Yep thats right

Really. Try 1.30?

Yeah i figured everything they showed on their ui was the stable version

Nope.

Good to know thanks. Im deploying another cluster with version 1.30.6.

Check dzver.rfed.io. I wrote an app they scraps GitHub.

Looks like cluster v1.30 loadbalancers still use the master node for loadbalancers

you can actually skip the LBs and VIPs with better control of DNS and nginx ingress.

How? Got a video? Im still newish to kubernetes. There seem to be a lot of way to do the same thing lol. Still trying to figure out all the options

I will make one for you tomorrow. 😄

Cool lol thanks are you a harvester/rancher terraform user? Ive got everything in terraform currently

I am a harvester / bash kind of guy. 😄

Nice!

Hope this helps

@creamy-crayon-86622 the vip is used by other worker nodes to join the cluster, hence, it needs to be on the mgmt node

Oh you want to use it for rke2 control plane. The same idea works.

iiuc, you are trying to add a wildcard ingress in front of the

kube-system/ingress-expose

LB

service

, to route (application?) traffic through the rke2-ingress-controller, to the worker nodes

That is how I run my clusters. To the pods.

so your vip is exposed to north-south?

Yup.

interesting

Honestly, there really isn’t.

Thanks @thousands-advantage-10804 for the video, im still watching it. Thanks for the shoutout lol. Also my rke2-ingress pods are only running on the worker nodes. Not masters. Thats default out of the box at least with the rancher rke2-ingress

No worries. Let’s think big picture. What are trying to solve? What is the current setup doing or not doing?

My primary goal at the moment is an automated cluster deployment that gives me a single static IP address that I can add a wildcard record to. I figured I'd use native services such as the harvester loadbalancer behind the native RKE2 ingress controller. I've done this at work with openstack/rancher but harvester is a bit different it seems

Ok. would it make sense to get on a huddle to show me what you got? Are you seeing the loadbalancer being created?

Id be down for a huddle whenever your free. I do see the loadbalancer is getting created. It does work as long as my ingress controller service and kube VIP are on the same nodes

Ah you might not need the kubevip then. In theory the LB will get attached to the nginx serivce.

Okay so I just added a few tolerations to my rke2-ingress helm chart to deploy ingress controller pods to my control plane nodes and everything is working as expected

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-ingress-nginx
  namespace: kube-system
spec:
  valuesContent: |-
    controller:
      # <https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#add-headers>
      config:
        use-forwarded-headers: "true"
        enable-real-ip: "true"
        proxy-buffer-size: "16k"
      publishService:
        enabled: true
      service:
        enabled: true
        type: LoadBalancer
        external:
          enabled: true
        externalTrafficPolicy: Local
      kind: DaemonSet # Deploy the ingress controller as a DaemonSet
      tolerations: # Allow the ingress controller to run on control plane nodes
        - key: "node-role.kubernetes.io/control-plane"
          operator: "Exists"
          effect: "NoSchedule"
        - key: "node-role.kubernetes.io/etcd"
          operator: "Exists"
          effect: "NoExecute"
      nodeSelector: {} # Allow the ingress controller to schedule on any node

This should give me flexibility if I ever need to deploy a standalone loadbalancer without ingress. Tho I prolly wont need that

lol. glad to be able to point you in “A” direction.

Yeah to many directions available lol. This is how i have things setup at work (kinda) as well so its a bit better for me

so the key is

externalTrafficPolicy: Local

, do you want to preserve the client ip address for incoming requests? If that’s not your intention, you can change it to

Cluster

, and you don’t have to worry about where the kube-vip pods are running.

Thanks. Let me know if there are any video ideas you have.

@thousands-advantage-10804 I was actually hemming-hawing about bothering you via email on a best practice suggestion from your brain vault. But I read all this, and why not strike here (partially off topic) 1. What do I do if intend to avoid all in-cluster load balancing? We terminate all certs/tls on a HA pair of F5 BIG-IPs. Everytime I start to build a solid production cluster, I end up with a messy cert manager, self signed thing. (My lack of understanding ) 2. we’re working with f5 this week to wrap our heads around CIS, but with all the CNI options, I’m not sure which I should be pursuing? 3. Eternal debate…is STIG’d OS image is the goal, which do you suggest Ubuntu or Rocky9

Hey Bryan. We should start a new thread. i’ll tag you.

Heck yeah, you got it!