This message was deleted.

we just do single-node harvester on hetzner/etc. seems like an unorthodox setup but sidestepped having the deal with netops complexity

That would be fantastic - any suggestions on how to go about achieving that? I see the documentation here: https://docs.harvesterhci.io/v1.3/advanced/singlenodeclusters But I don't see how that avoids me having to deal with the whole virtual IP stuff and all that. I'm obviously missing something simple here. 😞

Yeah depends what you're trying to do. We're not running untrusted code. But basically we set the management interface to the external IP, vip to random internal IP, VMs are on management network with virtio masquerade. You can't externally reach VMs directly but we setup services/ingresses in Harvester kubernetes necessary to route from the outside + calico/hetzner firewalls.

Nice! How did you avoid the whole problem with Harvester throwing out requests on the main interface from different MAC addresses? That's an instant warning from Hetzner immediately upon detection and folks have had service suspended / terminated for that.

ehmm, not an expert but I think it's just the virtio masquerade basically works as-is. I set this up a while back and had to twiddle / look at this issue. but e.g. if I do

tcpdump -i [nic] -en | grep -v [mac]

it's empty, all external traffic going to/from the NIC MAC. I also setup locally to test before deploying to dedi

I set the VIP to 10.0.0.1 and I do have traffic egress over the wrong MAC address. What address did you use?

our config basically looks like:

vip: 10.180.0.111
  vip_mode: static
  management_interface:
    interfaces:
    - name: {{ metal_box.net_device }}
    default_route: true
    method: static
    ip: {{ metal_box.ip }}
    subnet_mask: {{ metal_box.netmask }}
    gateway: {{ metal_box.gateway }}
    mtu: 9000
    bond_options:
      mode: balance-tlb
      miimon: 100