What is the recommended setup to run my worker nodes with a second network bridge connected via a dedicated firewall to the internet? The "usual kubernetes traffic" should stay on the internal network all nodes are connected to. The control plane nodes are not supposed to be connected to the external network. I read through the Multus guides on github. If I got this correctly I have to create a NetworkAttachmentDefinition on each worker node with type "macvlan" and mode "bridge" and a set of undocumented (as it seems) parameters, install lets say another Ingress Controller and tell it to use the other network using an annotation. Is this correct?

I recently try the cilium CNI and it offers the egress policy that allows to route a traffic to a specific IP on a node so I think that the traffic will going through the Right interface.