This message was deleted.

I figured it out! For anyone running into a similar issue it was actually related to a bug in Harvester 1.4.0 (not sure if present in earlier versions or not). This bug report while not the same high level issue I was running into, had the solution. TLDR: run

sysctl net.bridge.bridge-nf-call-iptables=0

as root on all harvester nodes in the cluster. https://github.com/harvester/harvester/issues/7210