https://rancher.com/ logo
Title
p

powerful-elephant-25838

05/09/2023, 10:39 AM
RKE2 with cilium on Ubuntu 22.04 on VMWare is not working, node reachable but endpoints NO. The internal communication between nodes not working. Any support on this?
k

kind-air-74358

05/10/2023, 3:12 PM
Are you actually using NSX-t with VMWare?
p

powerful-elephant-25838

05/11/2023, 5:41 PM
No we are using nsx-v
k

kind-air-74358

05/11/2023, 6:52 PM
We had some similar issues on our side using nsx-t. We still don’t have a clear understanding of the issue, but it looks that this command will somehow fix it:
ethtool -K cilium_vxlan tx-checksum-ip-generic off
. One downside is that this command is not persistend after a reboot.
(So not a very good fix unfortunatly)
And we verified it also against an RKE2 with Cilium on Ubuntu 22.04 on vSphere 8 without nsx-t which is working fine as far as I can tell now
p

powerful-elephant-25838

05/11/2023, 6:56 PM
k

kind-air-74358

05/11/2023, 6:58 PM
I’ll give this a look tomorrow. Thanks for the link
👍 1
p

powerful-elephant-25838

05/11/2023, 6:59 PM
I will also try it in the next days, we have other activities scheduled and for now we reverted to 20.04 that works smoothly
👍 1
a

acoustic-addition-45641

05/15/2023, 6:11 PM
Ran into this same issue today. Ran the "ethtool -K cilium_vxlan tx-checksum-ip-generic off" command, and the Cilium connectivity test passes. This was after making the adjustments to account for foreign routes in Ubuntu 22.04 (before installing RKE2+Cilium). By chance, did you happen to figure out how to make the ethtool changes permanent?
Added the command as a crontab that runs every 5 minutes. This is "good enough" for now. Hopefully VMWare will resolve this at some point.
k

kind-air-74358

05/16/2023, 6:59 AM
No unfortunately we didn’t found a solution yet. One possible solution is mention is this thread: https://github.com/projectcalico/calico/issues/4727. But not tested the change of vxlan port yet
👍 1
p

powerful-elephant-25838

05/24/2023, 12:23 PM
tested it and seems that setting "tunnelPort: 4789" make it work; we have VMWare with nsx
🙌 1
👍 1