I don't have a super beefy, new, hot machine, just a 5yr old Dell Precision laptop with 32GB of RAM and a 6core/12thread Core i7 with Ubuntu 20.04LTS. I have run an iperf3 client process/container -> iperf3 server process/container under 5 scenarios: 1. Bare metal via shell processes - 47.8Gbits/sec 2. Bare metal via docker compose - 42.2Gbits/sec 3. VM (8cpu/8GB) as Bare metal via shell processes - 56.4Gbits/sec 4. VM as Bare metal via docker compose - 50.3Gbits/sec 5. VM as Bare metal via k3s using a service - 35.2Gbits/sec The VMs are VirtualBox/Vagrant

ubuntu/jammy64

with minimal additional packages installed over the base, versus my laptop which is running a full desktop environment. The k3s install is just the simple installation from the docs, but with setting svc and cluster CIDR ranges to something other than 10. subnets. The results are repeatable and a bit curious. I assume the VM performs better partly because of newer kernel and less competition for resources, given it's minimalist configuration. Still the difference between Docker and k3s within the VM is rather large.

I'd be curious if others have investigated anything like this. I am aware of some scholarly studies on the performance of lightweight kubernetes distributions: • https://doi.org/10.1145/3578244.3583737 • https://doi.org/10.1007/s11227-022-04430-6 I don't have access to read the details of the 2nd one, but the 1st one cites it as using throughput metrics as part of their benchmark. The 1st one does not directly look at network layer throughput, but does provide some measurement of "data plane" performance using memtier to drive redis requests (if I'm following correctly). This is not a raw network metric, but latency and ops/sec through put are measured, but the authors find that k3s is the worst of the Lk8sd's they study.