This is one of the reasons I have a strong aversion to "cloud" technologies like docker and kubernetes. You take networking, something with decades of development and hardware support, and you put it all in the CPU.
To be clear, Linux has a very robust networking stack. But it will never come close to the natting and routing performance of an actual router.
And so we develop things like DPDK to spend even more CPU just to keep things usable, but it still feels like a big step backwards.
A typical k8s deployment runs in containers that are in VMs. So each packet you want to send to or from a container needs to touch a cpu and traverse a networking stack six times. That's dumb.
To be clear, Linux has a very robust networking stack. But it will never come close to the natting and routing performance of an actual router.
And so we develop things like DPDK to spend even more CPU just to keep things usable, but it still feels like a big step backwards.
A typical k8s deployment runs in containers that are in VMs. So each packet you want to send to or from a container needs to touch a cpu and traverse a networking stack six times. That's dumb.