The Linux network stack has been rapidly evolving; nowadays we have XDP for a fast path through the kernel. But there's also a lot of moving parts that ensure high throughput on noisy networks, for different application workload patterns, to avoid bufferbloat, and to do this fairly across multiple processes/containers. I'd imagine one can develop a user-space stack that appeared better in a simple iperf microbenchmark in a lab, but would lose for real-world applications and networks (unless your real-world use case was very narrow).
I'd recommend understanding the stack before choosing to replace it (the old programmer's advice about understanding a gate before removing it). I summarized the TCP send path in my next book (sysperf2, with help from Linux network engineers); it involves:
- Socket send buffers
- Pacing
- TSO
- Congestion controls
- Nagle
- TSQ
- qdisc (optional)
- GSQ
- BQL
- TSO (NIC)
That's just the TCP send path.
I'm happy to expand all these acronyms, but if any are new to you, that's kind-of my point. These parts have a purpose, and starting from scratch may lead you to eventually re-invent them all as you encounter the problems they solve.
I'd recommend understanding the stack before choosing to replace it (the old programmer's advice about understanding a gate before removing it). I summarized the TCP send path in my next book (sysperf2, with help from Linux network engineers); it involves:
That's just the TCP send path.I'm happy to expand all these acronyms, but if any are new to you, that's kind-of my point. These parts have a purpose, and starting from scratch may lead you to eventually re-invent them all as you encounter the problems they solve.