My side project has been to rewrite
https://github.com/erpc-io/eRPC which does RPCs over UDP with some congestion control supposedly quite fast. Paper:
https://www.usenix.org/system/files/nsdi19-kalia.pdf. I never got the code to work in AWS; I believe the author focused on Mellanox NICS but that's not really commodity H/W, which is where my interests lay.
So I dug into it .. and well I'll have my own library soon. I should be able to send UDP w/o congestion control sometime this week.
eRPC uses DPDK (100% user space NIC TX/RX control) plus the author's own other ideas to get performance. Since I'm getting into DPDK way, way late in the game, I hope and believe DPDK is and will be the better way to go than to turn to kernel fix-ups.
Getting the kernel out of the way with pinned threads seems -- or more exotic with special H/W if preferred --- is cleaner if one can develop from scratch. DPDK is all day long zero copy, and always has been.
This library will be a part of something bigger. For many frameworks, a key architecture point is:
I got a RPC/packet/message. Ok, now what?
* process it in-place e.g. on the thread that was doing RX?
* if I don't delegate am I making back-pressure?
* delegate it to another core? How to efficiently do that? Hint: https://blog.acolyer.org/2017/12/04/ffwd-delegation-is-much-...
* if I delegate ... to whom? Maybe I'm partitioning?
* If I delegate how do I get a response back?
In DPDK I believe these are easier to decide and well-manage in code.
So I dug into it .. and well I'll have my own library soon. I should be able to send UDP w/o congestion control sometime this week.
eRPC uses DPDK (100% user space NIC TX/RX control) plus the author's own other ideas to get performance. Since I'm getting into DPDK way, way late in the game, I hope and believe DPDK is and will be the better way to go than to turn to kernel fix-ups.
Getting the kernel out of the way with pinned threads seems -- or more exotic with special H/W if preferred --- is cleaner if one can develop from scratch. DPDK is all day long zero copy, and always has been.
This library will be a part of something bigger. For many frameworks, a key architecture point is:
I got a RPC/packet/message. Ok, now what?
* process it in-place e.g. on the thread that was doing RX?
* if I don't delegate am I making back-pressure?
* delegate it to another core? How to efficiently do that? Hint: https://blog.acolyer.org/2017/12/04/ffwd-delegation-is-much-...
* if I delegate ... to whom? Maybe I'm partitioning?
* If I delegate how do I get a response back?
In DPDK I believe these are easier to decide and well-manage in code.