anujkaliaitd
Joined 8 years ago
Karma
16
Submissions
6
Member since
Apr 26
Recent submissions
Comment · 2 years ago
Comment · 4 years ago
(Author of eRPC.) I'm glad you've found the project useful, and my apologies for the code and build quality, and NIC compatibility issues that you faced. I'm very interested in learning from your experience (perhaps even as a GitHub issue on our repo), especially w.r.t. how the code organization should be improved, although I understand the list might be too long :).
I know of some issues myself that I plan to fix, e.g., using macros to decide which Transport class to use -- this should just use virtual methods; messy inter-mixing of congestion control and packet I/O; some "performance optimizations" like preallocated buffers that are just not worth it.
Comment · 7 years ago
Thank you for the questions.
1. Our throughput benchmark is designed to measure data transfer bandwidth, and the comparison point is RDMA writes. In the benchmark, eRPC at the server internally re-assembles request UDP frames into a buffer that is handed to the server in the request handler. The request handler does not re-touch this buffer, similarly to RDMA writes.
We haven't compared against RPC libraries that use fast userspace TCP. Userspace TCP is known to be a fair bit slower than RDMA, whereas eRPC aims for performance RDMA-like performance.
2. eRPC uses congestion control protocols (Timely or DCQCN) that have been deployed at large scale. The assumption is that other applications are also using some congestion control to keep switch queueing low, but we haven't tested co-existence with TCP yet.
3. 75 Gbps is achieved with one core, so there's no need to distribute load. We could insert this data into an in-memory key-value store, or persist it to NVM, and still get several tens of Gbps. The performance depends on computation-communication ratio, and we have tons of communication-intensive application
4. Packet loss in real datacenters is rare, and we can make it rarer with BDP flow control. Congestion control kicks in during packet loss, so we don't flood the network. Our packet loss experiment uses one connection, which is the worst case. An eRPC endpoint likely participates in many connections, most of which are uncongested.
Comment · 7 years ago
eRPC author chiming-in again.
I agree that our code is not production-ready, but an industry team at Intel is trying eRPC out: https://github.com/daq-db.
I am curious to know what features you believe we omitted in our ISCA 15 paper. Our key-value store (https://github.com/efficient/mica2) supports a memcached-like API over UDP.
Comment · 7 years ago
I'm the main author of eRPC, and I wanted to clarify some things.
Your comment suggests (please correct me if you meant differently) that (a) eRPC does not perform congestion control, and (b) eRPC requires a lossless fabric. In fact, eRPC implements congestion control, and it works well in a lossy network. Those are the two main contributions of the paper.
We get 75 Gbps with only UDP/Ethernet packet I/O, without RDMA support.
eRPC implements transport-layer functionality atop a fast packet I/O engine like DPDK, so comparing eRPC to DPDK isn't apples-to-apples.
While there are several other DPDK-based network stacks, Machnet provides the following unique benefits: (1) Specifically designed for and tested on public cloud VM environments. (2) Multiple applications on the same machine can use Machnet. (3) No need for DPDK expertise, or compiling the application with DPDK.
Architecture: Machnet runs as a separate process on all machines where the application is deployed and mediates access to the DPDK NIC. Applications interact with Machnet over shared memory with a sockets-like API. Machnet processes in the cluster communicate with each other using DPDK.
(In the past I'd built eRPC, discussed on HN here https://news.ycombinator.com/item?id=19418997. Machnet has an improved design that fixes several issues that we found made eRPC difficult to use in real applications.)