iyzhang

I consider POSIX to be high-level. The RDMA and DPDK interface are not.

The Demikernel is not a unikernel. It is a library OS compiled as a series of shared libraries. It is not compiled together with the application and doesn’t take into account what features the application uses. It is designed to work with kernel-bypass hardware, like DPDK.

It looks POSIX-like but uses high-level queues and fixes some issues with epoll. The lack of an atomic data unit and the overhead of the poor epoll interface cost too much to retain for kernel-bypass. Take a look at the paper for more details.

BTW, the Demikernel will be open-sourced shortly .. as soon as I return from giving a talk at KubeCon Europe.

It depends on the interface for the drivers to the application. However, UIO doesn't seem to support DMA, which is a non-starter.

RDMA and DPDK both use user-space drivers, which is necessary for kernel-bypass. I'm not advocating for a particular kernel-bypass solution. I'm arguing that if we use kernel-bypass for I/O, we should have a common, efficient, high-level interface for it.

High-level means not exposing hardware limitations to the application. The primary target applications are datacenter services, which spend much of their time processing network I/O. As network latencies lower to a few microseconds, datacenter applications like Redis will need kernel-bypass because the kernel will become too expensive for them. In our experiments with a 25Gb network, the Linux kernel and POSIX interface costs Redis 60% of its latency.

This implementation is an optimization of leader-based protocols like Raft. It removes the leader from the critical path using SDN.

No, that was our key insight. Basically, you can think of it as, using something like Paxos requires all replicas to run the same code. So every replica checks for transaction conflicts. However, you actually only need one replica to detect a conflict, so there is no reason for every replica to run the same checks -- this is wasted work. Instead, we just make sure that each conflict is checked by at least one replica.

The other interesting conclusion is that there are other workloads like this. For example, it is possible to build a reliable and better performing lock server in this way as well (and there is an implementation in the github repo). So, you'd get something similar to Chubby, but where the latency to the server is only a single round-trip in the common case.

Figure 14 in the paper gives comparisons with MongoDB, Redis and Cassandra using YCSB. The git repo has the YCSB bindings for TAPIR, so you can run TAPIR against any of the other systems that also have YCSB bindings.