Kudos on the project launch! I can share my experience from when I was evaluating multi-threaded architectures for Dragonfly back in 2022.
At the time, I had a choice: go down the KeyDB path and suffer from lock contention in real-world use cases, or go with a shared-nothing approach. The latter paid a somewhat heavy price for inter-thread communication, but when combined with a carefully chosen transactional framework could truly become a seamless Redis drop-in replacement. I chose the shared-nothing model.
"Dragonfly removes locks and pays the cost of message passing; KiviDB keeps locks but makes them so fine-grained that they rarely collide and never need to move a request between threads."
Reading this sentence, I immediately saw where it fits into the broader taxonomy of in-memory store architectures. KeyDB, Garnet, and KiviDB share the same foundation: employing fine-grained locks. The choice of programming language (Rust, C#, C++) is less important as it's the underlying architecture that dictates the trade-offs.
With lock-based approaches, features like MULTI/EXEC, Lua scripts, and multi-key operations introduce severe contention that becomes hard to manage. Furthermore, background tasks such as snapshotting, full replication sync, and memory defragmentation are more difficult to implement efficiently when relying on low-level locks.
On the other hand, I would definitely opt for a shared-everything architecture like KiviDB's if I were designing Memcached today - a minimalist key-value store without complex multi-key commands, replication, or snapshotting requirements.
In any case, best of luck with the project!
At the time, I had a choice: go down the KeyDB path and suffer from lock contention in real-world use cases, or go with a shared-nothing approach. The latter paid a somewhat heavy price for inter-thread communication, but when combined with a carefully chosen transactional framework could truly become a seamless Redis drop-in replacement. I chose the shared-nothing model.
"Dragonfly removes locks and pays the cost of message passing; KiviDB keeps locks but makes them so fine-grained that they rarely collide and never need to move a request between threads."
Reading this sentence, I immediately saw where it fits into the broader taxonomy of in-memory store architectures. KeyDB, Garnet, and KiviDB share the same foundation: employing fine-grained locks. The choice of programming language (Rust, C#, C++) is less important as it's the underlying architecture that dictates the trade-offs.
With lock-based approaches, features like MULTI/EXEC, Lua scripts, and multi-key operations introduce severe contention that becomes hard to manage. Furthermore, background tasks such as snapshotting, full replication sync, and memory defragmentation are more difficult to implement efficiently when relying on low-level locks.
On the other hand, I would definitely opt for a shared-everything architecture like KiviDB's if I were designing Memcached today - a minimalist key-value store without complex multi-key commands, replication, or snapshotting requirements.
In any case, best of luck with the project!