thekozmo
Joined 11 years ago
Joined 11 years ago
The problem for startups is that debt provider who are also a bank require to move most of the business to that provider. It's not the case other types of debt providers. w.r.t debt - everybody use this mechanism, doesn't matter if you're small or big.
The JVM is part of the problem, not all of it. The main issue is that it hides the hardware and makes tracing harder - instruction level and block level. At Scylla we strive for efficiency, every operation is tagged with a priority class for the CPU and I/O schedulers. Folks are welcome to read the blogs about those topic. Lots of details and hard work
- Comcast moved from 970 Cassandra nodes (and 60 nodes of cache) to 78 Scylla nodes - https://www.scylladb.com/2020/01/15/comcast-sprinting-from-c...
- Palo Alto Networks run 8600 clusters(!) of ScyllaDB https://www.scylladb.com/2022/06/14/how-palo-alto-networks-r...
ScyllaDB excels in throughput and latency, we have also a better compaction algorithm that saves 37% of storage compared to C*. Usually one can replace lots of small nodes with gigantic nodes that have more resources and it allows much better management.
To run 100PB Scylla will need more than 300 nodes, even thousands but definitely not what Apple throw at the problem.
The whole intent is to make it a learning experience, if you wish to reproduce, try it yourself. It's aligned with past measurements of ours and also with former Linux optimizations by Marc.
Some K/V workloads are affected by the networking stack and we recently seen issues if we chose not the ideal interrupt mode (multiqueue vs single queue in small machines)