> Everywhere I have worked that has DBs they claim are too large, I have been able to prove (typically by cloning one and remodeling it) that the reason they're suffering from performance issues is because they've made a series of decisions which were fighting against their RDBMS
Some are just physics. Say if you want SSD write saturation and multiple parallell write transactions, the traditional RDBMS model itself may not suffice. Its not just data size but concurrent write transactions, lock contention, etc. Additionally, think resource constrained devices like mobile phones. Or an aggressive cost reduction mentality where each cpu cycle shouldn't be unnecessarily wasted without feeling guilty. If you come from a systems engineering background like that, you wouldn't be much impressed with many traditional databases at all. I am generally not much interested in SQL databases anyway.
> They needed to create secondary indices as tables because they were storing everything in JSON blobs, which is difficult to directly index
Our database Dip stores raw binary as well as json as first class citizens. Can't speak for Docstore, but as far as I see, it is just an evolved form of its earlier self with different tradeoff according to their workload. Note the lack of mention of the removal of secondary indexes as separate data.
> I would counter that if you keep needing schema changes, you haven't done a good job modeling.
Disagree. Schema stabilizes - yes. But doesn't mean they don't go through cycles of change occasionally.
Some are just physics. Say if you want SSD write saturation and multiple parallell write transactions, the traditional RDBMS model itself may not suffice. Its not just data size but concurrent write transactions, lock contention, etc. Additionally, think resource constrained devices like mobile phones. Or an aggressive cost reduction mentality where each cpu cycle shouldn't be unnecessarily wasted without feeling guilty. If you come from a systems engineering background like that, you wouldn't be much impressed with many traditional databases at all. I am generally not much interested in SQL databases anyway.
> They needed to create secondary indices as tables because they were storing everything in JSON blobs, which is difficult to directly index
Our database Dip stores raw binary as well as json as first class citizens. Can't speak for Docstore, but as far as I see, it is just an evolved form of its earlier self with different tradeoff according to their workload. Note the lack of mention of the removal of secondary indexes as separate data.
> I would counter that if you keep needing schema changes, you haven't done a good job modeling.
Disagree. Schema stabilizes - yes. But doesn't mean they don't go through cycles of change occasionally.