I still don't understand why CockroachDB doesn't just use Spark as the compute layer. Spark seems perfectly capable as a platform for performing arbitrary distributed computations on distributed data sources. It even has a full-fledged distributed SQL engine!
As it stands, it seems to me that CockroachDB is mostly just reinventing Spark from scratch, except maybe from a more OLTP-centric perspective.
As it stands, it seems to me that CockroachDB is mostly just reinventing Spark from scratch, except maybe from a more OLTP-centric perspective.