vvern

I used to be a hater, but it’s kinda good now

Do SurrealDB next

How are folks doing CI on tangled?

Now I want to see benchmarks

Notably missing both Go and Rust

Why does clean room even matter given SQLite is in the public domain?

Still, I think it’s the right default to start with serializable. Then when you have performance issues you can think long and hard about whether relaxed isolation levels will work in a big free way. Better to start with a correct application.

About time for the 6th Edition, eh? What would folks include in it?

- Vector databases and hybrid search?

- Object storage for all the things? Lake houses. Parquet and beyond.

- Continuously materialized views? I'm not sure this one has made the splash but I think about Naiad (Materialize) and Noria (Readyset)

- NewSQL went mostly mainstream (Spanner wasn't included in the last one, but there's been more here with things like CockroachDB, TiDB, etc)

It should be this way. Clients should have some protocol to communicate the schema they expect to the database probably with some versioning scheme. The database should be able to serve multiple mutually compatible views over the schema (stay robust to column renames for example). The database should manage and prevent the destruction of in use views of that schema. After an old view has been made incompatible, old clients needing that view should be locked out.

You’re right that it would run on a block chain, but that fact would primarily exist to power some marketing. Everybody would end up interacting with it through a single centralized web site and API because it’s the only usable way to get it to work.

Thank you for writing this. This comes up constantly, and it'll be great to have another reference to cite.

Another interesting thing about TPC-C is how the cross-warehouse contention was designed. About 10% of new order transactions need to do a cross-warehouse transaction. If you can keep up with the workload, then the rate of contention is relatively low; most of the workload isn't pushing on concurrency control. If, however, you fall behind, and transactions start to take too long, then the contention will pile up.

When you run without the keying time, it turns out that concurrency control begins to dominate. For distributed databases, concurrency control and deadlock detection is fundamentally more expensive than it can be for single-node databases -- so it makes sense that a classically single-node database would absolutely trounce distributed databases. I like to think of tpcc "nowait" as really a benchmark of concurrency control because, due (I believe) to Amdahl's law the majority of its execution time ends up just in the contended portion of the workload.

Also very interesting that, as Justin points out, the workload sets up the warehouses so there is never cross-node contention. That's wild! I'm glad they didn't go and benchmark against even more distributed databases (like YugabyteDB, Spanner, or CockroachDB) and call it a fair fight.

Folks, for the love of god, please please stop running TPC-C without the “keying time” and calling it “the industry-standard TPCC benchmark”.

I understand there are practical reasons why you might want to just choose a concurrency and let it rip at a fixed warehouse size and say, “I ran TPC-C”, but you didn’t!

TPC-C when run properly is effectively an open-loop benchmark that scales where the load scales with the dataset size by having a fixed number of workers per warehouse (2?) that each issue transactions at some rate. It’s designed to have a low level of builtin contention that occurs based on the frequency of cross warehouse transactions, I don’t remember the exact rate but I think it’s something like 10%.

The benchmark has an interesting property that if the system can keep up with the transaction load by processing transactions quickly, it remains a low contention workload but if it falls behind and transactions start to pile up, then the number of contending transactions in flight will increase. This leads to non-linear degradation mode even beyond what normally happens with an open loop benchmark — you hit some limit and the performance falls off a cliff because now you have to do even more work than just catching up on the query backlog.

When you run without think time, you make the benchmark closed loop. Also, because you’re varying the number of workers without changing the dataset size (because you have to vary something to make your pretty charts), you’re changing the rate at which any given transaction is going to be on the same warehouse. So, you’ve got more contending transactions generally, but worse than that, because of Amdahl’s law, the uncontended transactions will fly through, so most of the time for most workers will be spend sitting waiting on contended keys.