Recent submissions

Comment · 6 months ago
I strongly hope that the authors are actually using O_DSYNC|O_DIRECT, and not just the O_DIRECT they discuss. The post seems to conflate the 4KB atomicity (support for which is still hit or miss from drives I’ve checked) with 4KB writes being assuredly durable. If there is a write cache on the SSD/NVMe, which is common, one must either issue a FLUSH via fsync or have FUA set on the write via O_SYNC/O_DSYNC. Relying only on O_DIRECT is not a correct solution, as it assumes a drive with sufficient capacitors to always write out all cached data on failure, and that’s not a part of the required drive contract.
Comment · 1 year ago
percona/sysbench-tpcc has been subsequently updated to include a stronger disclaimer that it's "TPC-C-like" and doesn't comply with multiple TPC-C requirements. Fingers crossed that this helps stop vendors from doing non-TPC-C benchmarking without realizing it.
Comment · 1 year ago
...but are they enforceable?
Comment · 1 year ago
(Hi! Post author here.)

It is written with a lean towards serializable, partly because there's a wide variety of easy examples to pull which all implement serializable, but the ideas mostly extend to non-serializable as well. Non-serializable but still MVCC will also place all of their writes as having happened at a single commit timestamp, they just don't try to serialize the reads there, and that's fine. When looking at non-serializable not MVCC databases, it's still useful to just try to answer how the system does each of the four parts in isolation. Maybe I should have been more direct that you're welcome to bend/break the mental model in whatever ways are helpful to understand some database.

The line specifically about MySQL running at serializable was because it was in the Spanner section, and Spanner is a (strictly) serializable database.

Comment · 2 years ago
I'll rephrase the line sometime, but the intention is to communicate that it's the default choice to go with if you're making this decision in 2024. Most of the projects which aren't using some form of direct IO are doing it because they predate O_DIRECT (e.g. postgres), didn't architect for using async IO (e.g. mongo), or have excessive portability concerns which means they only offer it as an option (e.g. innodb).
Comment · 3 years ago
A recent blog post about MongoDB's SBE detailed that the purpose behind their VM is that it serves as a way to re-use an execution component between two different query languages. SQLite's claim was just that the strong separation between compilation and execution makes issues easier to debug.

I wouldn't expect VMs to become the default design in databases, but it seems like it's getting increasingly common as an IR for query compilation. The ability to have a (comparatively simpler) interpreter for the VM also means you can apply simple fuzzing to great effect: if the results of interpretation vs compilation ever diverge, there's a bug.

Comment · 3 years ago
It’s interesting to me that in the original proposal for fixing fsync to actually be durable (https://lwn.net/Articles/270891/), there was thought given to the desire for a non-flushing write barrier (ctrl-f SYNC_FILE_RANGE_NO_FLUSH), but it appears that it never got implemented.
Comment · 4 years ago
There's one set of folk working on a btree, and another set of folk are now working on a RocksDB storage engine.

The original preference away from RocksDB was that it doesn't play well with deterministic simulation. Any code included into FDB needs to be able to be able to run with coroutines (strongly preferably stackless ones, though sqlite's btree has a stackful coroutine shimmed under it). RocksDB is definitely not written to support coroutines, and thus trying to use it anyway results in sacrificing developers' abilities to dig into failures.

Redwood has a couple design decisions that would make it a poor general purpose btree, but a great one for FoundationDB. But RocksDB will still have write and space amplification advantages.

Comment · 4 years ago
If you don’t like being told what to do, boy do I have bad news about PhDs for you...
Comment · 4 years ago
Comment · 4 years ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton