mbrt

Various attempts at AI personality shaping, having a very specific character in mind.

Wow, not sure how I missed this, but I see many similarities. They were also bitten by lack of conditional writes in S3:

> In Databricks service deployments, we use a separate lightweight coordination service to ensure that only one client can add a record with each log ID.

The key difference is that Delta Lake implements MVCC and relies on total ordering of transaction IDs. Something I didn't want to do to avoid forced synchronization points (multiple clients need to fight for IDs). This is certainly a trade-off, because in my case you are forced to read the latest version or retry (but then you get strict serializability), while in Delta Lake you can rely on snapshot isolation, which might give you slightly stale, but consistent data and minimize retries on reads.

It also seems that you can't get transactions across different tables? Another interesting tradeoff.

B2 is mostly S3-compatible, so if they add the same support for preconditions on writes as S3 and GCS, nothing prevents using it as a backend for GlassDB.

This builds on the same intuition I had, where data can be easily partitioned across objects. What seems to be missing is transactions across different objects though?

The flipside is that Cloudflare DO will be a lot faster.

Interesting that all these similar solutions are popping out now.

I think it would be interesting to combine a SQLite per-object approach with transactions on top of different objects.

You're right indeed:) but it depends on what you are comparing it with. In this case the comparison is against other managed cloud storage and databases, and in that context I think the claim holds.

Is it the cheapest possible storage in existence? No, if you take raw disks and put them in a rack, but I also feel it wouldn't be an entirely fair comparison.

It's a good observation, because I did and decided to keep it out of scope from the base layer.

But this is entirely possible. You can wrap GlassDB transactions and encode multiple keys into the same object at a higher level. Transactions across different objects will still preserve the same isolation.

The current version is meant to be a base from which to build higer level APIs, somewhat like FoundationDB.

Thanks for the details!

Consistency at the individual file can be guaranteed this way, but I don't think this works across multiple files (as you need a global total order of operations). In any case, this is a pragmatic solution, and I like the tradeoffs. Comparing against NFS rather than Spanner seems the right way to look at it.

Very nice, I like the approach. I assume data is partitioned and each file is handled by an elected leader? If data is replicated, you still need a consensus algorithm on updates.

How are concurrent updates to the same file handled? Either only one client can open in write at any one time, or you need fencing tokens.