> The tricky part is that when multiple users open the same Fig file concurrently, Figma’s infrastructure needs to ensure that they are all connected to the same server. That server can then be the sole authority on the state of that document, and write to it without race conditions.
This is the killer app for CloudFlare's new Durable Objects. They solve the routing and data distribution layer, allowing you to write single-threaded business logic to coordinate changes.
They even have a transactional storage capability which is priced more or less equivalent to DynamoDB (which Figma uses for their write-ahead log).
This pattern also helps scale write transactions without melting your database. http://ithare.com/scaling-stateful-objects/
I try to boost Durable Objects every chance I get, in order to "push the battleship" in the direction of another cloud provider implementing something equivalent.
While this article is written by the plane.dev team, which has an adjacent product, their approach seems more geared towards more demanding backends. Lots of use cases don't need to run e.g. a game or simulation on the backend, they just need a synchronization layer in front of a document store.
---
> They could stuff the whole document into a binary blob or json(b) column next to the metadata.
In my experience doing this in MySQL... do not do this. Once you have a dozens-if-not-hundreds-of-gigabytes large table full of JSON, it becomes a real nuisance. As a halfway measure, I think it would help to have a "blobs only" table separate to the metadata table. But as the OP points out, it is not economical anyway.
This is the killer app for CloudFlare's new Durable Objects. They solve the routing and data distribution layer, allowing you to write single-threaded business logic to coordinate changes.
They even have a transactional storage capability which is priced more or less equivalent to DynamoDB (which Figma uses for their write-ahead log).
This pattern also helps scale write transactions without melting your database. http://ithare.com/scaling-stateful-objects/
I try to boost Durable Objects every chance I get, in order to "push the battleship" in the direction of another cloud provider implementing something equivalent.
While this article is written by the plane.dev team, which has an adjacent product, their approach seems more geared towards more demanding backends. Lots of use cases don't need to run e.g. a game or simulation on the backend, they just need a synchronization layer in front of a document store.
---
> They could stuff the whole document into a binary blob or json(b) column next to the metadata.
In my experience doing this in MySQL... do not do this. Once you have a dozens-if-not-hundreds-of-gigabytes large table full of JSON, it becomes a real nuisance. As a halfway measure, I think it would help to have a "blobs only" table separate to the metadata table. But as the OP points out, it is not economical anyway.