crux
Joined 18 years ago
Joined 18 years ago
For some time now it’s felt clear (or at least extremely) compelling that agents need fast compile times in order to be effective, especially when you’re working in parallel. But the other thing that has felt just as obvious is that agents need strong type systems and narrow guardrails in order to constrain their outputs. These two things felt clear enough to me that, like the author, I wanted to choose a language ecosystem that maximized them. There _are_ languages that both have expressive type systems _and_ fast compile times. I wonder if the author investigated any of them, before deciding that no compilation time at all was acceptable.
In my case I landed in OCaml. I think there are other options in the space—Go if you want less typing but faster compiles; Rust if you want more types but slower compiles. My mostly vibes-based evaluation landed on OCaml, and I’ve been pretty happy with the results.
Happy to discuss further if anyone has any thoughts.
Notion does not sell its users' data.
Instead, I want to expand on one of the first use-cases for the Notion data lake, which was by my team. This is an elaboration of the description in TFA under the heading "Use case support".
As is described there, Notion's block permissions are highly normalized at the source of truth. This is usually quite efficient and generally brings along all the benefits of normalization in application databases. However, we need to _denormalize_ all the permissions that relate to a specific document when we index it into our search index.
When we transactionally reindex a document "online", this is no problem. However, when we need to reindex an entire search cluster from scratch, loading every ancestor of each page in order to collect all of its permissions is far too expensive.
Thus, one of the primary needs that my team had from the new data lake is "tree traversal and permission data construction for each block". We rewrote our "offline" reindexer to read from the data lake instead of reading from RDS instances serving database snapshots. This allowed us to dramatically reduce the impact of iterating through every page when spinning up a new cluster (not to mention save a boatload in spinning up those ad-hoc RDS instances).
I hope this miniature deep dive gives a little bit more color on the uses of this data store—as it is emphatically _not_ to sell our users' data!
[email protected]
[ my public key: https://keybase.io/zdsmith; my proof: https://keybase.io/zdsmith/sigs/nn3-X3J7_ifPWLjxXAIjIUqBjcVfDja9v4vLKxvf8-M ]