dneighman

This is what I've been working on for a little while and what helped me to condense my thoughts on it to this point. I've tried to separate out the _principles_ vs the product though which is why I haven't spoken about a product.

The schema-driven development isn't really about a particular format tbh. It's a set of principles that can be applied to any development process. Decide on the shape, then derive the rest. Protobufs fit into this category.

I really loved protobuf in the past. I've used them at Square and implemented gRPC at Opendoor. Protobufs have been really helpful with the runtime compiled codegen. They are heavy though, and there are some issues.

Getting a data schema to go all the way from the backend to the front end is challenging with Protos. You can use proto over json but the clients on the fe are not well looked after. You have to fight the tooling (or add copious amounts of it) to have the same contracts flow all the way through your code.

In my experience, protos work well inside a company for backend, but they're not really made for public use (agent to agent) or runtime use.

Even with the strong contracts in protobufs, which I've personally seen improve teams using them! There's still the questions... but, what does X or Y mean!?

One of the issues with writing meaning into your proto files in terms of field or message options is that only the authors can annotate it for the authors purposes. What we need is a way for authors to write it an annotate it, but _also_ for humans and agent consumers to add their own meaning to the schemas, fields and uses. Use-cases, gotachas, relationships, service annotations.

We've been re-building slightly different versions of the same things in each company / application, and every time we do, the reason for a field, the meaning is usually buried in email or slack threads, remembered only by "those who came before". In the age of agents where they need context, that type of tribal knowledge is holding us back.

Absolutely agree that many things need governance, human and agent. It seems to me that mostly, the things downstream of the data structures: storage, input, transport etc are derived from the data or can be applied as a layer on top. I'm not trying to boil the entire ocean for this but my thesis is that if you can agree on the shape of the data and what that means, you've got a great foundation for all work that follows.