mrothroc
Joined 8 months ago
Joined 8 months ago
The gates have both deterministic and stochastic (LLM as a judge) components. The blend depends on the artifact: prose leans heavily on LLMs, where code has a lot of tests.
I work by having a discussion with claude code about what I'm trying to do and how it might be built. I ask it how its proposal fits into the existing bounded contexts and if it is reusing the existing patterns and libraries. Once it can give me an approach I like, I have it create a plan. This uses a skill and it has guidance on the structure and contents of the plan.
It is mostly prose, so the deterministic checks just look for the sections in the correct order. The LLM reviewer is a different model entirely to avoid bias and it is agentic, so it can (and does) explore the code and is prompted to verify the plan is correct both in the abstract and in the context of the code.
Once approved, it moves to breaking the plan into one or "releases" which are batches of tasks. Another gate, then to coding, which is where it fans out to multiple implementing agents. Another gate.
As I said, very standard SDLC. I get asked this question a lot, so I wrote it up with a guide on how to do it on the second to last page. https://michael.roth.rocks/research/543-hours
There are definitely companies that are being short-sighted on this. I've seen this come back and bite them (usually, not always).
I'd argue that if the company doesn't provide this, then people should rely on their community. I'm lucky enough to live in a place that has lots of meetups and we talk about this. Absent that, there are plenty of active discussions online, as this very thread shows.
The bonus of working via the community is that's where you can find a new opportunity if the company doesn't take care of their people.
At the same time, what I saw during that time was that organizations that actively sought to help juniors transition their skills to match the tools did better than those who just left everyone to wander around in the dark.
How do juniors develop it? Yeah, that's the tough part. Those years spent slogging through very tedious work got me here. You have to exercise to build muscle. Maybe we need to find some way to let people exercise the skills without the hands-on slog? I'm honestly not sure, how are you seeing it handled where you are?
You raise another point though: agents are not human. The things that drive agentic loops are not at all the things that motivate humans to do good work.
That's probably where the human-management skill ends in this new world. The techniques that get high quality from humans do not apply to agents.
The skill I learned is exactly as you describe: understand the system. For me, this means "boxes and lines", the components and how they interact. I've encouraged developers on my team to make sequence diagrams part of their deliverable, because it helps both of us put the code in the context of the system.
The boxes can be opaque because I know the boundaries and expected inputs and outputs. This tells me when I need to go in a box: is it sending a malformed message? I can trace that back into the code.
I think that's the skill we have to cultivate now that coding agents produce far more than we can review line-by-line. The system view is the map, and the boundaries tell us where to go hunting inside the box.
The reason he said that is because the best way to learn how to properly build a complex system is by going through the process of building it. Sure, implementation is a bottleneck, but the real problem is that the experience of development is the best teacher.
Personally, I try to bring this lesson into my agentic coding practice by doing spikes and trial implementations first. This is where the AIs really deliver value: they can quickly build something out that touches a lot of the architecture so both it and I can get the proper shape of it. Only after that do I unleash it, constrained by what we learned.
For me, this means the checks have to be more than just "I looked at it". There are two things that happen before I ever see it: first, as others have mentioned, I have a model from a different family review it with well-defined criteria. Same-family reviewers share bias, so it must be a different one. Second, I have a core set of deterministic gates (like lint, but also unit tests) that must run. In either case, failures go back to the coding agent.
And only then do I bother. But I really don't read everything. If it is bog-standard CRUD operations, the agents are pretty good at that, especially if they are using mature packages. I focus on critical things, like how it enforces permissions.
This works well for me, though it leaves one major issue untouched: whether this is worth building. The agents tend to be a bit overeager, so I have to do a lot of work up front to make sure the output will add value. The gates can only check the artifact, not my intent.
"At the time of writing, about 4% of Bun's Rust code sits inside an unsafe block (~13,000 unsafe keywords across ~27,000 lines / ~780,000 lines), and 78% of those blocks are a single line — a pointer that came from C++, or one call into a C library. I expect this number to go down over time as we refactor from a faithful Zig port (which had no greppable unsafe keyword) to idiomatic Rust, but we are going to continue using C & C++ libraries like JavaScriptCore so it will always have more unsafe than pure Rust projects."
Humans and agents share one thing: they are both non-deterministic. He talks about the issue of tracking memory lifecycles manually in zig so it can be explicitly freed. As expected, this leads to a long list of bugs where people missed things.
Rust does this automatically. It removes an entire class of errors from his backlog. From an engineering management perspective, this looks like a pretty good trade.
The bonus here is that compiler errors are exactly the kind of deterministic guardrail you need to put around coding agents. Claude works really well if you give it a way to test for correctness and "make it compile" is a pretty good target.
There's a general version of this: the artifact you expose plus the test you run on it. Deterministic tests turn stochastic output into a hard guarantee. Wrote it up here if useful: https://michael.roth.rocks/blog/verification-surface/
First, recognize that most of what you want done is actually a sequence of steps, and those steps produce intermediate artifacts. And you can either review or gate artifacts as they flow through the process. The earlier you look, the more leverage you have.
I make software, which follows these steps: make a plan with tasks, make a design for each task, code each task, run the tests. I spend a lot of time on the plan, because mistakes here expand drastically once they hit code.
But it's not just all on me: I write scripts and use tools to make guarantees about the artifacts. For a plan, for example, I have a script that makes sure it has the correct structure. I also use another agent to review it according to my personal specification.
Every artifact has this, because the reliability of the final product of the pipeline is actually composed from all those checks.
I approach this by thinking of verification surfaces. Each one is a layer.
For SQL, I have a tool that the agent can call to run queries. (I actually keep a range of tools: for known queries the SQL is baked into the tool and only results come back; I also have a general SQL tool. Prefer the former; deploy the latter only if you must.)
The general SQL tool itself is multi-layered. The raw SQL produced by the agent is a verification surface, as are the parameters used in the tool call.
There are plenty of existing utilities that validate SQL and define its claims, scope, and if it mutates in a structured way. My tool runs those against the SQL as a verification surface and it hard gates some requests.
It also verifies the parameters. It does this in two ways: at a shallow level, it just makes sure they are valid and sane, so it can fail fast if the LLM does the wrong thing.
But it also does a deeper check, where it calls out to supporting microservices to verify a) permissions embedded in the request and b) consistency of the request with the rest of the system.
Since you asked about challenges in prod: if you are not careful, the LLM can write wildly inefficient queries. Some of those made it through and I had to go back and tweak the prompts to give enough context to have it tune properly.
My suggestion: think through the verification surfaces, figure out what you can verify deterministically, and use that as gates in your tool.
I actually did a little writeup of the "verification surface" here, seems relevant: https://michael.roth.rocks/blog/verification-surface/
https://michael.roth.rocks github.com/mrothroc