sadilek

True. But also, polishing it still takes considerable time and thought. So using something someone else (or a community) built can still be beneficial.

It seems they had to choose very a specific benchmark to come out on top of. But at least there is _something_ coming out of Europe. Maybe not all hope is lost.

I also never understood this desire to build an agent org chart with persistent identities, often giving them names. Paperclip seems to advertise that model. I think it's much simpler and more flexible to simply spawn an agent for each task.

One argument for the persistent identity model is that you can give them special skills and context. I don't believe that's a good argument. In my experience, you can just give each agent access to the full set of tools (including RAG search) in the form of documentation and skills, and they'll use that efficiently. Skills have a tiny front matter and hardly bloat the context, and the documentation just needs to be tree-structured with cross-links for progressive disclosure. LLMs navigate that efficiently, so the cost of having all context accessible is low, while the risk of cutting the context boundaries for the persistent identities non-optimal for a given task is rather high IMO.

Are they though?

If you've been a tech lead of a group larger than, say, 10 people, you'll know the feeling of not having enough time to review all the code changes. But still, you're expected to and usually can contribute to important directional decisions. You have some confidence in being able to smell issues from patterns of software behavior. I think it might be people with that experience who tend to be more comfortable in delegating more to agents. That seems like the more likely predictive factor to me than "good" vs "bad" developers.

I find the time unproblematic. A bug fix is typically 30 seconds, a new feature 1-3 minutes. I want to review those before they go out, and any other way to review them I can think of would take longer.

FWIW, if the work is better shown on an HTML page (e.g. to show stats of a previously flaky test), the agent can do so.

I was able to go from 5 to over 20 parallel sessions without burnout by changing my role. I switched from engineer to engineering director. I try to focus on directing and deciding. I built a software for that for myself. Two key features:

1. Each project (= one or more repos) gets a canvas. On it, work is laid out as cards. Position and color let me build a mental map.

2. Agents end each card with a demo, which is typically a narrated video of the new feature or bugfix. I review it like an engineering director in a launch review. If I have comments, the agent goes another round.

With this, I don't feel the pain of context switching because I don't need to keep the context all in my head in parallel. I get it presented every time a card wants something from me.

"I work with models edit by edit" – That could be the cause of bias: Maybe because you're supervising small step iterations of the model, you notice issues that the model would have noticed itself if you had let it tackle a larger scope by itself. Worth a try in my experience. And the extra review agents make a big difference. Even if it's Opus reviewing Opus. Just by having a fresh context window, the model finds issues it created itself in another session. And with a different model, you get yet another fresh perspective. I find Codex is particularly thorough in reviews.

Interesting. Two thoughts:

1. Are you using the latest frontier models? I know that feeling: "These things are creating some wild code. Then sticking to its guns in some cases and just rewriting the same bug over and over." But that was a few months ago. With Opus 5.5, it's gone.

2. For debugging, I give Claude read-only access to the prod DB, and it has access to logs and traces (Signoz in our case). With this, it has amazing debugging skills. Orders of magnitude faster than myself, and it has yet to fail finding the cause of an issue.

How do you judge that the e2e test is correct? Trust it, read it, watch it? I'm watching demo videos.

Haha, well, I don't because I have no reason to assume this is in the code. In fact, I have reason to assume it is not in the code because the models aren't trained to produce this.

Could there be a less egregious version of that in the code? Sure, but then that's just a normal bug where implementation doesn't match intention, and that happens to human developers as well. I find that Opus 5.5 is so good that it writes pretty good code that follows YAGNI, KISS, DRY, and is fine with writing a lot of tests, so my sense is that the rate of normal bugs is actually lower than with human developers.

Now, if you're assuming malintent on the side of the model creators, well, that's another story. FWIW, in a commercial project, where I also work at the "engineering director level" since Opus 5.5, all non-trivial changes go through reviews by Codex and Grok (locally) and Greptile (on the PR), so there would have to be some serious level of collusion by the labs to produce your example ;)

I think the answer is: Think of yourself as a maker, not as a programmer, and you'll be good.

I feel like a good chunk of my skills that I honed over decades as a software engineer are obsolete: programming, debugging, navigating code. But there are other skills and especially experiences that, at least with the current level of tools, allow me to do much more than someone who doesn't have them: knowing what computers can and cannot do, or only inefficiently; distributed systems; deployment; CI/CD pipelines.

I can't prevent them, and in fact, I don't want to. Didn't mean to imply that.