redox99
Joined 4 years ago
Joined 4 years ago
My pipeline is
Opus implements -> Opus and Sol review changes -> Opus verifies the issues from the reviews are real and worth fixing -> Opus fixes the issues.
These days your average Opus 5.5 PR with multiple passes of review by Opus 5.5 and Sol 6.1 agents is much less likely to contain bugs than your pre-AI PR.
Or is such understanding reliant in experimentation because there are "hidden constants" we can't figure out with math alone?
There's two thresholds.
1. Tasks models can do. 2. Tasks models can do with enough reliability that you can YOLO merge without looking at the code.
Most models can do most of 1. That's no longer the interesting part.
2. is quickly expand. I can trust opus 5.5 enough with easy to medium tasks that I can YOLO merge them. For open models, I can only trust them with easy tasks if I want to YOLO merge.
I'd love to have a stronger model that I can trust to YOLO merge hard tasks.
Yes, a more modern smaller model can beat an older bigger model at some tasks, that's what can be confusing.
Having said that, I was working on an AI for a certain video game, and I had Astra 6 and Sol 6/6.1 work for 30 hours straight, with some human steering to get it unstuck every now and then. It could not get it to work right, and the code it created was a mess.
When Opus 5.5 came out in like 4 hours it cracked it.
I didn't try but I'm very confident if Astra failed, the open models would've failed too.
If you think that, it means you're not demanding enough from your models.
Opus 5.5 still makes a lot of mistakes. Even if it can technically do all tasks you throw at it, there's no way it can do it at 100% reliability.
To be clear, AI writes 100% of my code since last year. But it still produces bugs, needs multiple prompts, and has less intuition than a human.