manny_rat
Joined 6 months ago
Karma
5
Submissions
6
Member since
Apr 25
Joined 6 months ago
My guess is that it is the fault of the model rather than the harness, I believe Opus to be much worse than it was for whatever reasons. Though I suppose it could be Code’s fault somehow. For the time being though Codex is much better which I never thought I’d be saying.
I plan to run tests using Pi so they have the same system prompt and harness, but I’m suspicious that it’s only the subscription level Claude Code that is worse and we’re not allowed to use that with Pi.
Is this true? I’ve worked in various software companies for over 20 years now and I’ve never had to worry about risk in particular, and the codebases were all somewhat bad in areas and buggy (as tends to happen when a codebase gets large and old).
The past 6 months LLMs have crossed into barely needing to check the output territory for what I work on, and generally write very similar code to what I was going to write. It needs some common sense to use it correctly of course, like going feature by feature and keeping the commits fairly small, but my job is easily 10x less work/time for the same results.
I’d be genuinely surprised if even 1% of software requires aviation levels of risk tolerance and testing, but maybe I’m mistaken and most people are working on much more mission critical things than I am?