practice9

You rather generously assume engineers are in touch with their systems.

Even before layoffs many teams just maintained things org has long lost coherent knowledge of

After layoffs and typical org knowledge churn - you can either rewrite it (but how? Product team responsible for original implement requirements is long gone too) or recoup (reverse document) some of that lost knowledge with AI and actually learn

It's because HN is in AI meta-psychosis :)

Our experience is very similar except we didn't really have a review process before, and now LLMs find bugs before PRs get merged in main.

We had 5x-100x speedups in some legacy but important pipelines, with no regressions (validated after extensively by humans). It's not that the code was actively bad. It's just only 1-5% people in the local SWE market would be able to write code that runs so fast and efficient and benchmark it correctly.

We found a subtle correctness bug that was in production for half of the decade (both GPT-5 and Claude Opus were able to find it), confirmed by human after.

And we keep finding subtle bugs that have been introduced by humans before (despite the human reviews, the particular domain is just difficult no matter how many docs and comments and tests one writes)

LLMs are getting quite good at reviewing the results and implementations, though

I find it hilarious/sad that the 0.5x cheaper Ergo M575 has much better design in that regard (just plastic that doesn’t degrade)

They should have used Claude Code for reviews

Humans cannot reason about code at scale. Unless you add scaffolding like diagrams and maps and …

Things that most teams don’t do or half-ass

A variation of “no taxation without representation”?

The human is a bad co-author here really.

I deployed lots of high performance, clean, well documented etc code generated by Claude or o3. I reviewed it wrt requirements, added tests and so on. Even with that in mind it allowed me to work 3x faster.

But it required conscious effort on my part to point out issues and inefficiencies on LLMs part.

It is a collaborative type of work where LLMs shine (even in so called agentic flows)

Well the system prompt is still the same for both models, right?

Kinda points to people at OpenAI using o1/o3/o4 almost exclusively.

That's why nobody noticed how cringe 4o has become

Kinda similar in a way to China or Russia “disappearances”

But who is the target group?

Last time only some groups of enthusiasts were willing to work through bugs to even run the buggy release of Gemma

Surely nobody runs this in production

I tried the square example from the paper mentioned with o1-pro and it had no problem counting 4 nested squares…

And the 5 square variation as well.

So perhaps it is just a question of how much compute you are willing to throw at it

With LLMs that even might be automated

Well none of the labs have good frontend or mobile engineers or even infra engineers

Anthropic is ahead in this because they keep their UIs simplistic so the failure modes are also simple (bad connection)

OpenAI is just pushing half baked stuff to prod and moving on (GPTs, Canvas).

Find it hilarious and sad that o1-pro just times out thinking on very long or image-intense chats. Need to reload page multiple times after it fails to reply and maybe answer will appear (or not? Or in 5 minutes?). Kinda shows they’re not testing enough and “not eating their own food” and feels like chatgpt 3.5 ui before the redesign

One of those guys needs to be fined for the pump & dump scheme (with SPCE: Virgin Galactic), and the other one should be investigated if he was receiving money from the Russian government or influence agents.