Silagi

That seems like the least charitable reading of what I wrote.

"system they're working on" refers to the actual physical hardware.

Yes, I look at the code regularly, and a significant portion is hand written, especially the performance critical sections.

A security issue I haven't been able to track came up in automated code reviews. Maybe you're such a savant that you can crack any security issue in a codebase you're working on, but the last few months of critical security bugs in infrastructure with the most experienced and capable maintainers in the world should at least humble you a little.

That was my experience in the Opus 4.5 - 4.7 era, but recently it's been on the nose with what I want it to carry over, or close enough that a full handoff would be more expensive than just letting it find what it needs again.

I'm mostly working on moderately sized rust codebases (600k~ lines) with reasonably long running tasks. The only time I intentionally do a handoff is if the next goal is in an entirely different section of the code, or I let the session cache expire, or I'm working with 6.1 Sol for more than a few hours, which admittedly has some horrendous context drift compared to Opus.

Can I ask why? The modern models deal with compaction remarkably well, to the point where I dropped my autocompact from 1M to 350k. I've seen no change in efficacy and a reduced usage rate.

I've been working on a game server in rust, and every once in a while a review will just stop and refuse to explain what happened. So I know there's a security issue in there somewhere, but neither OAI nor Ant models will tell me what it is. Despite the whole thing, top to bottom, having been built in the last two months on the system they're working on.

Planning to hop on openrouter this weekend to chase it down, but man, it's rough knowing I've already burned the tokens to find it probably half a dozen times at this point, but don't have any information that would help point me at it.

A lot of the work is carried over model to model, so I think the most comparable product is a live service game. Always have to be building on the last thing, and if you stop you'll essentially lose the entirety of the initial investment because you won't keep anyone's attention. Plus a healthy dose of "winner-takes-most".

Just. Yaknow. 100x the costs, at the least.

Maybe it's my tasks, but in the cases you're describing, I'm usually better off going down to Luna. I specifically don't see the gap where "It's too complex for Luna-high" and "It's not complex enough for Sol-low" exists.

Can anyone give me the compelling reason for Terra and Sonnet to exist?

Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.

That's wild to say when comparing to the OpenAI family of models. I've never seen so many tests that validate tests that validate tests.

Not OP, but it launched in June 2022, so less life than I'd like to see from a Thinkpad regardless of how you slice it.

The early image models had a sort of urban legend briefly developing around them about this mishappen, horrifying woman that would appear if you just let the models recursively recreate the same image over and over.

There was a weirdness going on with the first Midjourney models that I miss.

The videogame industry, in a nutshell.

Further, this benchmark looks to have done too poor a job of canvasing the model/reasoning level space to claim to represent the pareto frontier. Only running 6.1 sol and opus 5.5 on max seems absurd given that their other reasoning levels represent almost the entirety of the pareto frontier above luna for coding.

Is it low quality in the "It didn't match the meaning that was conveyed" sense, or in the "It didn't capture the exact words used" sense?

My reaction to seeing the project was that being an embedding model it'd be pretty good at conveying the meaning but not great as using the same terms, is that what you're seeing?

My experience has been that most people don't even understand what computers were capable of before LLMs; I work in a chemical company, and I can count on one hand the number of people who knew SAP has GUI automation built in.

Don't even try to explain that "copy data from X program to Y program" is something computers should have been handling for the past 20 years. It's been someone's job the whole time.

We haven't cleared the hurdle of "Processes a 100 line python script could automate in 2010", I don't think most people will be leveraging LLMs to their potential by any stretch.