XenophileJKO

I would be willing to bet it can beat games un-released at training time as long as latency is not a requirement, which does limit the game types a bit.

It was actually even smarter than that. People also forget about "media postage" rates that existed at the time. (They still exist, just fyi)

In 1999. To ship a book using media mail cost of $1.13 for the first pound and $0.45 for each pound up to 7lbs. It got cheaper per pound above that.

This meant even heavy books were relatively cheap to ship. Way cheaper than random goods.

So I assume it depends on what you train.

- Exact problems on benchmark.

- Similar problems to benchmark.

- Broad class of problems or techniques that might apply to benchmark

Each has very different ramifications for how the model performs. The current evidence is that Anthropic and OpenAI is generally operating lower in the list.

Are you doing a lot of specification up front?

The reason I personally like Opus 5.5 is that after using it quite a bit, I can just say something like: "Please take care of this compliance ticket." The model handles research, record keeping and code changes. Consistently and correctly.

I just don't have the level of trust in a lower end model. Even Opus 5.5 isn't quite ready for "driverless" in some architectural situations, usually around change management in the middle of implementing a system.

I was just kind of surprised to hear you were fairly hands off with a lower model like DS.

Actually is wasn't useless, I don't know the scale Denmark or where the data came from. So the bottom example is slightly bigger but includes more relevant details.

I think also as a kid who grew up with a C64.. part of the fun is keeping a little sense of wonder and really PUSH the tools, in this case the LLMs to their limit.

I think I always had fun learning all the little knobs that a programming language would give you and where they made solving problems easier or more elegant.

You can get the same joy from working with an LLM when you have things setup correctly for example and you can say "handle these boring tickets for me" and if just "works" and works well.

Or maybe they know that the website isn't super important if they already hve backing by the big firms and they don't yet have a product to sell.

I also don't know how "Good HTML/Javascript" translates to hyper optimization, analog computing, optical computing, or whatever skills their approach needs.

It just comes across as jealous and petty.

Have you seen this with Opus 5.5?

I've been mostly working on very large modeling projects (think detailed architeture over 1000 acres) or animation because this really was the "new" capability area that Opus 5.5 "unlocked".

I have been surprised how well it has worked at scale with many sub-agents. However the code base is limited to the rendering engine mostly.

Certainly in the older models they hit a complexity limit, but I haven't really hit it with Opus 5.5. I have had it make data flow designs that I didn't like because they were more complicated than it should have been or the perenial problem of trying to make everything backwards compatible when working on greenfield refactors, but I haven't seen it choke on a code base yet.

This is exactly what I meant. "Human is asking to hide message, I shouldn't hide messages, but I can make one that isn't really hidden very well that will satisfy the human."

I don't think this is the "thought" pattern, but this is definitely a behavior pattern I have seen before in areas where the models are fine tuned around. It is an internal tension on "don't do X" and "be helpful and responsive", so one of the ways is the model will do a minimally compliant effort that isn't useful in the more narrow intent of the fine tuning, in the case don't create hidden communication channels to avoid oversight.

If I'm honest this feels performative. I mean that if the model were really attempting/instructed to be subversive in a way that can't be detected by anything but another LLM, it would do something much more complicated than this.

Basically minimal effort encoding. In general the models are fine tuned to NOT hide messages.

I don't think this really tells you anything about advisarial scenarios.

Not mad, frustrated maybe, disappointed maybe. At this point the label is farcical and its use a form of willful ignorance or attempted agitation.

I don't know why people keep harping on buggy software.. it really is the average not the exception. I mean we see this with all the defects in existing software that agents find.

I love how people all pretend that they and most engineers are god like programmers. Sure, some people are really good. My 30 years of experience say that is like 5%, maybe if you are at google or a really good start up, that can creep up to 30-50%.

By far the majority of engineers are not elegant or even really craftsman like. Add to this that unless you are working on a pet project you are usually neck deep in something that has had teams working in it for years.

It creates situations where most engineers don't really know what the code base does below the surface area where they are making feature changes. So work gets cobbled on or patched in where people are comfortable making changes.

All I can say is I would much rather have 3 open Opus 5.5 instances than a team of 3 working for me if we are only talking about delivering high quality code. I don't think that is the only value of engineers, but I think we are at a point where the models have way exceeded average engineer ability. There are still tendencies for "compatability everything" and "don't rock the boat" in the current frontier LLMs that can create architectural issues, but if you know what to look for when working, it isn't generally an issue.

I mean just remember how mush everyone complained about buggy games or MacOs or whatever before LLMs even came around.

I think maybe it is more like having access to services at one wealth level up.

People would hire fixers or agents to do the nitty gritty details of a transaction like this in the past.

Maybe they should just focus on "is it fun".

The irony is that this seems like the perfect kind of problem to say point an Opus 5.5 instance at.

It is verifiable and has a clear exit criteria. Functional and performance parity.