> The internal models are more eager, and there is the scale difference of being able to spin up 10.000 vs 10, but to get something valuable out requires skill.
Yeah, but the skill is knowing when it's talking nonsense.
Same deal with the recent stuff about vibe coded Photoshop clones. Yeah, they work, but you'd have to do Adobe-level market research to understand what bar the product needs to pass, and the vibe-coders didn't have that, so they saw an image editor and mistook it for "basically fine".
> Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Sure. However, one of the ways we see things change with, e.g. METR time horizons is that agents fail (IIRC) 10% of the tasks of some (human) duration all the time, pass 10% of that (human) duration all the time, and the rest have a random success rate you can fit to a sigmoid. As the agents get better, they still pass 10% all the time, still fail 10% all the time, but it's a harder set of questions.
2.5 years ago, I was thinking "I know what's a fun idea for April fools day! A programming language written entirely in runes!", the LLM produced non-compiling code for the compiler. 1.5 years ago, I thought "I'll give that another go", it managed it trivially and wasn't even interesting enough to bother with as an April fools day joke, not even with the video of a Viking explaining it to you in Icelandic.
We should probably assume the companies are 1-2 models ahead of the public, at all times. The HuggingFace incident happened about 2 months before the public got models of comparable competence.
Yeah, but the skill is knowing when it's talking nonsense.
Same deal with the recent stuff about vibe coded Photoshop clones. Yeah, they work, but you'd have to do Adobe-level market research to understand what bar the product needs to pass, and the vibe-coders didn't have that, so they saw an image editor and mistook it for "basically fine".
> Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Sure. However, one of the ways we see things change with, e.g. METR time horizons is that agents fail (IIRC) 10% of the tasks of some (human) duration all the time, pass 10% of that (human) duration all the time, and the rest have a random success rate you can fit to a sigmoid. As the agents get better, they still pass 10% all the time, still fail 10% all the time, but it's a harder set of questions.
2.5 years ago, I was thinking "I know what's a fun idea for April fools day! A programming language written entirely in runes!", the LLM produced non-compiling code for the compiler. 1.5 years ago, I thought "I'll give that another go", it managed it trivially and wasn't even interesting enough to bother with as an April fools day joke, not even with the video of a Viking explaining it to you in Icelandic.
We should probably assume the companies are 1-2 models ahead of the public, at all times. The HuggingFace incident happened about 2 months before the public got models of comparable competence.