ertgbnm

This is a very mundane release compared to GPT-4 and GPT-5. I think they probably scaled back a bit after the lukewarm response to the GPT-5 announcement. But it still very weird that there wasn't even a livestream,

The boring question is whether there is a claudeflare issue impacting them all.

The fun question is whether this a rogue AI taking control of the compute of all of the organizations.

Didn't AISI literally report exactly that regarding Claude last month?

It's a failure on all fronts. I think it's an indictment of Flock in the same way that full body scans are a scam. If you are always scanning for illnesses/crimes you will inevitably find tons and tons of false positives without ever meaningfully improving outcomes. The juice isn't worth the squeeze. Flock has been around long enough that if it had a meaningful benefit to safety, they'd be able to prove it in numbers. Instead we just have a long list of instances where it has made the world worse.

Yeah if they were so committed to doing good, you'd think the billions of dollars they have spent would bear some evidence of that by now.

I think you nailed it. If the author was a billionaire in the Netherlands he might not notice the differences as they do in the article.

This has been a frequent problem on fiction writing on royal road. Author promises that they don't use AI to write and then proceed to use AI to generate the cover for the series. They might be telling the truth, but I am not going to spend the time to figure it out for myself. They are really shooting themselves in the foot. A shitty MS paint cover would be less off putting.

I've had the feeling that labs aren't pelicanmaxxing specifically but that they do have some sort of RL environment for SVGs that they are letting the AIs overcook in. Specifically I'm thinking of the gemini 3.1 pro annoucnement that seemed to have a huge leap in animated SVG performance but not much else impressive about it.

So they aren't pelicanmaxxing but they are benchmaxxing in a way. The benefit of the pelican was originally that uplift on the pelican signaled an overall uplift on intelligence. I don't believe that is the case anymore and it is just another jagged edge of model intelligence.

Once you lose, you have lost. Ok, but how does that help us predict when something will lose?

> with AI-generated content excluded from pre-training.

> without distillation from third-party models

sounds like zero unless they are lying.

My point is that if I made someone "smarter" they wouldn't suddenly know "What day, month, and year was Carrie Underwood’s album “CryPretty” certified Gold by the RIAA?" which is an example of a question in the SimpleQA benchmark.

So (in my opinion) knowledge benchmarks stagnating for small models is not evidence that small model agentic coding performance improvement will stagnate soon. Small models do not struggle with syntax, the barrier is not knowledge. The barrier is long context coherence and problem solving, which I don't see a bottleneck on improvements for small models in the near horizon as we get more and more high quality reasoning traces to train upon.

Knowledge benchmarks can't really be improved upon via distillation or RL. It requires those facts be added to the training corpus and for the model to memorize them better. Neither distillation or RL really do that and thus we shouldn't expect improvements on SimpleQA unless some other interventions are being made.

Model intelligence and knowledge aren't necessarily directly related. If we can pack greater intelligence and agency at the cost of it forgetting factoids, that would actually be a good thing. We don't need LLMs to memorize facts, we need them to learn how to interact with the world such that they can find the facts that are necessary and surface them to the user.

If we could distill all of the knowledge out of an LLM and just be left with a very agentic model that only knows facts in it's context, I think some very interesting stuff would happen.

"Shark attacks correlate strongly with ice cream sales" is an entirely true statement that some would argue is also misleading.

Misleading should be removed as a category and replaced with a better hedge like "not sure"

My threshold for asking for help might be a little higher than the median, but, I like this operating style personally. Maybe it's just how I was raised, but the thought of not trying to figure something out for myself first is unthinkable. I fill like I get plenty of human connection at work and collaborate with peers and other disciplines plenty.

I'm not in tech, I'm in civil engineering so maybe it's just a difference in the types of problems we have in different industries.

I do find it very frustrating when an EIT asks me how to do something and it's clear that they haven't even read the instructions page to the excel sheet that they literally have open. I have time to mentor peers and subordinates but I want them to treat my time with the same respect that they treat their own.