carodgers

Researchers asked frontier models to play chess. Have a look at the MAR rates in Table 3. When not explicitly told which moves were legal, no model identified legal moves at a rate better than 80%. Many asked for more illegal moves than legal moves. And even when explicitly told which moves were legal, the models continued to ask for illegal moves. With illegal asks discarded, none of the bots could beat a chess model calibrated to 1100 ELO.

The author of the originating post says that "current frontier models need laborious oversight and guardrails on even the simplest tasks", and he's absolutely correct.

Comment · 1 month ago

"It just can't play it on a master level."

Technically correct. Losing to random play is definitely not master level.

Comment · 1 month ago

"Chess is not an interesting target for AI Research and not for AGI."

I don't care if it's interesting or not. A bot too stupid to play chess is too stupid to be a threat to me. So I reject Dario's fearmongering and that of anyone allied with him.

Comment · 1 month ago

"Do you really think if you would get unlimited tokens and a compute cluster that something like a fable or astra frontier model setup couldn't build a better chess engine?"

Disconnect it from the internet, where it cannot copy from humans, and no, it certainly will not. If it can copy code from humans, then maybe, with guidance. But if a human is not there to offer guidance, no, I don't think it will improve on stockfish.

Worse, if it cannot copy code from humans, it will fail embarrassingly at playing chess on its own. This April 2026 paper found that frontier models which did not have specialized (human guided) tailoring could not beat an amateur-level bot (1100 ELO), and some were so bad they lost to random play.

https://arxiv.org/html/2509.24239v4

Look at the MAR rates in table 3. When not explicitly provided with a choice of legal moves, no bot could ask for legal moves at a rate better than 80%. Many of them asked for more illegal moves than legal moves. All of them continued to request illegal moves even when explicitly asked to choose from legal moves. That's not just less than perfect. That's abysmal.

So, recapping, if the most advanced bots on the planet, in April 2026, are not allowed to copy code written by humans, and are forced to play chess by themselves, they will embarrass themselves. And you and others are asking me to believe they are a threat to me somehow. That kind of talk is disconnected from reality.

What is common with all of the "impressive" examples you gave me? The bot had material to copy from, or a human standing nearby to tell it when it screwed up and suggest a path forward. They are not intelligent. They're so stupid, in fact, I could not trust them to balance my checkbook without knowing guard rails are in place. So spare me the hysterics.

"When do you draw the line of LLM is just some stupid thing vs. AI/AGI?" I'm not sure, but I do know the line is well beyond playing a game of chess without requesting illegal moves 20% of the time. There are 5-year-olds who can do that.

You need to ask yourself some hard questions.

1) Why have I been conditioned to expect that given enough compute, a bot can build anything, unguided?

2) Why have the embarrassing details about bot performance been hidden from me, or at least drowned out by hysterical claims about their wild successes? Why is the human role in these successes never highlighted?

3) What do the people training these bots stand to gain if they convince 51% of the voting population that bots are actually a meaningful threat?