kbau

I can compile stockfish from source and use it to beat other kids in my class in chess. Behold, I am a chess genius.

I suppose if you all you need is a good enough opponent for the average person out there, sure this is good enough.

I was more talking in reference to why the LLMs in the above linked paper were producing so many illegal moves, and it is because they are not hard constrained by the rules of the game. Of course, a loop can prompt until a valid move is produced and then rendered on a screen. But why do this? I suppose, who am I to say what should be done or not, but a specialized tool being better than a general one at its specific job isn't particularly surprising.

Neural Nets can be trained to play chess very well and have been doing so for a long time (see Stockfish and Leela as some of the most popular/strongest ones - top GMs have no chance against them), but these are dedicated models, where the game rules are encoded in the learning process, as opposed to large language models which are natural language processing models. Technically you can give an LLM a lot of chess books and games and it will be able to spit out chess notation. Put a webapp on top that renders text moves to the board and it looks like it's playing chess. But it isn't really.

I suspect (in a probably ignorant fashion) that this is because learning process has been reading a lot of algebraic chess notation (such as "1. e4 e5 2. Nf3 f6 3. Nxf6 gxf6 4. Qh5! +-") then, to play, generating more of it without considering the rules of the game. This is exactly how it's always felt to me when playing chess against LLMs. Sure, "1. e4 e5 2. Nf3 Nc3" looks innocent to somebody simply learning the syntax of algebraic notation, but that Nc3 by black is an illegal move.

An LLM is the wrong approach for playing chess.