This is so easy to disprove it makes it look like the author didn't even try.
Here is the convo I just had:
me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed
ChatGPT: Sure, I'd be happy to help! Please provide the move sequence and I'll give you my response.
me: 1. e3
ChatGPT: 1... e5
me: 2. ne2
ChatGPT: 2... d6
me: 3. g4
ChatGPT: 3... h5
me: 4. g5
ChatGPT: 4... Ne4
Ne4 is illegal. Also you'd be hard pressed to find any human player that would miss the hanging pawns much less a 1400 elo player. GPT has been trained on chess games and is predicting patterns, this has been known since the GPT-2 days. The daily hype articles where someone coerces a statistical model into "demonstrating" an emergent capability sure are getting old. Claiming emergent capabilities (like having learned chess) are an extraordinary claim (LLMs most likely cant learn things) and I really wish people would put more effort when they make these claims. But the AI hype problem is as old as AI research itself...
Here is the convo I just had:
me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed
ChatGPT: Sure, I'd be happy to help! Please provide the move sequence and I'll give you my response.
me: 1. e3
ChatGPT: 1... e5
me: 2. ne2
ChatGPT: 2... d6
me: 3. g4
ChatGPT: 3... h5
me: 4. g5
ChatGPT: 4... Ne4
Ne4 is illegal. Also you'd be hard pressed to find any human player that would miss the hanging pawns much less a 1400 elo player. GPT has been trained on chess games and is predicting patterns, this has been known since the GPT-2 days. The daily hype articles where someone coerces a statistical model into "demonstrating" an emergent capability sure are getting old. Claiming emergent capabilities (like having learned chess) are an extraordinary claim (LLMs most likely cant learn things) and I really wish people would put more effort when they make these claims. But the AI hype problem is as old as AI research itself...