I find it hard to believe that every single problem has a solution on the internet, which is what "overfitting" would imply.
I was playing Fallout: New Vegas on Wine, and for some reason, the music on my pipboy wasn't playing. I searched the internet for the Wine errors from the terminal to no avail, and as a last resort asked ChatGPT. It gave me step-by-step instructions on how to fix it, and it worked.
If that doesn't demonstrate that LLMs have some kind of internal model of the world and understanding of it, then I don't know what will.
I was playing Fallout: New Vegas on Wine, and for some reason, the music on my pipboy wasn't playing. I searched the internet for the Wine errors from the terminal to no avail, and as a last resort asked ChatGPT. It gave me step-by-step instructions on how to fix it, and it worked.
If that doesn't demonstrate that LLMs have some kind of internal model of the world and understanding of it, then I don't know what will.