Comment by missingrib

Right, it's an illusion of understanding. There is some sort of symbolic understanding, but that is completely due to the fact that the training data was made by humans who actually do understand, can interact with the world, and can write their thoughts down so that the LLM can insert some sort of reference to "basketball" and "Michael Jordan" in their embeddings or whatever.