Comment by binarybits

What isn't really true? The passage is about a hypothetical LLM so obviously the exact steps depicted here don't correspond to any particular LLM. But LLMs undoubtedly modify hidden states to reflect context gleaned from other words, right? I don't understand what point you're making.