Comment by Lockal

Well, you can always ask it to predict a next bit in the code. Or even a letter. Maybe combine multiple letter into one and call it a token. We'll give a name to a procedure that gets tokens - decode. To improve speed, we can predict multiple tokens speculatively. To improve quality we'll give it extra space to store Chain of Thought. Ah, wait, we just invented LLMs