Comment by GaggiX

What is stored is not the tokens, but all keys and values of all attention layers for each token.