Comment by hashmap

how long until cutting out the token middleman and just keep the cache constant sized but edit it and just send the model new information instead of the same thing over and over. token bad latent good.