This is why I think the next evolution will be giving models full control of their own context, where they can edit it just like a file. This idea was in a recent paper, where models could even be specifically trained for context management [1]. For the problem of the prefix cache being invalidated, they thought of implementing a suffix cache
[1]: https://arxiv.org/abs/2609.37725