Comment by tomlagier

I wonder if there's something that you can do as you're nearing the end of the context window to summarize the "state" of the world so far and put it into context so that the model always has context on the most important details. I'd imagine that you'd lose some of the minor stuff by compressing context this way, but, hey, humans forget minor details too.