Comment by sigmoid10

This kind of context management is not that hard, even when building LLMs. Especially when you have huge windows like we do today. Look at how ChatGPT can remember things permanently after you said them once using a function call to edit the permanent memory section inside the context. You can also see that in Anthropic's latest post on Claude 4 where it learns to play Pokemon. The only remaining issue here is maybe how to diffuse explicit knowledge from the stored context into the weights. Andrej Karpathy wrote a good piece on this recently. But personally I believe this might not even be necessary if you can manage your context well enough and see it more like RAM while the LLM is the CPU. For your example you can then always just fetch such information from a permanent storage like a VDB and load it into context once you enter an area in the real world.