The article can be summarized as: context richer prompt history yields answers that are better aligned with expectations.
> the practical constraint of a finite context window coupled with meta-in-context learning’s rapid prompt length increase
The model is not influenced by the added context. The answers are. The abstractions, or, what the LLM "knows", are in the model itself, whereas the answers are just byproducts.
There is an ongoing related discussion on LLMs lacking world model. One could say LLMs do implement a computable model, albeit a dead, static, non-editable one.
> the practical constraint of a finite context window coupled with meta-in-context learning’s rapid prompt length increase
The model is not influenced by the added context. The answers are. The abstractions, or, what the LLM "knows", are in the model itself, whereas the answers are just byproducts.
There is an ongoing related discussion on LLMs lacking world model. One could say LLMs do implement a computable model, albeit a dead, static, non-editable one.