My impression is that there's only a small area at the beginning of the context window that is artificially boosted, after that DLM becomes important.
Full disclosure: this is info I got from Claude. I was discussing DLM ideas with it a while back and while it mostly confirmed (maybe sycophantically) my ideas, it also said that rather than a pure decaying distribution effect with distance, it was U shaped because of the small boost at the beginning of the context window.
Full disclosure: this is info I got from Claude. I was discussing DLM ideas with it a while back and while it mostly confirmed (maybe sycophantically) my ideas, it also said that rather than a pure decaying distribution effect with distance, it was U shaped because of the small boost at the beginning of the context window.