To some degree, attention is already a mechanism to make computations from previous tokens useful later. (You can think of the KV cache as a representation of the text so far and all the models thoughts on it.) And since language models are trained on sequences end-to-end, I think this is likely to happen. Multi-token prediction encourages this behavior explicitly but only for the small n token window you define.
That said, there are many works attempting to increase the compute utilization of transformer language models (early exit, mixture of depths) and novel architectures (SSMs etc.).
That said, there are many works attempting to increase the compute utilization of transformer language models (early exit, mixture of depths) and novel architectures (SSMs etc.).