Comment by wilg

I was just suggesting something like this to a friend yesterday! (Neither of us know enough to do it or know if it's a good idea.)

However, I do think the context length is one of the top improvements that would make LLMs much more useful.