That’s cool. I know context windows are arbitrarily larger now because consumers think that larger window = better, but I think the sentiment that the model can’t even use the window effectively still stands?
I still find LLMs perform best with a potent and focussed context to work with, and performance goes down quite significantly the more context it has.
I still find LLMs perform best with a potent and focussed context to work with, and performance goes down quite significantly the more context it has.
What’s your experience been?