Comment by spdustin

What do you consider to be an “average length” prompt? How about a “long” prompt? You mention those in the text, and I’m curious of the token-length thresholds you’re seeing before performance degrades, and if that varies more when higher-importance tokens are distributed across the length versus clustered at the beginning.