Comment Comment by famouswaffles That's not the only reason.More tokens = more useful compute towards making a prediction. A query with more tokens before the question is literally giving the LLM more "thinking time" parent famouswaffles 2 years ago
More tokens = more useful compute towards making a prediction. A query with more tokens before the question is literally giving the LLM more "thinking time"