A common theme in papers like these is that the model chooses word predictions greedily, instead of “thinking” and gaining confidence in its next word prediction.
This begs the question - why don’t people force the model to generate more tokens, until it has very high confidence in its next word prediction?
This begs the question - why don’t people force the model to generate more tokens, until it has very high confidence in its next word prediction?
I can imagine several ways of doing this.