Comment by hyferg

You can return log probs per token generated. This can be used to asses the confidence the model has in handling tasks which involve nominal data.

If that’s not helpful, were you getting at having the model return some rich data about the attention weights that went into generating some token?