Comment by avereveard

btw that how mmlu score things to answer question instead of producing all the answer token they look at logprob of a b c d keys in 2020 making this technique old as dirt in nlp