Comment by zodzedzi

> Generally, the models are compressing their understanding of all text, and in doing so, they're learning high order concepts

Are these higher order concepts accessible to us? E.g. can we list those learned concepts?

(Re-reading the paper you linked now...)