I'm trying to understand what difference does this make over LLMs.
LLMs are universal simulators, their latents model the world. So I bet if you compare their logprobs with probabilities output by this model, it will be highly correlated.
Someone should do this quick experiment. I bet there won't be enough of a meaningful difference.
LLMs are universal simulators, their latents model the world. So I bet if you compare their logprobs with probabilities output by this model, it will be highly correlated.
Someone should do this quick experiment. I bet there won't be enough of a meaningful difference.