I suppose what I was trying to say is that in the research community, people were still a lot more willing to entertain alternative modelling paradigms for language at that point, there was much less of a monoculture than there is today. It was really only after ChatGPT that non-autoregressive language modelling came to be seen as a fringe pursuit. Of course that is a highly subjective assessment, and my perspective is inevitably coloured by my own research interests at the time, and those of the people around me.
For what it's worth, I don't believe post-training diffusion models (of language or otherwise) is uniquely difficult, just a lot less thoroughly explored so far.
For what it's worth, I don't believe post-training diffusion models (of language or otherwise) is uniquely difficult, just a lot less thoroughly explored so far.