> That's an interesting choice. One question I had when looking at the jev copy on their blog is if one "line" in their output looks / attends to other lines. I think not, since they say it's parallel and not autoregressive.
I don't understand the connection between the lack of autoregression and options attending to each other.
Non autoregressive models can attend to all the inputs simultanously.
An autogregressive model can can attend to all the options in the context of each other by simply writing the options out twice. Autoregressive models actually requires this, since one of them will come later, and the earlier prefill inputs can't attend to the later ones.
I don't understand the connection between the lack of autoregression and options attending to each other.
Non autoregressive models can attend to all the inputs simultanously.
An autogregressive model can can attend to all the options in the context of each other by simply writing the options out twice. Autoregressive models actually requires this, since one of them will come later, and the earlier prefill inputs can't attend to the later ones.