The way this model operates is the equivalent of machine learning shitposting.
Broke: Use a text encoder to feed text data to an image generator, like a GAN.
Woke: Use a text and image encoder as the same input to decode text and images as the same output
And yet, due to the magic of Transformers, it works.
From the technical description, this seems feasible to clone given a sufficiently robust dataset of images, although the scope of the demo output implies a much more robust dataset than the ones Microsoft has offered publicly.
Broke: Use a text encoder to feed text data to an image generator, like a GAN.
Woke: Use a text and image encoder as the same input to decode text and images as the same output
And yet, due to the magic of Transformers, it works.
From the technical description, this seems feasible to clone given a sufficiently robust dataset of images, although the scope of the demo output implies a much more robust dataset than the ones Microsoft has offered publicly.