wr
hn
Top
New
Best
Ask
Show
Jobs
Back
d
dome271
Joined 3 years ago
Karma
17
Submissions
5
Member since
Jun 27
View on Hacker News
Recent submissions
Comment · 3 years ago
I also explain more in the video I made about Paella (
https://youtu.be/zdE1I6kYKYc
). Maybe that clarifies things more.
Comment · 3 years ago
Fully correct, also the v2 of the paper introduced a model that is bigger and slower, however generates better images. So the 500ms was only for the first model we introduced in v1. I also want to mention our new work as it is very much related to this whole topic of "speeding up models" -> either training or sampling: Würstchen:
https://github.com/dome272/wuerstchen/
With a current version we are training at the moment we can sample (using torch.compile) 4 1024x1024 images in 4 seconds. Also the training of this kind of model is very fast due to spatially encoding images much much more -> 3x512x512 images -> 16x12x12 latents => 42x spatial compression, whereas StableDiffusion has an 8x compression (3x512x512 -> 4x64x64).
Comment · 3 years ago
Yea you can. It's the same as with any other CNN based model that is not forced to have a specific shape (like transformers if they use specific positional embeddings). You can also look at the blog post where different resolution images are generated
https://laion.ai/blog/paella/
Comment · 3 years ago
that's the correct answer. I love the previous idea posted above, but it was really just that the model was good at food initially and Pablo is from spain, so we decided to name it after same spanish dish. We are holding up this tradition and follow-up models were called Arroz-Con-Cosas, Risotto and Würstchen.
Comment · 3 years ago
Hey, (one of the authors here). First of all the blog post is talking about the v1 of the paper, which was extremely fast, but not comparable to SD in any way. The v2 in arxiv is slower and does not achieve 0.5 seconds, but performs much better and closer to SD. So no doubt on this. But I just want to mention that you should not compare apples with oranges. Torch.compile also makes Paella much faster and using an optimized sampling pipeline it would always be faster than SD at 256x256 if you keep the conditions the same. Of course you could talk about distilling SD and then you can achieve maybe 1 step predictions etc. But you could probably do the same to Paella. I think it's important to stick with the main improvement from the paper that naive sampling can be done with much less steps when sticking to the original method, while being simple in its theory and implementation. But hey, way to go and improve on Paella in the future maybe :D