lmxyy

Introduce Radial Attention — a static sparse attention mechanism with O(nlogn) complexity for long video generation! Here are some key features: * Plug-and-play: works with pretrained models like Wan, HunyuanVideo, Mochi * Speeds up both training&inference by 2–4×, without quality loss * Compatible with pre-trained LoRAs. When applied to 8-step FusionX LoRA, Radial Attention further delivers a 1.6× speedup

Thanks for pointing this out. I've fixed the prompt. Both FLUX and PixArt use T5 for the text encoder, which has limited capability. Our quantization method can preserve the image quality and contents of the original 16-bit ones well.

I think so. There are already some techniques called rotation, which have similar effects. But it will incur additional overheads in diffusion models.

FLUX-schnell is only 800ms on RTX 5090.

With the arrival of the RTX 5090, we built a high-performance workstation to maximize its AI computing potential. In this blog post, we share our experience—from overcoming setup challenges to testing its performance.