Comment by regularfry

If this is a way to get equivalent results to a much larger network in the same FLOPs but with a fraction of the VRAM, it's transformative.

I'm particularly keen to see if you could do speech-to-text with this architecture, and replace Whisper for smaller devices.