For people interested in super quick models:
"One cool trick" I like for running those kinds of models very quickly on macOS is converting them to CoreML so they run in the Neural Engine.
I had amazing success with Spleeter, Demucs, MVSEP-MDX23-music-separation, DeepFilterNet, Whisper, Hunyuan3D for 3D mesh generation. Plus a lot of others for both work and leisure.
I also convert the code to Swift without dependencies, but that's a personal preference.
The runtime on some of them went sometimes from 60s to 10s. And since it's just Swift, I can just give them to non-technical co-workers and they don't have to setup a whole Python environment.
If you don't wanna do this by hand, this is something that existing LLM models are good at. Even Luna can do it without much fuss.
"One cool trick" I like for running those kinds of models very quickly on macOS is converting them to CoreML so they run in the Neural Engine.
I had amazing success with Spleeter, Demucs, MVSEP-MDX23-music-separation, DeepFilterNet, Whisper, Hunyuan3D for 3D mesh generation. Plus a lot of others for both work and leisure.
I also convert the code to Swift without dependencies, but that's a personal preference.
The runtime on some of them went sometimes from 60s to 10s. And since it's just Swift, I can just give them to non-technical co-workers and they don't have to setup a whole Python environment.
If you don't wanna do this by hand, this is something that existing LLM models are good at. Even Luna can do it without much fuss.