The good thing with most NNs is that you can go with fp32 almost always (I put almost because I'm not 100% sure that this covers all the cases) for almost 0 performance degradation.
For Parakeet I'm 100% sure because I used it yesterday and it was using just 1GB.
Why do you use GLM 5.3 tho? Unless you have an instance running "free" 24/7 you could use a local Gemma model. It should be good enough, no? Or maybe with openrouter it's cheap enough for such simple requests to be considered free?
Why do you use GLM 5.3 tho? Unless you have an instance running "free" 24/7 you could use a local Gemma model. It should be good enough, no? Or maybe with openrouter it's cheap enough for such simple requests to be considered free?