After 3 years of running local models I think the model which fits unquantized (BF16) in the VRAM is the best model for general purpose tasks; fine-tuned SLMs or utilities based on non language models for solving a specific problem (e.g. TTS,STT,RMBG etc.) have been the best use of local AI for me.
Local models for coding, is just not worth the effort IMO; unless of course you have the hardware to fit it unquantized in your VRAM.
Local models for coding, is just not worth the effort IMO; unless of course you have the hardware to fit it unquantized in your VRAM.