Ask HN: Are third party gguf/mmproj safe on Llama in production env
So i had been building a side project screenmind ,sort of local ai desktop assistant, and hit a confusing multimodel failure...
texts worked,but vision and image failed no crash just returning <unsused49>. i narrowed it down to unsloths gemma 4 gguf +mmproj broke on llama(atleast thats my hypothesis , feel free to correct me) b10244, and the fix was to just switch to ggml.
So is pinning the tested llama build the only option or how do people using llama on production handle this
texts worked,but vision and image failed no crash just returning <unsused49>. i narrowed it down to unsloths gemma 4 gguf +mmproj broke on llama(atleast thats my hypothesis , feel free to correct me) b10244, and the fix was to just switch to ggml.
So is pinning the tested llama build the only option or how do people using llama on production handle this