Comment Comment by simonw llama.cpp offers compiled releases for multiple platforms. This release has the new vision features: https://github.com/ggml-org/llama.cpp/releases/tag/b5332On macOS I downloaded the llama-b5332-bin-macos-arm64.zip file and then had to run this to get it to work: unzip llama-b5332-bin-macos-arm64.zip cd build/bin sudo xattr -rd com.apple.quarantine llama-server llama-mtmd-cli *.dylib Then I could run the interactive terminal (with a 3.2GB model download) like this (borrowing from https://news.ycombinator.com/item?id=43943370R) ./llama-mtmd-cli -hf unsloth/gemma-3-4b-it-GGUF:Q4_K_XL -ngl 99 Or start the localhost 8080 web server (with a UI and API) like this: ./llama-server -hf unsloth/gemma-3-4b-it-GGUF:Q4_K_XL -ngl 99 I wrote up some more detailed notes here: https://simonwillison.net/2025/May/10/llama-cpp-vision/ parent simonw 1 year ago
On macOS I downloaded the llama-b5332-bin-macos-arm64.zip file and then had to run this to get it to work:
Then I could run the interactive terminal (with a 3.2GB model download) like this (borrowing from https://news.ycombinator.com/item?id=43943370R) Or start the localhost 8080 web server (with a UI and API) like this: I wrote up some more detailed notes here: https://simonwillison.net/2025/May/10/llama-cpp-vision/