osanseviero

Hi! What implementation are you using? Right now VLLM is the one recommended. llama.cpp is in an early draft

Hi! The model is 8B if you also load the vision and audio components. We just used the text model in LMArena.

Hi! Omar from the Gemma team here.

Last time we only released the quantized GGUFs. Only llama.cpp users could use it (+ Ollama, but without vision).

Now, we released the unquantized checkpoints, so anyone can quantize themselves and use in their favorite tools, including Ollama with vision, MLX, LM Studio, etc. MLX folks also found that the model worked decently with 3 bits compared to naive 3-bit, so by releasing the unquantized checkpoints we allow further experimentation and research.

TL;DR. One was a release in a specific format/tool, we followed-up with a full release of artifacts that enable the community to do much more.

Please make sure to update to the latest llama.cpp version

Yes, they are still used

- Encoder based models have much faster inference (are auto-regressive) and are smaller. They are great for applications where speed and efficiency are key. - Most embedding models are BERT-based (see MTEB leaderboard). So widely used for retrieval. - They are also used to filter data for pre-training decoder models. The Llama 3 authors used a quality classifier (DistilRoberta) to generate quality scores for documents. Something similar is done for FineWeb Edu

Yes, there are a few dozen full open source models (license, code, data, models)

Hi all! I'm Omar from Hugging Face. Happy to answer any questions you might have about Hugging Face in general, llamas, and open ML!