Comment

Comment by prophesi

They were given day zero access to release their quantizations[0]. The lowest quant will still require 256GB VRAM to run inferences, but as always I'm amazed by the Unsloth team by how much they can reduce the requirements for these powerful LLM's, and the easy-to-follow guides to get started with using them.

[0] https://unsloth.ai/docs/models/mistral-large-4

parent prophesi 2 days ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton