Comment by nutrientharvest

Ollama can already run Llama-3 70B with a 4GB GPU, or no GPU at all, it'll just be slow.

Considering this says it's "not designed for real-time interactive scenarios" it's probably also really slow