Comment by simonw

I got a quantized Llama 70B model working, using most of my 64GB of RAM but it's usable:

    ollama run hf.co/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF:Q3_K_M

That's a 34GB download. I'm accessing it via https://github.com/open-webui/open-webui which I ran like this:

    uvx --python 3.11 open-webui serve

I have TailScale on my laptop and phone so I can run experiments directly from my phone while leaving my laptop plugged in at home.