Comment by notarealllama

OpenwebUI and you can run quantized or low end models (Llama 3 4b or gemma 4b) on a 4-6gb graphic card.

It's a game changer to run local (no usage caps for a weekend blitz project)