Comment

Comment by embedding-shape

GPT-OSS-120b/20b is probably the best you can run on your own hardware today. Be careful with the quantized versions though, as they're really horrible compared to the native MXFP4. I haven't looked in this particular case, but Ollama tends to hide their quantizations for some reason, so most people who could be running 20B with MXFP4, are still on Q8 and getting much worse results than they could.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton