my 'local llm on a budget' has cost me just about $2,400 in total, spread over a bout 3 years of accumulating stuff.
I got my first 7900xt 20gb on a good sale shortly after they came out in an open box deal on ebay for $590.
After another 7900xt 20gb popped up on facebook marketplace recently for $420, I sniped that and then got a microcenter deal on an asus proart board, 32gb of ddr5, and a 9950x for $1050.
A couple more deals for used cases, an older 1tb nvme ssd I had lying around, and a nice PSU finished the build.
I run Qwen 3.8 27b on linux and have tuned the llama.ccp / rocm setup substantially with the help of codex and claude. I get ~80-110 tok/s depending on all kinds of factors and full 256k context windows with a 6-bit quant.
It's surprisingly capable and I love that I can run it indefinitely, tuning as it goes to behave more like a frontier model.
I got my first 7900xt 20gb on a good sale shortly after they came out in an open box deal on ebay for $590.
After another 7900xt 20gb popped up on facebook marketplace recently for $420, I sniped that and then got a microcenter deal on an asus proart board, 32gb of ddr5, and a 9950x for $1050.
A couple more deals for used cases, an older 1tb nvme ssd I had lying around, and a nice PSU finished the build.
I run Qwen 3.8 27b on linux and have tuned the llama.ccp / rocm setup substantially with the help of codex and claude. I get ~80-110 tok/s depending on all kinds of factors and full 256k context windows with a 6-bit quant.
It's surprisingly capable and I love that I can run it indefinitely, tuning as it goes to behave more like a frontier model.