macwhisperer

lmao *qwen3.8-27b runs in the background*

honestly boggles my mind that actual companies put load-bearing (lol) work behind 3rd party api's like claude or gpt.. also its really funny because some of these companies fired actual humans to put said 3rd party api to work in place of humans... like what happens when ur boy claude stops showing up for work huh!?

super dope!! the future of musical instruments may be web-based.. exciting times

4-bit quant sits at 15.72GB

yeah this model is chefs kiss..

running an untouched, vanilla 4-bit version (Q4_0) I baked myself today (benched it against Q4_K_M (16gb) and IQ3_M (12gb), Q4_0 (15gb) is king)...

this model--

1: over 60% faster than qwen 3.6 version of the same dense 27b model, same engine setup (don't ask how, im not sure either)

2: has better reasoning quality, less "loopy" with its thinking patterns.. most certainly the smartest model on my roster currently

3: has the longest task horizon ive ever experienced (locally or otherwise)...I sent it a bunch of compressed ideas for an app, it sent me back the largest python app ive ever seen in one single ai response pass (80kb text file)

thanks qwen!! hoping to see the full model range get released...

pro tip: if u are building ur own harness with python (recommended) , use "llama-cpp-python"...

I started with "llama-server" and custom stuff around it, which is great for single model setups.. but for multi-model harness with quick switching, llama-cpp-python is peak

big week for open models... seems like companies are noticing the 26-35b sweet spot... though I think a 12b-a1b-MoE model would be helpful for the 16gb folks

I feel like all the ppl complaining itt don't even run open weight models.. wahh billionaire bad is true, but you are missing the forrest for the trees.. they are literally bending the knee and ur mad about it? it literally means local models won.

Basically, he proved that *information is power.* If you don't know which way to go (the subgradient), you're gonna be calculating forever!