imrehg

Thanks for making it :)

Yeah, using it a bunch with local LLMs. It's fast (small system prompt), focused on the basics, made around simple choices, and possibility to call it in scripts.

So far I have very positive experiences, the results are generally better (and not just faster) than in other harnesses with the same models. I've done the same (user) prompts for tasks in a range of them, and compare the outcome to eval between them. Simple (harness) beats complicated one many times. I do wish it could use MCP though, but not as often as I think, and CLI tool usage often compensates on this front with little extra effort)

That is now my usuall go-to harness for things usually. Still figuring out the edges though (when we get off the happy path, like models reaching their max context, or such...)

It's very light, recommed you try it out, and see if it fits your toolbox.

Out of curiosity, what's makes the 125B unsuable? (performance of running it, the quality of that version of the model, or something else?)

Just FYI, I only get a "Welcome to nginx!" page there.

Ripping off the plaster moment? One can only hope.

Recently re-watching the Matrix after reading this book, I had to exclaim - "Ha, that's not the real one, the actual book extremely thin compared to that codex!"

And it's just funny for what details the brain is unable to suspend disbelief...

Interesting case of "fail closed" error handling: "we can't find a contract, so we destroy". I guess their worst case scenarios weigh being sued much higher than losing some stuff they cannot replace (not this manual, but other cases, which might be one-off).

I do wonder what would have happened if some months later they do actually find a contract, maybe slid between a filing cabinet and a wall... Absence of evidence and evidence of absence, aye?

Also enjoy Stallman's title, President and Chief GNUisance, which he definitely tried very hard to live up to, with that reply. (And I won't fault him for that. :)

I did report my test results on the first one. :)

The second I didn't say it's any of llama.cpp's "fault", but it is _related_ to llama.cpp since it's being shipped in another system, aye?

Can't stick to the old hash either, because older version have different bugs. E.g. on older versions the same Qwen3.6 model reliably fails to call specific tools due to template issues, while just having the newer llama.cpp version has that fixed. So different versions - different bugs, rather than no bugs.

Why the beating you are trying to gimme, mate? :)

I just switched to Vulkan, and be done with it. :)

As much as I can tell, the ROCm version of llama.cpp would be a bit faster on prompt processing, but about the same on the token generation as Vulkan. Real life benchmarks don't seem to give any "ROCm or nothing" sort of vibes. And the difference between the performance of different models are way bigger than the difference between the llama.cpp versions (and versus different runtimes like the llama.cpp/GGUF and the MLX runtimes on Mac for the same models)...

I've tinkered enough with the serving, that I'd rather do something with them with, say 10% slower speed, than spending hours on seting things up again... YMMV

I have sent countless small bug fixes even before, to tools that I use that I could dig into when something was off... No big heavy lifting, but plenty of "tinker on your tools" stuff. And I don't think what I did was that special, so maybe the experiences are different?

The current LLM-driven stuff seems to break down the expectations, and now there are a lot of places which just ignore anything I send in (the same tinkers as before), generally, for a while. Though there are some tools that picked up the pace and actually react faster (so YMMV here too).

But there are few things more frustrating as being half-way. Case to point is LM Studio. It's closed source, has bugs (duh!), and there's at least a GitHub issue tracker to report the bugs -- but then by and large nothing happens to those reported things. It's almost worse than not having an issue tracker (then I could justify never to really touch LM Studio again, this way I keep hoping against hope that reports will turn into fixes and thus I keep using and keep reporting...)

Well, a had some new account causing havoc (posting crap on packages, requesting delete of packages...) and they were registered today. I guess I'll have to start flagging.

Also, haven't realised that it was off for almost a month (the email is from 13th July, so I guess few others have realised it either way, if it just makes its way to HN).