rayboy1995

All of it is really about scale, same reason as a single person you can't sell french fries as cheap as McDonalds.

I moved directly from 2.5 flash lite to deepseek v4 flash, its already cheaper and if your prompt caching is good you can save so much more money.

Very interesting I'll have to check this out thank you. This is why I love HN.

Thanks!! I had disabled that previously while debugging, I can confirm this is helping accuracy from what I can tell so far. (And speed since the cache is preserved more often!)

I'm running Qwen 3.6 27B Q5 K M GGUF on a Tesla P40 and koboldcpp using pi.dev as the harness, I gotta say I am impressed. Took some setup and configuring but I already have some code it has made commited and pushed. It can be slow on my hardware at >50k tokens, but the fact I bought this one P40 for like $150 back when the LLM trend started I can't complain. (I have a second one too but I couldn't physically fit the card in my server unfortunately.)

The setup I had to do was important and I had to compile koboldcpp with a few special params for my hardware, I mostly just had Claude figure it out. I don't remember everything I did now but it was very slow and would often stop mid task, it seems it was mostly a parsing issue. It made the model seem broken/dumb, but once I had all that settled I actually am able to use this how I use Claude Code. Disclaimer, I am pretty explicit with requirements, I imagine this fails more when you leave it to figure out things on its own but for my flow its pretty rad.

Currently setting it up as an automated agent now to pull Trello cards, create PRs for them, and move the card to be reviewed.

Command I am using to run: python koboldcpp.py \ --port 61514 --quiet --multiuser --gpulayers 999 --contextsize 262144 --quantkv 2 \ --usecublas normal --threads 4 --jinja --jinja_tools --jinja_kwargs '{"enable_thinking":true, "preserve_thinking":false}' \ --skiplauncher --model /data/models/Qwen3.6-27B-Q5_K_M.gguf --smartcache 5

> Monkey see other monkey cannot go through wall, monkey don't try go through wall.

I mean... we are just monkeys. Did we not learn this way when we were younger?

This is true, I think its good to find a balance between living life and making sure you at least take some good memories.

I have had a number of times I just don't remember a moment, only for a friend to show an old photo they happened to see again and a flood of good memories come back.

This is the difference between science and engineering. What they have done is engineering. If the result is 90% of the way there with barely any effort, its best to move on to something else that may be low hanging fruit than to spend time chasing that 10%.

What is the source for the diminishing returns? I would like to read about it as I have only seen papers referring to the scaling law still applying.

Status page apparently isn't automatic because it is yet to be updated.

Haha not a problem at all.