jminnl

Asan!

All big companies pull these kind of tricks. Another variation is to retire the old setting and introduce a new one with a deceptive name that is default on again.

Don't give them any ideas. Trump hates Ukraine ever since their refusal to back up his bullshit claims.

And the aftereffects last for months. Gas stations can't simply switch their red dyed diesel tanks for regular diesel tanks because then everybody tanking from that reservoir will potentially end up fined. The detection gear is so sensitive that if you used red dyed diesel once and then half a year of regular you could still prove it.

Again!

The humor in those is so perfect. "Have you seen this chicken?"

This is without reserve the most idiotic thing I've ever read on HN.

Why is it up to other countries to solve your domestic issues?

The llama.cpp guys don't get nearly enough credit for their work. Though the quality of the codebase is dropping over time, it is still quite high compared to most of the alternatives, and it is still one of the most stable ways to run a large variety of models.

Definitely worth looking at if you have only a single 5090 is ninfer, and various hardware specific forks (3090, 4090).

The problem is that with diverse hardware such a set of defaults is much harder to make. You get this matrix of possibilities: gpu, VRAM and memory configurations and then the model axis. This leads to way too many options. The better alternative would be to have the runner self-benchmark what the best settings are given that it already has access to that one particular configuration.

With llama.cpp once you have the model + the runner on the same box you have from 1 ... 40+ configurations of GPUs (depending on how many gpus you have and how many sub-classes of GPUs) for basic options that will load the model. Then you can start multiplying by different batch sizes (1024, 4096, 8192), CPU thread counts (4, 8, 16), tensor splits (this can get really hairy), P2P enabled/disabled, various caching options, speculative decoding options and so on.

The effect is that you can easily spend a day or more benchmarking. On first run of a new model the software should figure this out by itself.

llama-bench is next to useless for this purpose.

3090 and 4090 are amazing value for the money on the second hand market. 5090's are sold for more than they were worth when they were new and new ones are ridiculously priced.

AI can produce code faster than you can review it and if you are not careful you find yourself on the other side of a trapdoor with absolutely no way back. Your codebase has become a mess that you can only maintain with more AI. But if you are careful and prune regularly you can do well and gain a very good increase in development speed.

The risks are:

- creating a lot of dead code - ending up with substantial repetitions (this is getting better over time but the risk is definitely still there) - testing only on the happy path rather than all execution paths - mixing current and outdated information resulting in subtly broken code - inability to reason past a certain level of complexity, but no signal that this is the case

For each of these risks there are remedies, one of the more powerful ones for me is the ability to just roll back when things have gone too far off the rails, realize in hindsight what caused it and to retry with a much better initial prompt.

Except - detail - it doesn't work...

So, on my 100% hand written blog it comes up as 50% 'likely human written with AI assistance'. Back to the drawing board I guess.

> I use it for that because it's too slow for execution

What kind of hardware and what particular quant?