trouve_search

Why would ai advancement make self hosting painful? Apart from increased cost of hardware (which you implicitly would pay for in PaaS as well anyways).

If anything, AI makes running linux boxes much easier than 4 years ago; it's now faster and the learning cliff is gone to get the correct CLI commands and digging into system logs.

It's rarely reminded that when dealing with professionals (doctors, lawyers, accountants, etc.), it's important to get second opinions if you have any doubts.

It's an article with no clear conclusion it's normal to feel confused.

My read of the article is balancing the fact that there's a lot of overlap between CPTSD/ADHD/ASD in the symptoms (emotional dysregulation, hyperarousal, etc) and in hereditary factors (undiagnosed parents causing trauma more often on average). Also that traumatic childhood experiences are more likely to stick around as CPTSD in adulthood if there's also neurodivergence.

The author says there can be incredible relief to be correctly diagnosed with ADHD/ASD, so obviously she says it's helpful.

But she also warns that wrong treatment can often happen (eg. Giving stimulants to perpetual fight or flight PTSD brains), or inneffective therapy.

It's important to note that languages evolve much faster without a writing tradition to anchor the language down over time.

Couldn't you put some sort of faraday cage around the antenna?

pi.dev + anything else

That's AMD's fault.

RDNA4 is pretty similar to CDNA4, yet over a year after the release of "pro AI" cards like the r9700, they had basic kernels lacking in vllm (like w4a16 int4 kernels) while they were implemented in the datacenter CDNA4 cards.

AMD hardware runs well on llama.cpp because basically anything runs on llama.cpp, especially with vulkan. It's not high praise of AMD's software team to say llama.cpp runs well on their hardware

Yes, it runs in vllm happily. It gets >900TPS output reliably on a single 5090 with the nvfp4 model.

It's clearly worse than vanilla 26B-A4B, and lacks some things like structured outputs, and gets some tool calls wrong.

So you have to find a usecase or a hand rolled harness that leverages the cerebras-level TPS while not going off track during (even short) tasks.

thanks for posting your setup! I think it's smart to set the reasoning effort default to something saner in the base config.

Here's a VLLM command for 3.6 (I'll update to 3.8 today) to test out:

```

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \

vllm serve Qwen/Qwen3.6-27B-FP8 \

--dtype auto \

--kv-cache-dtype fp8 \

--enable-chunked-prefill \

--enable-prefix-caching \

--trust-remote-code \

--enable-auto-tool-choice \

--reasoning-parser qwen3 \

--tool-call-parser qwen3_coder \

--speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":3}' \

--default-chat-template-kwargs '{

    "enable_thinking": true, 

    "reasoning_effort":"medium"

 }' \

 --tensor-parallel-size 2 \

 --max-model-len 250000 \

 --gpu-memory-utilization 0.9 \

 --max-num-batched 12000 \

 --max-num-seqs 24

```

I took the liberty of adding your reasoning effort chat template to my setup. You can play around with the last few parameters. In generall VLLM will be better in higher concurrency scenarios, so if you only use it for a personal vibe coding assistant and less as a general home model for task execution llama.cpp may be better.

Yes, I mentioned the setup, but on vllm you can only use TP with speculative decoding or pipeline parallelism without, so there's tradeoff to both.

I gave general numbers of what I'm getting above, the performance ratios seemed similar regardless of setup (eg. getting a AWQ-in4 quant on a single GPU vs PP without speculative decoding vs TP with speculative decoding).

Overall single GPU is fastest, and TP+speculative decoding is still faster than PP, but for fp8 models you need dual GPUs whether you want it or not.

I think it's a vllm vs llama_cpp performance thing, will pay more into it.

One note I had between the two is that gemma has a much higher prefix cache hit rate in general.

What configuration are you using? On both vllm and llama-cpp, I get significantly higher speeds from gemma4 than qwen3.6 (with their respective speculative decoding methods).

Output TPS in vllm for instance:

- Gemma4 26B-A4B: 200-300TPS

- Qwen3.6 35B-A3B: 120-180TPS

- Gemma4 31B: 80-120TPS

- Qwen3.6 27B: 60-80TPS

This is for a first request on a dual 5090 setup, with their respective speculative decoding methods.

Using 98.css would still leave you with the AI slop text wording.

The core problem is that some people don't even seem to notice / care.

Laguna XS is MoE, however.