msp26

Fortunes weave doesn't even run at 60 frames. The hardware could be better.

These are hilarious thanks for the rundown

> Our friends at Hugging Face

Hahahahhahahaha

Claude Code Remote control only works with telemetry enabled too.

This Luna pricing is obscene man. 5.6 was good enough for so many use cases (data analysis, structured extraction etc).

Incredible.

Got it, I'll test more deployment params when I benchmark models for vision tasks.

Do you have any tips for Gemma 4 31B in particular? I quite like the model but I feel like I'm underutilizing my rented GPU hard due to skill issues. Throughput should be way higher than this, right?

---

Gemma 4 31B NVFP4, vLLM 0.29, single B200 (modal), FlashInfer, fp8 KV, prefix caching, 32k ctx

Workload: ~6k-token shared prefix (~98% cache hit) + short input, ~250 tokens out, ~250 seqs running. Get ~5-6k output tok/s, flat from 128 to 384 concurrency.

Spec decode (DFlash, n-gram) didn't give a good boost which is annoying because I feel like this task should be easy for a draft model to predict.

Individual request latency doesn't matter I just need as much throughput as possible. Its only active for a few hours when I need it.

Do you have any advice on this front? I use vllm for a project but only for text tasks at the moment.

Relatable. Every time I finish working on something, I spot so many more new things that can be built.

It's completely mental that HF ran into cyber safety blocks trying to use OpenAI models to help defend against the attack. They could only rely on a local hosted chinese model in the end.

Not sure how to fully fix this but I remember a session last week where I got so fed up mid way though reading a response that I used the following:

"give me this again without jargon invented this session at high density

and with a couple (maybe more or less) simple useful ascii diagrams underneath each design"

The context is that I was discussing an experimental new idea for my video game review analysis product.

Designs 1,2, and 3 were horrible: the model even suggested a rejection after the word soup so it would have been pointless to waste my fleeting time on Earth reading it.

Otherwise, I generally really enjoyed using fable for bouncing ideas. It was an absolute joy to have this thing provide useful criticism, analyse sample data, and create prototypes so that I could elevate my understanding of the problem without stepping down from a pure intuition/design headspace.

But I don't consider the purely model written code usable for a feature this important. I'll probably scrap it entirely and start from scratch with newfound understanding.

No the models are just ass at communication without being directed.

Try asking them to make useful diagrams for some stuff in a codebase, out of the box without excessive hand holding they don't make good choices about what's worth communicating and how to do it.

You see this in their pointless frontend copy all the time too.

yep matches my experience completely

But even fable has the annoying tendency to invent new jargon and produce an incomprehensible soup of text.

the hand drawn diagrams and highlights are charming

Asking fable to read it's own model card triggers this btw. Or asking if mitochondria is the powerhouse of the cell.