nacs

With Strata and specialized engines, Qwen Flash Next runs even on 16GB cards.

I use it at 60+ tok/second on a single 24GB card.

They might have spent next to 0 time actually 'hacking' - they may have just told Opencode to hack Opencode website and let it run for hours or days till it succeeded.

Hm? Jev's actual (network) latency is not that great and even small local models are doing far better latency-wise.

The Microsoft article GP linked even shows the MS model having 95ms latency.

Even on price Jev being matched (the same MS model is "Input tokens cost $0.042 USD per million tokens. Output tokens are free.", same as Jev).

Cloudflare's Clef-flash model is actually slightly cheaper: "$0.038 in / $0 out per 1M" too.

Jev is being matched or exceeded in performance and price within a month of them going public.

Coder is not as good and seems to have noticable loss.

Use the normal one - I've used that for long coding sessions and it worked well on multi-hour runs.

@the_mitsuhiko - How is the session replay of Pi embedded in the article?

It doesn't look to be asciinema?

I'm seeing 128GB DDR5 RAM costing $3500+ just by itself.

How are you going to build an entire system shipped for $3k?

This spec sheet looks more like a dreamed-up spec sheet than pricing reality.

Is the Magic 8 ball immune to hallucination as Jev is?

It would be good to list 1) zero-shot accuracy and 2) latency on the models page . The LLM-based models' latency is probably much higher than the BERT approaches I would assume.

Also curious, it seems from looking at the accuracy scores you gave that it seems to be NLI > Gliclass > Laya (for Bert types)? Why do you seem to feature/recommend Laya more - is Laya better in some way?

Apple product won't be the cheapest but it is a full package (CPU, RAM, VRAM/GPU, fast-storage, etc).

If you look at the pricing of a full (x86) AI workstation you'd need around the nvidia GPU, you'd approach $10k easily (and be using a ton more wattage too).

Good to know thanks.

That's with CPU offload to a DDR5 6000 RAM though which is around $3-4k at least.

5090 has 32GB VRAM.

It'd be silly to buy the 18k model to run a tiny model like Qwen 27B. You use models like GLM Flash and Qwen Next which won't fit on a single 5090.

People don't buy Sparks and M5 Ultras to run a 27B model - you buy it to run an MoE model like Qwen Next which this M5 excelled at.

That's a dense model. Of course it will do worse.

Now try running that Qwen 3.8 Next model on the 5090 and tell me what TPS you get (hint: it's near 0 since it doesnt fit the 32GB VRAM on 5090 vs the 256 in OPs M5).

It was yes, lots of words in the input to describe state of the game-world at each checkpoint.

Easily. It took around 4GB VRAM on my nvidia card so a 48GB unified mem should have no issue. < 1s inference