ycui1986

exactly. we need branch predictor for expert weight.

There are a lot of SSD streaming engines these days. But few to actually try some hard features.

There is one that could really improve the speed. Given almost all major models come with MTP head for speculative decoding. The same MTP head could also be used to speculative prefetch the expert weight residing on the SSD. If the expert weight can be preloaded before the GPU actually need them, the speed penalty from VRAM cache miss will be quite reduced.

If the technology demonstrates successful token rate improvement. future models could also come with pretraining heads to preload expert weights, and even make the training be aware of it.

i always thought Ryzen AI Halo, together with DGS Spark, has mismatched compute capacity with memory size. Given 128GB VRAM, people would want to run large models, but the GPU compute is constraint in these types of use case. If the box runs models that don't need high compute, then there is no need of 128GB VRAM.

On the high end side, it is too slow. On the low end size, it waste money on VRAM.

So, dual RTX PRO 6000

I really like the pro version. The pelican is so cute.

32GB RAM on mac also need to host OS, software, and other stuff. There may not even be 24GB VRAM left for the model.

just because a bunch of rockets went up without blowing up, does not mean they are profitable. it cost money to shot rocket, and it is very expensive, reusable or not. most launches are internal launch without external paying customers.

another 60 billion to save a failed AI endeavor.

outputting docx files does not have much to do with model capability. it is about whether tool calling has be configured .

There are also many Chines AI-target GPU/NPU producers. You can get a hold of some boards on taobao.com. They are usable in some way.

No, nVidia and AMD are not the only ones benefiting.

i give it in real ubuntu, no vm, no docker. so long I don't ask it to organize files, it will behave. it has not screw me so far.

qwen3.5 and qwen3.6 are both good at tool calling.

For many LLM load, it seems ROCm is slower than vulkan. What’s the point?

he won't. if anything, openai is falling behind recently. the trend won't change easily. it is like the old time Netscape.