delicious_apple
Joined 5 months ago
Karma
12
Submissions
4
Member since
May 21
Joined 5 months ago
Prefill: ~1000 tok/s
Decode: 75-100 tok/s
It'll be far faster on a 5090, but I find the above performance to be acceptable. I've seen some claims that it even works OK on an AMD RX 7900XT (~$500USD)
Some folks on Reddit are having the same experience: https://www.reddit.com/r/LocalLLaMA/comments/1vkm42m/muse_gl...
It uses an order of magnitude less VRAM at longer contexts which is a huge advantage over Qwen 3.6 27B
At least for my use cases, Qwen 3.6 35B-A3B appears better than this new release. An MoE for Qwen 3.8 would be even better than that!