> if a llm will run with usable performance at that scale?
Yes.
The reason: MoE.
They are able to run at a good speed because they don't load all of the weights into the GPU cores.
For instance, DeepSeek R1 uses 404 GB in Q4 quantization[0], containing 256 experts of which 8 are routed to[1] (very roughly 13 GB per forward pass). With a memory bandwidth of 800 GB/s[3], the Mac Studio will be able to output 800/13 = 62 tokens per second.
Yes.
The reason: MoE. They are able to run at a good speed because they don't load all of the weights into the GPU cores.
For instance, DeepSeek R1 uses 404 GB in Q4 quantization[0], containing 256 experts of which 8 are routed to[1] (very roughly 13 GB per forward pass). With a memory bandwidth of 800 GB/s[3], the Mac Studio will be able to output 800/13 = 62 tokens per second.
[0]: https://ollama.com/library/deepseek-r1
[1]: https://arxiv.org/pdf/2412.19437
[2]: https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac...