sheephess44

MoE weights are much friendlier to low-bit quants than dense weights, because only a few experts are active per token so the per-expert error stays local. But the trap people fall into is that once weights are down to 2-3 bits, the KV cache and activation buffer at long context become the dominant term — further weight quantization stops moving the number. Memory-planning for MoE is really a context-length problem wearing a quantization costume.