Comment by bigyabai

So about 24gb/expert at NVFP4. If you compressed the KV cache, you might be able to get away with 2x 24GB cards and stream the experts.