Comment Comment by mickeyp If all you're going to do is chat with an AI, you don't need a 400B parameter MoE model.13 TPS is unusable for agentic coding, because if decode is that slow, then prefill is absolutely abysmal, and agentic work live and die by prefill speeds. parent mickeyp 12 days ago
13 TPS is unusable for agentic coding, because if decode is that slow, then prefill is absolutely abysmal, and agentic work live and die by prefill speeds.