Comment Comment by acchow > like 180W-250W TDP > running GLM 5.2 would be cool at like ~100 tokens per second for a single sessionYour power consumption estimates are off for this generation of GPUs. A 27B dense model gets 50-80 tps on an RTX 6000 using 600 watts. parent acchow 3 months ago
Your power consumption estimates are off for this generation of GPUs. A 27B dense model gets 50-80 tps on an RTX 6000 using 600 watts.