Because concurrency with a single "accessible" card quickly diminishes. I have dual 3090s, and on Qwen 3.6 35B A3B at 80k a single card with max concurrent set to 4 will get 70-80tps single request, 50-60 TOTAL tps with 2, 45-50 with 3, and around 40tps with all 4 going.
I know this doesn't exactly match your request, but I'm happy with my 3.6's performance. I've seen people claim they can get 120+tps on a 3090, but I'm not impatient when it comes to streaming text faster than I can read it.
I know this doesn't exactly match your request, but I'm happy with my 3.6's performance. I've seen people claim they can get 120+tps on a 3090, but I'm not impatient when it comes to streaming text faster than I can read it.