Comment by cgdl

Exactly what I was thinking.

What sort of latency do you think one would get with 8x B200 Blackwell chips? Do you think 1500 tokens/sec would be achievable in that setup?