Comment by azath92

While this may be true, their throughput is practically unbounded. this has been a primary challenge to using local models at work, where its normal for me to have 1-6 sessions churning away at once. So while 100t/s on a single session is very fast, theres no way you could get a cumulative ~500t/s from a consumer setup like this, do the bandwitdth bottleneck on a single card/cpu/ram/ssd (tends to be gpu ram bandwidth is the limiter in my setup, but they all have respective limiters to this kind of throughput)