Comment by magicalhippo

And a 5090 has 1.79 TB/s.

So yeah on its own not very impressive.

The large amount of memory means you can serve larger models, or less quantized, but that also means they're going to be slower due to the low bandwidth.

https://www.techpowerup.com/gpu-specs/geforce-rtx-5090.c4216