Comment by decide1000

On the DGX I get 44.5 tokens per second (NVFP4). With 8 concurrent it's 241 t/s total.

I am using the PrismaAQUA

standard 9.7 t/s

+ Dflash2 30 t/s

+ torch-compile 37 t/s

c8 = 177 t/s