Comment by metadat

The article mentions Q4, Q5, Q8, and NVFP4. It's total AI slop though, tough read.

In my testing I got 150 tokens/sec with a single 5090 RTX.