Comment Comment by yjftsjthsd-h In case anyone else clicked in wondering what counts as "real time" for this:> On a Pi 5 (16GB), Q3_K_S-2.70bpw [KQ-2] hits 8.03 TPS at 2.70 BPW and maintains 94.18% of BF16 quality.And they talk about other hardware and details. But that's the expanded version of the headline claim. parent yjftsjthsd-h 10 months ago
> On a Pi 5 (16GB), Q3_K_S-2.70bpw [KQ-2] hits 8.03 TPS at 2.70 BPW and maintains 94.18% of BF16 quality.
And they talk about other hardware and details. But that's the expanded version of the headline claim.