Mindless2112
Joined 14 years ago
Karma
1,827
Submissions
547
Member since
Jun 30
Joined 14 years ago
I'm generally pretty pro-AI, but I find this icky. Of course, I wouldn't have noticed except the whiteboard drawing seemed not quite right, so I'll probably be fooled in the future.
I thought the bug was a missing check for the COW flag, but looking at it again it seems it was missing both setting and checking the flag.
I could be misunderstanding the bug, of course.
Though, looking inside my "gpt-oss 20B MXFP4 MoE" model, it looks to also be quantized the same way as the Q8, so that was probably an overstatement on my part.
Still, the Q8 is 12.1 GB and the FP16 is 13.8 GB. Not the ~1:2 ratio you might expect.