Impressive debugging skills, and thank you for the benchmarks. Now I'm wondering if mlx-engine / mlx-lm have these bugs too.
One minor thing: as you are concerned with honest numbers, the graphs should be logarithmic on the y-axis too (like they are on the x-axis). Otherwise it's hard to see whether the curve is sublinear or linear.
One minor thing: as you are concerned with honest numbers, the graphs should be logarithmic on the y-axis too (like they are on the x-axis). Otherwise it's hard to see whether the curve is sublinear or linear.