Comment by liuliu

Still need 2 or 3 days to properly quant this. But it is really good model to run if you have 64GiB or larger RAM on Mac. On M5 Ultra, you get ~3400 tok/s for prompt processing and ~50 tok/s for token generation, quite usable state.