Comment Comment by krychu Self-plug. Here’s a fork of the original llama 2 code adapted to run on the CPU or MPS (M1/M2 GPU) if available:https://github.com/krychu/llamaIt runs with the original weights, and gets you to ~4 tokens/sec on MacBook Pro M1 with the 7B model. parent krychu 3 years ago
https://github.com/krychu/llama
It runs with the original weights, and gets you to ~4 tokens/sec on MacBook Pro M1 with the 7B model.