Comment by srush

Nowhere near as neat as candle or ggml, but just released a 4-bit rust llama2 implementation with simd. Runs pretty fast.

https://github.com/srush/llama2.rs/