Comment by Lindon4290

Looks their performance is better than llama.cpp - https://news.ycombinator.com/item?id=37018989 - and scales to batches of prompts.