Comment by Catloafdev

Neat! But, what do you do with a 0.5tk/s LLM?

Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?