Comment by bodegajed

1.5B models can run on CPU inference at around 12 tokens per second if I remember correctly.