Comment by mike31fr

With eval rate numbers:

- phi4: 12 tokens/s

- mistral-small: 9 tokens/s

On Nvidia RTX 4090 laptop:

- phi4: 36 tokens/s

- mistral-small: 16 tokens/s