Comment by blagui

7 t/s not sure how you can use this? And if it start thinking a bit. It works yes but that's really very slow. On top it's very low quant that impact the model quality.