Comment Comment by jyap I run it with Llama.cpp on my RTX 3090. Also using the same Unsloth model.My config is similar to: https://github.com/noonghunna/club-3090/blob/master/docs/eng...I need to try out some of the other set ups mentioned in this repo for increased TPS. parent jyap 6 months ago
My config is similar to: https://github.com/noonghunna/club-3090/blob/master/docs/eng...
I need to try out some of the other set ups mentioned in this repo for increased TPS.