Comment Comment by binyu > Now at 40-50tok/s generation and ~2000 tok/sNot clear how you went from ~11-14 to ~40-50 tok/s. Is it by running the quant native model and adding a second Spark?Cheers parent binyu 4 months ago
Not clear how you went from ~11-14 to ~40-50 tok/s. Is it by running the quant native model and adding a second Spark?
Cheers