Comment Comment by WhitneyLand If speed is a metric for you, tokens required to solve a problem affects that metric.All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor. parent WhitneyLand 3 months ago
All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.
Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.