Comment by copperwire

Quantization and speculative decoding unlocked major savings for our smaller models. Still chasing that ideal cost/performance ratio.