> Cache is cache, you can't get better cache than... cached...
No other provider is getting the cache hit rates deepseek does or the retention. They use a unique setup that no other provider is bothering to replicate.
> Quantization is not trash, there's a year of evidence that shows Q4 provides ~4% degradation and Q8 provides ~1% degradation, and you don't need that severely quantized to gain benefits in inference performance.
Lol. Quanting at all pretty much neuters a model, just look at how astra performs now compared to at launch.
No other provider is getting the cache hit rates deepseek does or the retention. They use a unique setup that no other provider is bothering to replicate.
> Quantization is not trash, there's a year of evidence that shows Q4 provides ~4% degradation and Q8 provides ~1% degradation, and you don't need that severely quantized to gain benefits in inference performance.
Lol. Quanting at all pretty much neuters a model, just look at how astra performs now compared to at launch.