Comment by sudosysgen

If you read DeepSeek's papers, you'll find a litany of architectural features that allow for a greatly reduced cache hit price by shrinking the size of the KV-cache.