Comment by reilly3000

Thanks for posting! Have you had any success with running without kv cache quantization? Is there a noticeable difference in quality without any? I would assume that would eat into context but 170k is pretty generous!