Comment by pornel

I don't like that it throws away the whole KV cache when compacting. It costs a cache miss of the whole conversation length, and that's a waste of time and money.

LLMs are perfectly capable of summarising the conversation without a new system prompt.