Comment by bluegatty

"nd then it gets taken out of the conversation so that it doesn't get sent to the agent the next turn (no context bloat). "

Uh - dude - this means you're paying 10x in token costs because there's no caching.

If you 're-write token history' then you can't cache tokens.

It means for any reasonably long conversation, the llm has to reprocess the entire history as preflow on every prompt.

Are you sure you're really doing what you say you're dong, and how is it not blowing up your budget?