>Most agent loops reorder, rewrite, or inject fresh timestamps each turn
That's really surprising, since it'd defeat the whole point of KV caching. I mean I buy it considering how sloppily coded the harnesses seem to be, but this like obvious low hanging fruit.
I've also often wondered why LLMs aren't trained with a format of having a dedicated contextual system-instruction role at the _end_, which you could use to put context like current time or other misc stuff.
That's really surprising, since it'd defeat the whole point of KV caching. I mean I buy it considering how sloppily coded the harnesses seem to be, but this like obvious low hanging fruit.
I've also often wondered why LLMs aren't trained with a format of having a dedicated contextual system-instruction role at the _end_, which you could use to put context like current time or other misc stuff.