Comment

Comment by guess__who

Some harness (cc) injects system messages in the middle, this kills the cache as the model provider moves the system to the top and concatenate them together. Use homemade proxy, you will get same 99% hot rate with even cc.
parent guess__who 3 months ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton