Comment

Comment by mrgaro

Because everybody has the same system prompt, the KV caching will make this a non-issue. The only cost is the reduced max context length.
parent mrgaro 2 months ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton