You're failing to distinguish between token count spent and context window used.
You're correct that the former is not a concern for Anthropic, but it's also not the problem I'm pointing out.
The actual is not token count, it's context window. Your context window is finite, and every token you add to it reduces the model's accuracy during inference.
You're correct that the former is not a concern for Anthropic, but it's also not the problem I'm pointing out.
The actual is not token count, it's context window. Your context window is finite, and every token you add to it reduces the model's accuracy during inference.