Another question I have, is from a product perspective, how do you decide how much of the internals you show to users (like token usage).
I always thought that seems like (from a non technical users POV), like a implementation detail; and quite hard to map to actual usage. For example, one request might use lots of tokens, and another will barely use any; but it can be difficult for a non technical user to undertand the difference.
Another question I have, is from a product perspective, how do you decide how much of the internals you show to users (like token usage).
I always thought that seems like (from a non technical users POV), like a implementation detail; and quite hard to map to actual usage. For example, one request might use lots of tokens, and another will barely use any; but it can be difficult for a non technical user to undertand the difference.