Nice approach to the per-key cost cap problem. We built something similar for tracking AI spend across providers - the "one forgotten loop" scenario is real and expensive.
The JSON-on-disk pattern works surprisingly well for this scale. We found the key insight is making costs visible in real-time rather than waiting for end-of-month bills. Even just seeing token counts per request changes behavior.
Curious if you've hit the CLI latency wall yet with concurrent users - that 3-8s overhead compounds fast with a team.
The JSON-on-disk pattern works surprisingly well for this scale. We found the key insight is making costs visible in real-time rather than waiting for end-of-month bills. Even just seeing token counts per request changes behavior.
Curious if you've hit the CLI latency wall yet with concurrent users - that 3-8s overhead compounds fast with a team.