Nice dashboard. Cost visibility is the first step — you cannot optimize what you cannot measure.
We have found that the natural next step after tracking is automated routing. Once you see that 70% of your frontier model calls could have been handled by something much cheaper, you want a system that does that automatically.
At Komilion we route each API call to the cheapest model that fits based on benchmarks. The combination of monitoring (like this) + routing gives you both visibility and automatic savings.
One thing I would suggest adding: cost-per-quality metrics. Raw cost is not the full picture — $0.80/M tokens that fails 20% of the time is more expensive than $2/M tokens that works reliably.
We have found that the natural next step after tracking is automated routing. Once you see that 70% of your frontier model calls could have been handled by something much cheaper, you want a system that does that automatically.
At Komilion we route each API call to the cheapest model that fits based on benchmarks. The combination of monitoring (like this) + routing gives you both visibility and automatic savings.
One thing I would suggest adding: cost-per-quality metrics. Raw cost is not the full picture — $0.80/M tokens that fails 20% of the time is more expensive than $2/M tokens that works reliably.