Comment by binarydreams

One big issue is prompt caching - how are you (or any other router) solving for it? The moment caching is gone real agentic workloads would spike like crazy in terms of cost.