This is interesting! Sort of a super mixture of experts model. What's the latency penalty paid with your router in the middle?
The pattern I often see is companies prototyping on the most expensive models, then testing smaller/faster/cheaper models to determine what is actually required for production. For which contexts and products do you foresee your approach being superior?
Given you're just passing along inference costs from backend providers and aren't taking margin, what's your long-term plan for profitability?
The pattern I often see is companies prototyping on the most expensive models, then testing smaller/faster/cheaper models to determine what is actually required for production. For which contexts and products do you foresee your approach being superior?
Given you're just passing along inference costs from backend providers and aren't taking margin, what's your long-term plan for profitability?