brunaxLorax

Cool

It's hard to keep track of all those launches, I agree. I would separate 2 usages of LLMs: - LLM is integrated into a workflow/app for a classification or whatever job: scoring outputs (evals, KPIs) and choosing a fit between quality and cost is enough - LLM is used as a tool you interact with (like coding or research): here is where you have to know your brushes. As humans we get used very quickly by the model personality, the format of output, the way it "thinks" and interacts.

a French human doing its best to sound punchy in English :)

Yes. orchestrator > smart router

Sounds like an orchestrator for me ratter than a router

Agree

We tried different things: heuristic, TF-IDF and LLMs. I voluntarily didn't talk about the tech because it doesn't fix the problems mentioned by switching models on the fly.

100% agree. MoE is the perfect example: reducing FLOPs while keeping intelligence

All inference providers (labs and neoclouds like TogetherAI or Fireworks) are incentivized to be efficient to be more competitive. For example MoE reduces compute without reducing output quality. I would not be surprised if they end up implementing some kind of internal routing at some point.

Cheers, good luck!

Could be. In that case many routers are concerned as they all have more or less the same category buckets.