I started building it out that way but found it very challenging to create parity between the models. E.g. they have different tools, system prompts, interruption semantics, cost tracking etc. The spirit of the product is decoupling the LLM from the UI, so we went with a custom loop / tools that can perform decently across all models.
I started building it out that way but found it very challenging to create parity between the models. E.g. they have different tools, system prompts, interruption semantics, cost tracking etc. The spirit of the product is decoupling the LLM from the UI, so we went with a custom loop / tools that can perform decently across all models.