Comment by tcarambat1010

You can use LiteLLM _as the LLM provider_ which is just a relay to a ton of other models. For people who already have it set up connecting to x,y,z providers can keep that work and use it for inferencing as well