Comment by carom

Is this just for local inference? We have a server that the team hits via an OpenAI-compatible APIs but serving local models.