Ollama serves models on an OpenAI compatible API directly on the inference machine, which most open source harness software will connect to. The inference machine has a name like inferencemachine.local on my home network so the Ollama URL looks like http://inference machine:11434/v1 to connect to any model I've downloaded.
Ollama serves models on an OpenAI compatible API directly on the inference machine, which most open source harness software will connect to. The inference machine has a name like inferencemachine.local on my home network so the Ollama URL looks like http://inference machine:11434/v1 to connect to any model I've downloaded.