Comment by givenkiban1

Use ollama to manage and download models locally.

there's a python ollama module that can allow you to run inference with these models.

that's the first part.