dataversity

The goal is to make working with local models easier for developers, especially when experimenting with different models or sharing setups with others.

Would love to hear back from the community

Some of the features llmpm supports today:

• Install and run open-source models (almost all Hugging Face models)

• Package models with projects so environments are reproducible

• Serve models through an OpenAI compatible API

• Built-in chat UI for interacting with running models

• CLI workflow designed for developers

• Model benchmarking and rankings to help choose models Example:

llmpm serve llama3

This exposes an API endpoint similar to:

http://localhost:8080/v1/chat/completions

Hi folks, I’ve been working on llmpm, a CLI tool that makes open-source LLMs installable like packages. The idea came from the frustration of trying to run different models locally. Every model tends to have slightly different instructions, dependencies, or runtimes. I wanted something that worked more like npm or pip, install a model and run it immediately. With llmpm you can do things like:

llmpm install llama3

llmpm run llama3

Tried this out this morning. It’s a pretty powerful CLI tool for developers working with open-source LLMs.

Also, is there is way I can invoke the models or is there an API which your tool exposes?