sarthaksaxena
Joined 8 years ago
Karma
5
Submissions
16
Member since
Sep 25
Recent submissions
Comment · 8 months ago
Comment · 8 months ago
I am building npm for LLM Models, you can now install run and ship AI Models.
It’s a CLI tool that makes open-source LLMs installable like packages.
llmpm install llama3
llmpm run llama3
You can also package models with your projects so others can reproduce the same setup easily.
Website: https://llmpm.co
Comment · 8 months ago
I am building a command line package manager for AI models. It’ll make installing and running models locally incredibly easy.
Checkout: https://llmpm.co
Comment · 8 months ago
Yes indeed there is, run `llmpm serve <model_name>`, which will expose an API endpoint http://localhost:8080/v1/chat/completions & also host a chat UI where you can interact with the local running model https://localhost:8080/chat.
Follow the docs here: https://www.llmpm.co/docs
Pro tip for your use case: Checkout the `llmpm serve` section
Comment · 8 months ago
Yeah :)! Just run `llmpm init` to start packaging your models along with your code.
Comment · 8 months ago
https://llmpm.co
I have built npm for LLM models, which lets you install & run 10,000+ open sourced large language models within seconds. The idea is to make models installable like packages in your code:
llmpm install llama3
llmpm run llama3
You can also package large language models together with your code so projects can reproduce the same setup easily.
The idea came from the friction we kept seeing when trying to run models locally. Setting up models often involves downloading weights, configuring runtimes, and figuring out which models are actually good for a given task.
With llmpm the goal is to make models installable like packages:
llmpm install llama3
llmpm run llama3
We’ve also been working on the ability to package models with applications, so projects can declare model dependencies and reproduce the same setup easily.
Alongside the CLI, we’ve been experimenting with a model ranking and benchmarking tool on the website to help developers compare models across benchmarks and choose the right one before installing.
Check out rankings at: https://llmpm.co/rankings