ericcurtin
Joined 1 year ago
Karma
4
Submissions
16
Member since
Oct 8
Recent submissions
Comment · 11 months ago
Comment · 11 months ago
I'm one of the devs, happy to answer any questions
Comment · 11 months ago
Will be around the next few hours
Comment · 1 year ago
Hi everyone, we're the maintainers.
We're rebooting the model-runner community and wanted to share what we've been up to and where we're headed.
When we first built this, the idea was simple: make running local models as easy as running containers. You get a consistent interface to download and run models from different backends (llama.cpp being a key one) and can even transport them using familiar OCI registries like Docker Hub.
Recently, we've invested a lot of effort into making it a true community project. A few highlights:
- The project is now a monorepo, making it much easier for new contributors to find their way around.
- We've added Vulkan support to open things up for AMD and other non-NVIDIA GPUs.
- We made sure we have day-0 support for the latest NVIDIA DGX hardware.
- There is a new "docker model run" UX: https://www.docker.com/blog/docker-model-run-prompt/
Comment · 1 year ago
Keep opening pull requests and issues, we need these things, you are right!
Comment · 1 year ago
Any new features you think we should add to further enhance your usage? Glad you find it useful
Comment · 1 year ago
Is there overlap with "Show HN: docker/model-runner – an open-source tool for local LLMs" posted today, we encapsulate AI workloads in containers using docker model runner. Some clear encapsulations advantages at a minimum. Could we work together?
Comment · 1 year ago
What did you like? Anything stand out?
Comment · 1 year ago
Thanks very much. It worked well for you? Which hardware? :) Any other feedback, keep it coming!
Comment · 1 year ago
Hi everyone, we're the maintainers.
We're rebooting the model-runner community and wanted to share what we've been up to and where we're headed.
When we first built this, the idea was simple: make running local models as easy as running containers. You get a consistent interface to download and run models from different backends (llama.cpp being a key one) and can even transport them using familiar OCI registries like Docker Hub.
Recently, we've invested a lot of effort into making it a true community project. A few highlights:
- The project is now a monorepo, making it much easier for new contributors to find their way around.
- We've added Vulkan support to open things up for AMD and other non-NVIDIA GPUs.
- We made sure we have day-0 support for the latest NVIDIA DGX hardware.
No cloudflare no problem
https://github.com/docker/model-runner