If you're looking to run Llama 2 locally via a CLI or REST API (vs the web ui this article highlights), there's an open-source project some folks and I have been working on over the last few weeks: https://github.com/jmorganca/ollama
More projects in this space:
- llama.cpp which is a fast, low level runner (with bindings in several languages)
- llm by Simon Willison which supports different backends and has a really elegant CLI interface
More projects in this space:
- llama.cpp which is a fast, low level runner (with bindings in several languages)
- llm by Simon Willison which supports different backends and has a really elegant CLI interface
- The MLC.ai and Apache TVM projects
Previous discussion on HN that might be helpful from an article by the great folks at replicate: https://news.ycombinator.com/item?id=36865495