G github.com Show HN: Run Unsloth Dynamic GGUFs using Docker model runner We've been working with Docker to make running quantized models easier with Docker. This integration allows users to pull and run Unsloth-optimized models using the native Docker CLI.Key features:Uses docker model run syntax.Supports Dynamic GGUFs (handling quantization overhead).Cross-platform support.Example usage: docker model run ai/gpt-oss:20BHappy to answer questions about the implementation or the quantization methods used.https://docs.unsloth.ai/models/how-to-run-llms-with-docker 3 points ericcurtin 11 months ago 1 comments
Key features:
Uses docker model run syntax.
Supports Dynamic GGUFs (handling quantization overhead).
Cross-platform support.
Example usage: docker model run ai/gpt-oss:20B
Happy to answer questions about the implementation or the quantization methods used.
https://docs.unsloth.ai/models/how-to-run-llms-with-docker