Comment by MDK8888

SageMode is an open source python library for deploying, scaling, and monitoring your LLMs at scale. It's designed to be highly flexible but also standardized. You can deploy any Huggingface Model that exists in version 4.26 or earlier as well as any PyTorch Model. If you find any issues with it, open an issue-I'd love to iron out any problems.