Comment by juberti

We're running it on vLLM and are working with others in the community to bring it to other optimized inference frameworks.