Comment by sfifs

Very nice! I also saw you have a vllm branch and I validated it works on my system. There is bugfix which sent you a PR for to auto-discover served-model-name which vllm hard validates for.