Comment Comment by mohsen1 There is an open PR for VLLM to do this via DefussionGemmahttps://github.com/vllm-project/vllm/pull/57250 parent mohsen1 1 month ago
https://github.com/vllm-project/vllm/pull/57250