Comment by himata4113

depends on how they're spliced, they have to be tuned for each configuration as there is no silver bullet. It also requires customizations to sglang/vllm to get it working in the first place so it's definitely pretty involved. However, it's still cheaper than obtaining more gpus.