Anybody could fire up vLLM now to do exactly what you're talking about. And they do? There's dozens of places hosting DS4.1 Flash, at good speeds. So I'm not sure what your point is.
What this provides is the give contributors to vLLM, sglang, etc. a reference implementation of some of DeepSeek's own hard-won inference serving techniques.
No doubt we'll see patches to vLLM inspired by bits in here.
What this provides is the give contributors to vLLM, sglang, etc. a reference implementation of some of DeepSeek's own hard-won inference serving techniques.
No doubt we'll see patches to vLLM inspired by bits in here.