>Like any program, inference engines like vLLM or SGLang may contain exploitable bugs. Because the LLM controls the tokens passed to the inference engine, a malicious LLM could therefore emit a sequence of tokens that a poorly written inference engine mistakes for code or instructions to execute rather than data to return to the user.
They aren't talking about model providers. These are the tools you use with model weights locally (but you could set up remote infrastructure a la data center if you have the fundage).
They aren't talking about model providers. These are the tools you use with model weights locally (but you could set up remote infrastructure a la data center if you have the fundage).