There's no complete solutions, but there are mitigations.
- Limiting user input
- Decoupling the UI from the component that makes the call to an LLM
- Requiring output to be in a structured format and parsing it
- Not just doing a free-form text input/output; being a little more thoughtful about how an LLM can improve a product beyond a chatbot
Someone motivated enough can get through with all of these in place, but it's a lot harder than just going after all the low-effort chatbots people are slapping on their UIs. I don't see it as terribly different from anything else in computer security. Someone motivated enough will get through your systems, but that doesn't mean there aren't tools and practices you can employ.
- Limiting user input
- Decoupling the UI from the component that makes the call to an LLM
- Requiring output to be in a structured format and parsing it
- Not just doing a free-form text input/output; being a little more thoughtful about how an LLM can improve a product beyond a chatbot
Someone motivated enough can get through with all of these in place, but it's a lot harder than just going after all the low-effort chatbots people are slapping on their UIs. I don't see it as terribly different from anything else in computer security. Someone motivated enough will get through your systems, but that doesn't mean there aren't tools and practices you can employ.