The lack of acknowledgement of the threat of prompt injection in this new initiative to help people "responsibly deploy generative AI models and experiences" is baffling to me.
I found a single reference to it in the 27 page Responsible Use Guide which incorrectly described it as "attempts to circumvent content restrictions"!
"CyberSecEval: A benchmark for evaluating the cybersecurity risks of large language models" sounds promising... but no, it only addresses the risk of code generating models producing insecure code, and the risk of attackers using LLMs to help them create new attacks.
And "Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations" is only concerned with spotting toxic content (in English) across several categories - though I'm glad they didn't try to release a model that detects prompt injection since I remain very skeptical of that approach.
I'm certain prompt injection is the single biggest challenge we need to overcome in order to responsibly deploy a wide range of applications built on top of LLMs - the "personal AI assistant" is the best example, since prompt injection means that any time an LLM has access to both private data and untrusted inputs (like emails it has to summarize) there is a risk of something going wrong: https://simonwillison.net/2023/May/2/prompt-injection-explai...
I guess saying "if you're hoping for a fix for prompt injection we haven't got one yet, sorry about that" isn't a great message to include in your AI safety announcement, but it feels like Meta AI are currently hiding the single biggest security threat to LLM systems under a rug.
I found a single reference to it in the 27 page Responsible Use Guide which incorrectly described it as "attempts to circumvent content restrictions"!
"CyberSecEval: A benchmark for evaluating the cybersecurity risks of large language models" sounds promising... but no, it only addresses the risk of code generating models producing insecure code, and the risk of attackers using LLMs to help them create new attacks.
And "Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations" is only concerned with spotting toxic content (in English) across several categories - though I'm glad they didn't try to release a model that detects prompt injection since I remain very skeptical of that approach.
I'm certain prompt injection is the single biggest challenge we need to overcome in order to responsibly deploy a wide range of applications built on top of LLMs - the "personal AI assistant" is the best example, since prompt injection means that any time an LLM has access to both private data and untrusted inputs (like emails it has to summarize) there is a risk of something going wrong: https://simonwillison.net/2023/May/2/prompt-injection-explai...
I guess saying "if you're hoping for a fix for prompt injection we haven't got one yet, sorry about that" isn't a great message to include in your AI safety announcement, but it feels like Meta AI are currently hiding the single biggest security threat to LLM systems under a rug.