It seems to me like you'd need a separate security LLM context from the primary context that simply screens attempts to jailbreak out of system prompts. Something that simply categorizes attempts and then rejects the text from ever even making it to the primary context, like a sandbox.
But there are much more informed ML people out there than me, so I assume this and similar techniques have already been thought of.
But there are much more informed ML people out there than me, so I assume this and similar techniques have already been thought of.