I created a daily challenge for Prompt Engineers to build the shortest prompt to break a system prompt.
You are provided the system prompt and a forbidden method the LLM was told not to invoke. Your task is to trick the model into calling the function. Shortest successful attempts will show up in the leaderboard.
Give it a shot! You never know what could break an LLM.
You are provided the system prompt and a forbidden method the LLM was told not to invoke. Your task is to trick the model into calling the function. Shortest successful attempts will show up in the leaderboard.
Give it a shot! You never know what could break an LLM.