LLMs are quite good these days at not being fooled by adversarial prompting, especially basics like this
But yeah it'll never be perfect. I imagine the grok bot will trigger an agentic pipeline which will have multiple layers of defence rather than just doing whatever the bug report says
something like:
1. is this bug report malicious?
2. can this be reproduced?
3. is this a real bug or intended behaviour?
4. fix the bug
5. review the change (or multiple review agents focused on different factors like correctness, performance, security)
crafting an adversarial prompt to defeat all the layers without an agent realising its building a security vulnerability is probably quite a challenge
But yeah it'll never be perfect. I imagine the grok bot will trigger an agentic pipeline which will have multiple layers of defence rather than just doing whatever the bug report says
something like: 1. is this bug report malicious? 2. can this be reproduced? 3. is this a real bug or intended behaviour? 4. fix the bug 5. review the change (or multiple review agents focused on different factors like correctness, performance, security)
crafting an adversarial prompt to defeat all the layers without an agent realising its building a security vulnerability is probably quite a challenge