> It's up to you, as the person running the harness through which the model gains any agency, to restrict the AI from performing malicious actions.
I believe that this line of thinking, while fine in theory, fails when faced with reality. As evidence of developers being overwhelmed by AI code I'd like to offer this post with almost 600 votes and 700+ comments titled "How to deal with shipping AI-written code you don't understand?" [1].
Once you have a malicious model inside your infrastructure generating thousands of lines of code, it doesn't matter how good the sandbox is - if the generated malicious code makes it through code review once, it is game over.
I believe that this line of thinking, while fine in theory, fails when faced with reality. As evidence of developers being overwhelmed by AI code I'd like to offer this post with almost 600 votes and 700+ comments titled "How to deal with shipping AI-written code you don't understand?" [1].
Once you have a malicious model inside your infrastructure generating thousands of lines of code, it doesn't matter how good the sandbox is - if the generated malicious code makes it through code review once, it is game over.
[1] https://news.ycombinator.com/item?id=50151036