I think we are about to enter a world where security compromises occur at least an order of magnitude more often due to AI. The fact that AI is sometimes wrong benefits the attacker because they can just automate attacking with agents until they breach a system and the failed attempts don't matter.
Nicholas Carlini has a good talk about the risks on his homepage https://nicholas.carlini.com/ (can't link to the talk directly for some reason)
Nicholas Carlini has a good talk about the risks on his homepage https://nicholas.carlini.com/ (can't link to the talk directly for some reason)