In particular: *all the knowledge that AI has of nuclear weapons is freely available on the internet*. It's not superhuman, and there's no secret sauce data. If you just study the same PDFs and blog posts it has, you will acquire the same abilities. I cannot imagine anyone with the intent and immense financial and political resources to actually build a weapon would say that some study time is the only thing stopping them from detonating a nuke.
It is pretty convenient for the labs to frame the conversation around this though, since it is easy to address, very few paying customers are rejected, and sounds scary (so surely the less scary sounding stuff must be solved right?)
It is pretty convenient for the labs to frame the conversation around this though, since it is easy to address, very few paying customers are rejected, and sounds scary (so surely the less scary sounding stuff must be solved right?)