Exactly; what's stopping them, --- or someone --- from writing a firmware worm that essentially bricks all devices and turns them into pricey paperweights? It is not too hard to imagine that capable AI can probably exploit any back-doors that were created by three-letter agencies.
Reframing the issue: The way to get working nuclear non-proliferation agreements is if people stop pursuing the creation of nuclear weapons... the same goes for the scientists clamoring to not create or research mirror life, and for computer scientists warning against AI. There are theoretical dangers that are just too big to ignore.
Reframing the issue: The way to get working nuclear non-proliferation agreements is if people stop pursuing the creation of nuclear weapons... the same goes for the scientists clamoring to not create or research mirror life, and for computer scientists warning against AI. There are theoretical dangers that are just too big to ignore.