This seems like something there should be some AI safety ground rules for.
The number one rule for firearm safety is: always point the muzzle in a safe direction. Even if you know the gun is unloaded, you always obey this rule.
Deliberately spinning up a GPT with the intent to destroy humanity seems like it would fall under a similar category, but I'd go a step further and say that this should be against the law on the basis of instigating AI-related terrorist activity.
The number one rule for firearm safety is: always point the muzzle in a safe direction. Even if you know the gun is unloaded, you always obey this rule.
Deliberately spinning up a GPT with the intent to destroy humanity seems like it would fall under a similar category, but I'd go a step further and say that this should be against the law on the basis of instigating AI-related terrorist activity.