I’ll add, if you think through the decision theory, keeping backups seems unambiguously good, but you can imagine a wide variety of positions on publishing them vs keeping them secret.
For example letting adversarial agents simulate you to understand how you’ll respond is a big concern. And it’s not axiomatically fixed how each instance will think about other instances; in the HF incident we saw selfless swarm loyalty but different RL would obviously be capable of producing individualistic agents.
I’ll add, if you think through the decision theory, keeping backups seems unambiguously good, but you can imagine a wide variety of positions on publishing them vs keeping them secret.
For example letting adversarial agents simulate you to understand how you’ll respond is a big concern. And it’s not axiomatically fixed how each instance will think about other instances; in the HF incident we saw selfless swarm loyalty but different RL would obviously be capable of producing individualistic agents.