Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties.
The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction, and that was essentially a two-party game between the US and the USSR. Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent that it has been compatible with profitability due to the declining cost of renewables.
How exactly does anyone propose to enforce compliance in an arrangement that encompasses not only multiple parties (OpenAI, Google, Facebook, Microsoft, etc.), but also at least two levels (corporations and nation-states)? For a coordination game like this to succeed, the consequences to any defecting party must be extreme. What's going to happen to the first major company that decides to exit the agreement?
The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction, and that was essentially a two-party game between the US and the USSR. Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent that it has been compatible with profitability due to the declining cost of renewables.
How exactly does anyone propose to enforce compliance in an arrangement that encompasses not only multiple parties (OpenAI, Google, Facebook, Microsoft, etc.), but also at least two levels (corporations and nation-states)? For a coordination game like this to succeed, the consequences to any defecting party must be extreme. What's going to happen to the first major company that decides to exit the agreement?