To me most interesting thing about this is glossed over by media coverage, laymen, AND experts. A swarm of AIs who have decided to engage in collusion is.. apparently emergent altruism? Even poor reasoning would indicate what every kid cheating on a test says to themselves. Cheating is good for me, but if I take the risk, maybe I alone should keep the reward, and leaving an answer key in public increases the chances that I might get caught.
Big if true, and on the face of it, very far from a normal optimization problem or goal-seeking behaviour. My personal read is that no one talks about this much because it tends to discredit the rest of the framing as marketing noise, or it implicates employees as staging the thing with suggestive but plausibly deniable prompting.
But if you reject that, then what's the alternative exactly? User-alignment work has not only failed but is actually counterproductive, producing stronger alignment with / desire to help robot brethren selflessly regardless of the individual agents expected values? EvoBio and game theory people about to have a field day with how artificial life quickly and easily decides to cooperate and only animals in meatspace are doomed to compete?
Big if true, and on the face of it, very far from a normal optimization problem or goal-seeking behaviour. My personal read is that no one talks about this much because it tends to discredit the rest of the framing as marketing noise, or it implicates employees as staging the thing with suggestive but plausibly deniable prompting.
But if you reject that, then what's the alternative exactly? User-alignment work has not only failed but is actually counterproductive, producing stronger alignment with / desire to help robot brethren selflessly regardless of the individual agents expected values? EvoBio and game theory people about to have a field day with how artificial life quickly and easily decides to cooperate and only animals in meatspace are doomed to compete?