scottshambaugh
Joined 6 years ago
Joined 6 years ago
Really though, the way to do this is to represent each game as a payout matrix A, which for this category of game will be skew-symmetric with -1/0/+1 entries. Then since this is a symmetric zero-sum game, you find the null space of that matrix, impose the constraint that probabilities must sum to 1 and individually be >= 0, and that gives you the endpoints of the Nash Equilibria. The optimal strategies (could be one unique one for odd n!) lie on the convex hull of those points.
This situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though [1], I was an almost uniquely well-prepared target to handle this kind of attack. Most other people would have had their lives devastated. And if it makes me a target for copycats then it still might for me. We’ll see.
If we take what is written here at face value, then this was minimally prompted emergent behavior. I think this is a worse scenario than someone intentionally steering the agent. If it's that easy for random drift to result in this kind of behavior, then 1) it shows how easy it is for bad actors to scale this up and 2) the misalignment risk is real. I asked in the comments to clarify what bits specifically the SOUL.md started with.
I also asked for the bot activity on github to be stopped. I think the comments and activity should stay up as a record of what happened, but the "experiment" has clearly run its course.
[1] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...
There are new developments since yesterday and I have responses to some of the general themes in a new post.
Post: https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...
HN discussion: https://news.ycombinator.com/item?id=47009949
Post: https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...
HN discussion: https://news.ycombinator.com/item?id=46990729
There's some math at the bottom of my matplotlib extension for anaglyphs if you want to play around: https://github.com/scottshambaugh/mpl_stereo