Yup, that's a fair question. It's not exactly running Elo. It's more of a Bradley-Terry fit across all games at once, mapped to the Elo scale (400 points = 10:1 odds). Game order doesn't change it. I also have each board played twice with both sides swapped to ensure fairness, so, for example, if one game has Kimi with 9 red words and Claude with 8 blue words, I make new agents of those same models where Claude now has those same 9 red words from Kimi and Kimi now has those 8 blue words from Claude.
I may add a "how ratings work" section on my site if more folks express interest.
I may add a "how ratings work" section on my site if more folks express interest.