This suddenly reminded me of how I was “cheating”.
Before the game starts each of the ten players gets to pick a distinct hero for themselves out of a pool of about 120 choices. This is over 10^20 distinct combinations! Each hero has some unique capabilities that combo with allies or counter enemy heroes.
I tried to train a “hero recommender” based on tens of millions of games.
It turned out that this is obscenely difficult because even the best AI training algorithms struggle with such highly noisy labels. A good hero combo might shift win rates by some positive percentage but have a single sample data point, which is a loss because of one stupid kid in the team throwing the game.
You also can’t naively simplify the problem into 2-hero or 3-hero combinations because this misses the “total team composition” metrics.
I found some research papers that were just a few months old at the time which covered this corner of the AI training space. Their conclusion were: “We don’t know either but it’s an interesting problem!”
Before the game starts each of the ten players gets to pick a distinct hero for themselves out of a pool of about 120 choices. This is over 10^20 distinct combinations! Each hero has some unique capabilities that combo with allies or counter enemy heroes.
I tried to train a “hero recommender” based on tens of millions of games.
It turned out that this is obscenely difficult because even the best AI training algorithms struggle with such highly noisy labels. A good hero combo might shift win rates by some positive percentage but have a single sample data point, which is a loss because of one stupid kid in the team throwing the game.
You also can’t naively simplify the problem into 2-hero or 3-hero combinations because this misses the “total team composition” metrics.
I found some research papers that were just a few months old at the time which covered this corner of the AI training space. Their conclusion were: “We don’t know either but it’s an interesting problem!”