lum

Thanks! Regarding the Kelly Criterion—that's a great idea, I'll look into it!

I think we were off to a good start after Morocco vs Iran (for which we had significantly different predictions and got it right), but lost quite a bit with Danemark vs Peru (we had Peru as favorite).

Yes, exactly!

Thanks a lot for the comments!

- about friendly games: agreed, in the future we'd like to downweight them (as well as downweight older games over newer ones)

- about substitutions: at training time, we look at the number of minutes each player has spent on the field. At test time, we assume there will be no substitution. Could certainly be improved :-)

- about teams that are already qualified: true, our model does not encode all these contextual factors, and arguably they are very important for certain games. If the lineup changes (e.g. star players resting) it does impact the prediction.

Yes, for now we only predict win / lose. It is fairly reasonable to assume that draws are more likely to arise if the winning probabilities are close to 50/50 (and less likely if there is a big difference) - but you are also right that there are some contextual factors that influence this.

You know the saying, "all models are wrong, but some are useful" - our model is clearly too simple to encode many of these complex patterns, but we believe that's alright!

You're right. We took Wales' last lineup, but it will very probably be a different lineup against Slovakia. We'll update the prediction shortly before the game, when we know who's going to play.

We did some preliminary experiments in this direction. Basically, we tried to do a regression on the score difference instead of using only binary outcomes. In our experiments it didn't improve the predictive accuracy - but there are many more things to try. It does feel a bit wasteful not to take score data into account.

Nice that you had some similar ideas :-)

Excellent point - this one is definitely on our todo list. There are several simple extensions of the Elo model that take draws into consideration (i.e., give a non-zero probability to draws), for example the Rao-Kupper model. There are only minimal changes needed w.r.t. the original model, but still we didn't manage to make the changes in time for this version of the site.

In short: at its core, the "Elo assumption" postulates that every team can be represented by a real number (that can be interpreted as the strength of the team), and that the probability of the outcome depends on on the difference in strength. In the vanilla Elo model, the outcome is binary, but it's easy to make it ternary.

We scraped it from soccerway.com.

You are correct about Portugal, not exactly the squad that will likely play against Iceland. We used the starting lineup of the last official game (I believe in this case it was Portugal vs England on June 2nd), which did not include e.g. Cristiano Ronaldo.

The actual lineup does impact the predictions - we will update the lineups before the start of each game, when the lineup is anounced.

In a future version, we would like to make it possible for visitors to change the players in the team, and automatically update the prediction.

For those who are interested in the details: we model the strength of teams using the well-know Elo model (used e.g., for official chess ratings).

We innovated on two aspects of the traditional Elo model, in which every team has an independent parameter:

1) We actually model the strength of players instead of teams. This makes it possible to learn from games that were played in championships between clubs, and transfer this knowledge to games between countries

2) There is a not-so-widely-known connection between Elo-type comparison models and Gaussian process classification. We leverage this, and get a full posterior distribution for each team's strength. (Information on the uncertainty of our estimates helps a lot in coming up with sensible predictions)

If anyone wants to know more (explanations on the web page are very superficial at the moment), please drop me a line!