On Monday we released FiveThirtyEight’s World Cup predictions, which are based on an algorithm I created with ESPN in 2010, the Soccer Power Index (SPI). Our model has Brazil as the clear frontrunner — partly because of the substantial advantage to the host nation in the World Cup, and partly because SPI rates Brazil as the best team in the world to begin with.
But not everyone agrees with the latter assessment. So we thought we’d compare SPI against three other systems that rank national soccer teams:
- First, the Elo ratings, maintained at www.eloratings.net. The Elo ratings were originally developed for chess but have since been adopted to soccer and other competitions. The system is transparent, simple and relatively elegant. Elo ratings tend to be highly correlated with SPI.
- Next, the FIFA/Coca Cola World Ranking. I’m not a big fan of this system. Like the old BCS formula for American football, it’s full of arbitrary parameters and coefficients that are constantly changing in response to controversy, but which only wind up generating new problems; FIFA had Brazil ranked as just the 22nd-best team in the world as of a year ago, for instance. Unfortunately, these ratings have a lot of influence in soccer, in part because FIFA uses them to place teams into groups in the World Cup and other tournaments.
- Finally, the cumulative market value of each nation’s players as estimated by the German website Transfermarkt (www.transfermarkt.com). Transfermarkt does not estimate the team figures directly; it only lists the value of individual players. But we’ve seen several websites roll them up to the team level (the version we’re using in this article is from Time). I’m intrigued by these — SPI is also partly based on evaluating the skill of individual players — but there are two potential problems: First, a team may be more or less than the sum of its parts, and second, Transfermarkt’s method for determining player values is a little opaque.
All of these ratings systems are on different scales — Transfermarkt’s figures are in dollars (or euros), for instance, while Elo ratings are denominated in points. So, we’ve standardized them to make them comparable to one another. The numbers listed below are z-scores. A score of zero represents an average World Cup team according to the system in question. Positive scores are associated with above-average teams. For instance, a score of +1 represents a team that’s one standard deviation better than the mean. Negative scores are associated with below-average teams. (I also made one other adjustment to correct for the fact that SPI ratings are slightly nonlinear with respect to team quality, though it makes very little difference for this analysis.)
Here’s how the systems compare:
SPI and Elo both put Brazil on top. Our system gives it a 45 percent chance of winning the World Cup, while an analysis by Goldman Sachs based on the Elo ratings puts Brazil’s chances at 49 percent. Transfermarkt and FIFA rate Spain as the best team in the world. (This does not necessarily imply that Spain would be the favorite to win the World Cup according to these systems — Brazil’s home-field advantage and easier group might be enough to increase its chances.)
SPI is more pessimistic about the U.S. than both FIFA and Elo — but not quite as pessimistic as Transfermarkt. FIFA is the most bullish on the men’s national team (which should probably make U.S. fans bearish).
SPI has the lowest rating of any system for the following teams: Spain, England, Portugal, Italy, Switzerland, Croatia and Algeria. In other words, SPI is considerably more bearish on Europe than the other systems, particularly as compared to FIFA and Transfermarkt. It’s bullish, by contrast, on South America. Of the four systems, SPI has the best rating for Argentina, Colombia, Chile and Ecuador, along with Bosnia-Herzegovina, Ghana, Nigeria and Ivory Coast.
Aside from coming to different assessments about the comparative strength of the continents, the systems differ mostly based on whether they seek to evaluate individual players or only the teams’ records in national games. For example, SPI and Transfermarkt, which include a player-rating component, are considerably higher on France, a team that has a lot of talent but inconsistent results in international play.