With the NFL’s regular season concluding last Sunday, our Elo ratings have, in some ways, completed their broadest purpose: to track the league’s horse race by providing a weekly top-to-bottom ranking of all 32 teams. (You can see the final regular-season ratings below.) By dint of their historically great Elo rating, the Seattle Seahawks might still rank first in our post-Super Bowl rankings even if they lose in the divisional round of the playoffs — but that will be of cold comfort to fans in Seattle. As Al Davis might say, all that matters now is winning.
Even so, it will be useful to continue the ranking process in the postseason (despite almost two-thirds of the league being inactive until September). And going into the playoffs, the Seahawks are indeed the top-rated team in the NFL, by no small margin; Elo thinks they’d be favored by 2.9 points over the No. 2 New England Patriots on a neutral field. In fact, Seattle’s 1760 Elo rating is the fifth-highest mark any team has carried into the playoffs since the 1970 AFL-NFL merger, trailing only the 2007 and 2004 editions of the Patriots, the 1997 Green Bay Packers and Washington in 1983.
Having the NFL’s best Elo rating at the end of the regular season was a good formula for Seattle last season, as it ended up winning the Super Bowl from the pole position at the start of the playoffs. And up until the mid-to-late 1990s, regular-season Elo champs enjoyed a remarkable amount of postseason success, going on to win the Super Bowl 48 percent of the time between the 1970 and 1996 seasons.
But starting with the 1997 Packers (who lost a classic Super Bowl to the Denver Broncos), things have gone south for top-rated regular-season squads. They failed to win each of the next six Super Bowls, after never having gone more than four straight years without winning in the past. And, regular-season Elo champs — following a brief renaissance thanks to the top-rated Patriots’ back-to-back wins in 2003 and 2004 — went eight more seasons in a row without a title between 2006 and 2012. Since 1997, Elo’s top team at the end of the regular season has won just three Super Bowls in 17 tries, good for an 18 percent success rate.
It’s hard to say whether this is an anomaly or part of a larger trend toward more randomness in the NFL playoffs. (For what it’s worth, the average pre-playoff Elo of top-rated teams since 1997 was 1726; from 1970 to 1996, that number was 1711. So, it’s not as if the league is suddenly producing less dominant regular-season teams.) It’s also hard to say whether this is necessarily a bad thing for the football-watching public. (Would we really want a setup like this instead?) But on the eve of the postseason, it’s worth noting that being in the Seahawks’ position seems to be worth a lot less now than in the past.
Even so, our Elo simulations have Seattle as the clear Super Bowl favorites, with a 35 percent probability of winning it all — about 13 percentage points higher than that of the Patriots. (Meanwhile, betting markets consider the two teams co-favorites, at about 24 percent apiece after removing the vigorish.) The regular season’s final week widened that gap after New England lost at home against the Buffalo Bills — a fact that betrays one of the limitations of our Elo system.
With the Broncos losing to the Cincinnati Bengals on Dec. 22, the Patriots had already clinched the AFC’s top seed regardless of what happened against Buffalo. New England rested a number of starters and used quarterback/MVP candidate Tom Brady sparingly in its 17-9 loss. It was a result that really shouldn’t be held against the Patriots, but our Elo methodology doesn’t make exceptions for such games because it has no way to set them apart from more meaningful outcomes. Consequently, the Patriots lost 43 points of Elo rating last week (while Seattle gained five points for their victory over the St. Louis Rams), and the difference between Nos. 1 and 2 in our simulated Super Bowl probability increased by 7 percentage points.
A solution to this kind of quandary might be to tweak the Elo K-factor for each game using a leverage index-style metric, like the “swing” number FiveThirtyEight contributor Mike Beuoy uses to measure the playoff implications of a given matchup. Such a mechanic would elegantly put more emphasis on games in which teams had a great deal on the line, while simultaneously downplaying the importance of relatively meaningless games like Sunday’s Patriots-Bills tilt. Until then, however, Elo can only use the information it’s given, and that means we’re probably undervaluing the Patriots’ Super Bowl chances here.
Elo finished the regular season with a solid 129-108-4 record against the Vegas point spread — not a bad showing for a very simple system in its first year of use. But if history is any guide, it was also probably a huge fluke. Measuring its performance in the past against the spreads archived at Sunshine Forecast‘s website, 2014 represented Elo’s best season against the spread since 1989. So, while we were lucky to have begun using the system in what looks like a great year by its standards, please don’t use these numbers to place bets.
Even so, it’s always interesting to see where our algorithm differs from the lines set by sportsbooks in Vegas. This week, Elo and Vegas mostly see eye to eye, aside from the Carolina Panthers-Arizona Cardinals game Saturday afternoon. The books generally dislike the Panthers even more than they do the Cardinals, so you might expect Carolina to scrape by on home-field advantage alone (this is essentially why Elo has the Panthers as slight favorites). However, the Vegas consensus sees the Panthers favored by six over the Cardinals, who have lost both of their games since third-stringer Ryan Lindley took over as the team’s starting quarterback. So, this may be a case of Elo giving the Cardinals too much credit for their early-season games under injured QBs Carson Palmer and Drew Stanton.
Finally, while we’re on the subject of comparing Elo to Vegas, we’ll leave you with a comparison of Vegas’s preseason over/under win totals — namely, those released by the LVH Superbook — against the simulated win totals for each team using its preseason Elo rating (which were just a regressed-to-the-mean version of its rating from the end of 2013). There was a relatively high correlation (r = 0.82) between both sets of win totals, and both were (rightly) heavily regressed toward an 8-8 record for every team, but Vegas’s numbers had a slightly smaller root mean square error (RMSE) against actual win totals this season, narrowly beating Elo 2.4 to 2.5.
A bigger takeaway? The fact that things were so close between Vegas and something as simple as a regressed version of the previous year’s Elo ratings probably speaks to Brian Burke’s mantra that preseason predictions are relatively worthless across the board. Whenever we think we have football figured out, there’s always something to remind us how little we actually know.