For the past three years, FiveThirtyEight has assessed every NFL game and published forecasts showing the chance each team has to win based on Elo ratings, a simple system that estimates each team’s skill level using only the final scores and locations of each game. This year, we’re giving you the chance to play along and see if you can forecast NFL games better than Elo can.
Every week of the NFL season — including the playoffs — you can decide whether you agree or disagree with FiveThirtyEight’s forecast and then make your own using our interactive game. After each NFL game finishes, you’ll either gain or lose points based on whether you picked the winning team and how confident you were that it would prevail. The higher a win probability you assign a team, the more points you can earn — but also the more you can lose.
Our game keeps track of your point total throughout the season — you just have to sign in with either Facebook or your Google account to make your selections.44 As the season progresses, you’ll be able to see how your forecasts compare with FiveThirtyEight’s Elo forecasts and how your forecasts stack up against everybody else playing the game.
For NFL fans used to thinking about point spreads, forecasting games based on probabilities might not be intuitive — but at FiveThirtyEight, we like to think probabilistically. In our game, a team’s margin of victory doesn’t matter; all that matters is who wins. Take the Week 1 matchup between the Steelers and the Browns. It may seem like Pittsburgh has a 100 percent chance of beating Cleveland after the Browns went 1-15 last season and their No. 1 overall draft pick, defensive end Myles Garrett, sprained his ankle. But upsets happen; the Browns could eek out a win at home. FiveThirtyEight gives the Steelers a 76 percent chance of winning the game. Think that’s way too low? Make your forecast. But be careful: Overconfidence is penalized harshly in our scoring system.
Let’s look at an example of how the scoring works. Suppose you think that FiveThirtyEight is underestimating Pittsburgh in this one. You think the Steelers have a 91 percent chance of winning, and you make your forecast accordingly.
Our scoring system is based on Brier scores, which evaluate the accuracy of probabilistic forecasts and reward good forecasts while punishing those that overshoot. (We’ve adapted the system for our game essentially by rescaling the scores; the result is that players in our game can earn between -75 and +25 points, with zero points going to 50/50 predictions.) In the example above, you were very confident in Pittsburgh so a Steelers win would yield almost the maximum possible allotment (24.2 out of 25 points). But if they are upset by the Browns, you would lose 57.8 points — nearly as bad as you can do.45
The FiveThirtyEight model — with its more conservative 76 percent forecast — would gain just 19.2 points if the Steelers won but lose only 32.8 points if the Browns won.46
A few more details:
- You can make and adjust your forecast up until a game starts.
- Tie games will not be scored.
- Playoff games will count double toward your point total.
- Forecasts can’t be more precise than whole percentages, and points gained/lost for each game are rounded to one decimal place.
- You’ll receive 0 points for any game you skip, just as if you’d made a 50/50 forecast. So while forgetting to make a pick here and there won’t put you totally out of the running, we’d definitely encourage you to sign in every week. The NFL’s Sept. 7 opening game between the Patriots and the Chiefs wasn’t included as part of the contest, but future Thursday, Saturday and Monday games will be.
So how should you approach the game? Here are several possibilities:
- You can just pick the teams you think will win and crank the slider to 100 percent for every game. You probably won’t do very well, though. If you went back in time to last season and just chose 100 percent for every favored team, you’d end up with negative 2,800.0 points — not very good. Meanwhile, FiveThirtyEight’s Elo model would have ended up with (positive) 923.8 points.
- You can try to understand the limitations of FiveThirtyEight’s Elo model and make adjustments accordingly. For example, FiveThirtyEight’s model doesn’t take injuries into account. That means it doesn’t know that Indianapolis Colts quarterback Andrew Luck is dealing with a shoulder injury and has been ruled out for Week 1. FiveThirtyEight is still giving the Colts a 57 percent chance of winning on the road against the Los Angeles Rams — but perhaps you think that’s too high, considering Luck’s injury. So you adjust the forecast down to 52 percent or choose to favor the Rams. Other factors that Elo doesn’t consider: the weather, draft picks, trades or resting players before the playoffs.
- You could look at other models, or Vegas odds, and base your forecasts on these. Historically, our Elo model has held up pretty well against other models and experts, but a composite model will often do better than most individual models.
- You could build your own model! We’re putting some code and data on GitHub that shows exactly how our Elo ratings and win probabilities are calculated and how the scoring system works. You could use that to create a model and test it on previous seasons.
Good luck, and have fun.
