40 Percent Of The NFL Is Real

It’s Week 8 of the 2015 season, which makes it a weird time on the NFL calendar. Almost half the schedule has been played, so it feels like we should have a decent grasp on which teams are good and bad. But this is the NFL, where we don’t know anything and we never will.

For instance, the New York Jets, who were not supposed to be very good according to our in-house rating system Elo, have played like one of the NFL’s best teams² so far. And the Indianapolis Colts, who were supposed to be one of the league’s best teams, have played like one of its worst.

(For those new to FiveThirtyEight, Elo is a system for ranking NFL teams that assigns them strength ratings based on how likely they are to win in the future. Each team’s rating is calculated using the scores, locations and opponents from its games, plus a prior based on how good Elo thought the team was the last time its rating was calculated.)

We know some of what we’ve seen thus far is real. But some is an illusion, the product of injuries, personnel changes and the random variance that can occur in the small sample of a half-season. So what’s a football fan to think? How much stock should be placed in what we think we’ve learned over the past two months, compared with what we thought we knew before any games were played?

We can get a sense for how much we’ve really learned by looking to past seasons³ and searching for the mixture of preseason Elo and season-to-date Simple Rating System⁴ that does the best job of predicting games from Week 8 on. Since 1978, 40 percent of what we think we’ve learned through seven weeks is real. That is, after the first seven weeks of the NFL season, roughly 40 percent of a team’s deviation from its initial outlook should be preserved going forward. The rest tends to fall away in due time, as teams see corrections in health, coaches make adjustments (and counter-adjustments), and teams see aberrant feats regress their way to the mean.

That fraction should be enough to change our opinions about the extreme outliers: The Colts are probably closer to average than to a top-five team,⁵ while the Jets are likely an above-average squad on the verge of being in the league’s top third. But at the same time, it can’t really help us with the nuances of teams that look like they belong in a tier adjacent to where they started. For instance, the Cincinnati Bengals and Carolina Panthers are both 6-0 despite being in the league’s middle third before the season. We can think of them as top 10 teams, but it’s difficult to know where in the top 10 they fall.

Despite our proximity to the season’s halfway marker, there’s still a lot left to be learned about the NFL this year. And that means plenty more surprising outcomes, both real and illusory.