The Patriots’ Defense Is Good. That Doesn’t Mean It’s Going To Stay Good.

After four full weeks of play, the New England Patriots are one of only three undefeated teams left in the NFL,¹ and their defense is getting much of the credit for their early success. The Patriots lead the league in point differential at +95 and, on the strength of defensive backs Jonathan Jones and Jason McCourty, have been particularly good against the pass. New England has yet to allow a passing touchdown this year, making it one of just 10 teams since the AFL-NFL merger to accomplish that feat in the first four games of the season. And it’s not just the pass defense. The Patriots have surrendered just one touchdown in total on defense — a 1-yard run by Buffalo quarterback Josh Allen in Week 4.

Given these eye-popping stats, we would expect the Patriots to top the leaderboard in defensive expected points added per play. And that’s exactly what we find. New England stands atop the rankings with an EPA per play more than double that of the second-ranked defense, San Francisco’s.²

This list tracks well with our intuition of how defenses have performed and passes the “eye test.” New England, San Francisco, Chicago and Buffalo appear to be good defenses, and Miami, Oakland, Baltimore and Indianapolis appear to be bad. With a quarter of the season finished, we might even be tempted to draw some conclusions from this defensive leaderboard — it seems reasonable to expect the good defenses to stay good and the bad to remain bad. But does further evidence actually support making these inferences? Or are our eyes and the aforementioned data fooling us?

To find out, we took ESPN Stats & Information Group data from 2006 through 2018 and calculated the year-to-date EPA per play for each team prior to a given game. Then, we tested how well the team’s past performance predicted what the teams ultimately did in those games.

As an example, for a team’s fifth game of the season, we looked at the previous four weeks of games and calculated the EPA per play for each team.³ We then took those EPA averages and tested how well they predicted the EPA per play for that team in Game 5. If a defense was good in the first through fourth games, did it tend to stay good in the fifth? We conducted this analysis for each game of the season for both offense and defense, using the data from all previous games of the season. The results were surprising.

There’s a lot we don’t know about teams early in the season, but after nine or 10 games, we like to think that we have a decent idea about how each unit will perform in a given game, especially late in the season with the playoff picture in focus. This turns out to be partially true — for the offense, anyway. Our ability to predict a team’s offensive performance in an upcoming game does improve as the year goes on, peaking around Game 11. But, surprisingly, this isn’t true for the defense. We know roughly as much about a team’s upcoming defensive performance after one game of results as we do after 12 or 13.

In fairness, our ability to predict offensive performance is fairly modest,⁴ and by the very end of the season, we do see a small spike in our ability to predict defensive performance. But when comparing the predictive power of the two measures, the difference ranges from moderate to dramatic. Exactly why we can’t predict defense as well is hard to say with any confidence, but it might start with our inability to fully quantify the parts of a defender’s performance that are stable over time and reliably impact a game. In other words, it’s possible that we aren’t measuring the right thing by taking the aggregate performance of the offenses a team has faced and assigning the relative success or failure to the defense. Rather than capturing the strength of a team’s defensive unit, our metrics and stats may mostly be measuring a team’s strength of schedule.

Taking this view, the Patriots’ defensive performance is perhaps best attributed to the inept offensive performances of their opponents: the Steelers with an aging Ben Roethlisberger and newly without Antonio Brown; the tanking Miami Dolphins in a 43-0 blowout; a Jets team led by quarterback Luke Falk because of Sam Darnold’s bout with mononucleosis; and a narrow win over the Bills without a concussed Allen in the fourth quarter of Week 4.

Perhaps these performances aren’t as clear a harbinger of future defensive success as Patriots fans would like to believe. Conversely, it’s potentially good news for the defenses of Indianapolis, Baltimore and Oakland.⁵ We should probably be hesitant to read too much into their slow starts. Things can change quickly in the NFL, especially on defense.

Check out our latest NFL predictions.