A strong passing attack is crucial to NFL success. Seven of the top 10 teams this season in passing EPA per play are competing in the divisional round this weekend. That makes stopping the pass equally critical, of course. But quantifying pass defense in football has proven difficult. Our current metrics for measuring defensive performance just aren’t very good.
As has been noted before, much of the trouble comes from the fact that it’s hard to count the absence of a thing. For a cornerback or a safety in pass coverage, no news is good news: If they’ve done their job, the opposing QB often throws the ball to a more inviting target, throws the ball away or doesn’t throw at all. Pass breakups and interceptions are helpful and certainly signs of skill, but they’re rare. Starting defensive backs typically play more than 500 coverage snaps a year, yet in 2019, only 13 DBs had 10 or more pass breakups. Interceptions are even more infrequent, averaging 0.80 per game in 2019 across all positions. New England cornerback Stephon Gilmore tied for the league lead with just six INTs.
With the advent of player-tracking data, however, things are beginning to change. Michael Chiang has shown us where teams aren’t targeting opposing offenses, and now the NFL’s NextGen Stats team has released a metric that attempts to better quantify a defender’s performance when he is targeted.
Named (somewhat confusingly) Opponent Receptions Plus/Minus, what the metric actually does is estimate the number of incompletions over expected that a team creates; then, it assigns credit for the incompletion to the nearest defender on a play. The metric builds on completion probability4 and allows us to gather a larger sample of performance data on each defender. Couple this metric with the pass coverage classification system from ESPN analyst Brian Burke, and we can get a good sense of how well it does at identifying defender performance.
Stephon Gilmore led the league in forced incompletions
Top 10 and bottom 10 NFL defenders in man coverage by Opponent Receptions Plus/Minus, which measures incompletions forced over expected, 2019 regular season
| Rk | player | team | Pos | G | Targets | Opp. Recs. Plus/Minus |
|---|---|---|---|---|---|---|
| 1 | Stephon Gilmore | New England | CB | 16 | 77 | 6.16 |
| 2 | Malcolm Jenkins | Philadelphia | S | 16 | 19 | 5.99 |
| 3 | Bradley McDougald | Seattle | S | 15 | 20 | 4.90 |
| 4 | J.C. Jackson | New England | CB | 16 | 48 | 4.53 |
| 5 | Eric Kendricks | Minnesota | LB | 15 | 24 | 3.72 |
| 6 | Jimmy Smith | Baltimore | CB | 9 | 37 | 3.34 |
| 7 | Darqueze Dennard | Cincinnati | CB | 9 | 25 | 3.28 |
| 8 | Marshon Lattimore | New Orleans | CB | 14 | 45 | 3.11 |
| 9 | Carlton Davis | Tampa Bay | CB | 14 | 66 | 2.83 |
| 10 | Marcus Peters | Baltimore | CB | 16 | 56 | 2.70 |
| … | … | |||||
| 272 | Terrell Edmunds | Pittsburgh | S | 16 | 29 | -4.68 |
| 273 | Fabian Moreau | Washington | CB | 12 | 22 | -4.75 |
| 274 | James Bradberry | Carolina | CB | 15 | 34 | -4.77 |
| 275 | A.J. Bouye | Jacksonville | CB | 14 | 47 | -4.77 |
| 276 | Pierre Desir | Indianapolis | CB | 12 | 45 | -5.12 |
| 277 | Rashaan Melvin | Detroit | CB | 13 | 61 | -5.25 |
| 278 | Kevin King | Green Bay | CB | 15 | 51 | -5.43 |
| 279 | Adoree’ Jackson | Tennessee | CB | 11 | 28 | -5.76 |
| 280 | Vernon Hargreaves III | Houston | CB | 15 | 58 | -7.74 |
| 281 | Xavier Rhodes | Minnesota | CB | 15 | 45 | -10.52 |
For man coverage, the “face validity” of Opponent Receptions Plus/Minus is high. Gilmore is generally considered to have been one of the best corners in the league this season. He was named a first-team All-Pro and is a candidate for Defensive Player of the Year. According to the metric, he was the top man-coverage defender in the league, preventing approximately six receptions over expected on 77 targets. Moreover, Gilmore’s Plus/Minus game logs this season track fairly well with Pro Football Focus’s coverage grade,5 giving us further confidence that it’s capturing the right things.
The bottom of the defender list seems to make sense as well. I think most would agree that Xavier Rhodes has been about as useless as snake mittens this year, and PFF gave him a 46.4 grade for the season.
Perhaps more promising for Plus/Minus is that eight of the top 10 defenders identified are from teams who made the playoffs. Obviously, successful teams can and do overcome poor individual performances, as we see with Rhodes and the Vikings. But as a descriptive metric for man coverage, Opponent Receptions Plus/Minus appears solid.
But what about zone? Since Plus/Minus measures a defender’s proximity to a receiver on targeted passes, we might expect it to do better with man coverage versus zone. And that’s just what we find.
The zone coverage leaderboard is less convincing
Top 10 and bottom 10 NFL defenders in zone coverage by Opponent Receptions Plus/Minus, which measures incompletions forced over expected, 2019 regular season
| Rk | player | team | Pos | G | Targets | Opp. Recs. Plus/Minus |
|---|---|---|---|---|---|---|
| 1 | Denzel Ward | Cleveland | CB | 12 | 30 | 5.81 |
| 2 | Ross Cockrell | Carolina | CB | 14 | 46 | 5.17 |
| 3 | Eddie Jackson | Chicago | S | 16 | 19 | 4.97 |
| 4 | James Bradberry | Carolina | CB | 15 | 55 | 3.84 |
| 5 | Shaquill Griffin | Seattle | CB | 14 | 30 | 3.57 |
| 6 | Charvarius Ward | Kansas City | CB | 16 | 21 | 3.19 |
| 7 | Mike Hilton | Pittsburgh | CB | 16 | 32 | 3.07 |
| 8 | Sean Murphy-Bunting | Tampa Bay | CB | 16 | 25 | 2.96 |
| 9 | Jamal Adams | N.Y. Jets | S | 14 | 18 | 2.95 |
| 10 | Troy Hill | L.A. Rams | CB | 14 | 18 | 2.94 |
| … | … | |||||
| 272 | Kyle Fuller | Chicago | CB | 16 | 45 | -4.70 |
| 273 | Jordan Hicks | Arizona | LB | 16 | 31 | -4.75 |
| 274 | Prince Amukamara | Chicago | CB | 15 | 31 | -5.23 |
| 275 | Tahir Whitehead | Oakland | LB | 16 | 28 | -5.42 |
| 276 | Bobby Wagner | Seattle | LB | 16 | 56 | -5.59 |
| 277 | Desmond King | L.A. Chargers | CB | 15 | 37 | -5.97 |
| 278 | Trae Waynes | Minnesota | CB | 14 | 39 | -5.97 |
| 279 | Rock Ya-Sin | Indianapolis | CB | 15 | 29 | -5.98 |
| 280 | De’Vondre Campbell | Atlanta | LB | 16 | 40 | -7.08 |
| 281 | Byron Murphy | Arizona | CB | 16 | 39 | -8.59 |
The face validity of Opponent Receptions Plus/Minus is murkier in zone coverage. Denzel Ward is a former first round pick but has struggled at times this season, and Ross Cockrell has earned a grade of 61.3 in coverage from PFF this season.6 More worrisome, few of the players identified as positive contributors are on playoff teams. Zone accounted for roughly half of NFL coverage plays in 2019, so we would expect teams with elite zone defenders to have success.
The holy grail for a defensive metric is stability. Opponent Receptions Plus/Minus shows promise as a descriptive man coverage metric, but are players who are forcing incompletions now likely to continue their performance in the future?
Unfortunately for the metric — and for football analytics in general — the answer is no. How a player does in one season in Opponent Receptions Plus/Minus holds no predictive value for how he will do in the following season. This is also true for split seasons; how a player does in Plus/Minus for the first eight games of the year has no relation to his performance in the final eight games.7
Gilmore is an interesting example of the overall instability at DB. In 2016 — his last year in Buffalo — his Opponent Receptions Plus/Minus for all coverages was -0.6. In his first year with New England, his Plus/Minus dipped to -2.76, but he took a leap in 2018 to 2.42. In 2019 his Plus/Minus was 7.97 — second-best in the league behind Malcolm Jenkins. But even in perhaps the best season of his career, he still helped Devante Parker to eight receptions and 137 yards in a loss that cost the Patriots a playoff bye.
Whether defensive coverage is inherently unstable and highly variable, or it’s just that we’re still in the infancy of football analytics, accurately predicting player performance in pass defense continues to elude us.
Check out our latest NFL predictions.