Are Defensive Play-Callers Consistent Enough To Be Exploitable?

If NFL analytics is a set of festive Russian nesting dolls, the final pea-sized matryoshka is defense. We’re actually pretty good at understanding offensive skill position players at this point. When a quarterback change is made on a team, our models and the betting markets move in response. No Patrick Mahomes? “That’s bad for Kansas City’s chances! Let’s move the line down and predict fewer points scored.” But when a defender is out, everyone shrugs. No Xavien Howard?¹ “Um, well, I’m not sure what that means for Miami.” We know so little about defense that some unhinged analysts have even pounded out “defense doesn’t matter” on various social media platforms in fits of rage and frustration.

It’s likely that the lack of predictive power in defensive analytics has to do with the reactive nature of defense. NFL defenses are highly interconnected, weak-link systems. What does weak link mean? It means that if just one person in the defensive chain fails, the entire unit is assigned that failure.² We can measure that failure, but the fact that we can’t predict future defensive performance implies that the precise types of failures we’re measuring are unlikely to occur exactly that way again. Most NFL defenses are good enough to not repeat the same mistakes over and over — they just find new and exciting ways to fail. All of this makes apportioning blame and properly accounting for success pretty difficult.

But a recent tweet from Anthony Reinhard made me wonder if we can assign some of that blame reliably on coaches. Could we successfully predict how coaches will respond to game states like obvious passing or obvious running situations? After all, defensive coordinators typically call the defensive plays, and they’re just one person, not an interconnected unit. If we can predict what a coach will do, maybe we can gain a deeper understanding of how a defense is likely to react in certain situations.

Reinhard looked at how teams respond to probable passing and running situations by measuring the number of defenders that line up “in the box” — or near the line of scrimmage — and comparing it to the probability that a team will drop back to pass. In passing situations, it’s usually advantageous to drop a defender into coverage. Likewise, if it’s a more obvious running situation, the defense will want to make sure they have enough players near the line of scrimmage to be “gap sound” and properly defend the run.

I took Reinhard’s idea and calculated a new metric based on it: Defenders in the Box Over Expected (DBOE). Then I looked at how stable DBOE was year-to-year by subsetting the data to just teams with the same defensive coordinator from one year to the next.

It turns out that DBOE is remarkably stable,³ a rare feature for an advanced defensive metric, indicating that coaches tend to react to obvious passing and rushing situations in reliable ways over time.⁴

Interestingly, in the period for which we have data, the league has moved toward favoring lighter boxes over what we might expect. The downward trend was pretty reliable, right up until around 2019. And in 2020, the trend reversed itself entirely.

There’s more work that needs to be done to apply DBOE to football in useful ways, but the implications are interesting. If a defensive coordinator will reliably overcommit defenders to the box in obvious passing situations, opposing offenses could potentially take advantage. Similarly, defenses that back off and play the pass on more obvious rushing downs could be exploited by an offense spreading out the defense and running. Even better, DBOE is based on the probability of a play being a dropback, a metric that’s fairly opaque to a defensive play-caller since they often simply match big bodies to big bodies when the offense trots out their personnel. That means having a high or low DBOE isn’t a tendency the defensive coordinator can easily obscure without some effort. It’s likely an integral part of how that coach approaches calling a game.

Here is the full list of NFL defensive coordinators and their DBOE, through Week 15:

Through Week 15 this year, the coordinators who seem to be most concerned about getting beat through the air include Brandon Staley of the Los Angeles Rams and Ed Donatell of the Denver Broncos. The Rams have one of the best defenses in the league this year, allowing a league-low 4.6 yards per offensive play. (Of course, they also have Aaron Donald and Jalen Ramsey, which probably has something to do with that.) Meanwhile, the Broncos have been less successful, ranking 13th in the league at 5.5 yards allowed per play.

On the other end of the spectrum, Todd Wash of the Jacksonville Jaguars and Jack Del Rio of the Washington Football Team would prefer to stack the box to prevent the easy yards on the ground. Jacksonville has the league’s worst defense, allowing 6.3 yards per play, but Washington is fourth best at 5.0 yards per play. All of this suggests that while DBOE might help us understand coaches and their preferences, football is still a game predicated on having good players on the field — especially on defense.

Looking ahead: The best and most significant game of Week 16 — from a Super Bowl odds perspective, at least — features the Tennessee Titans paying a visit to the Green Bay Packers at Lambeau Field. Neither team is in danger of missing the playoffs, but there’s still plenty to play for: The Pack have not yet locked up the NFC’s lone first-round bye, but they could with a win and losses by the Saints and Seahawks. And as AFC South leaders by way of a tiebreaker with Indianapolis, the Titans are currently the No. 4 seed in the conference, though they need to keep pace with Indy in order to host a playoff game instead of going on the road. That sets the stakes in this one, where each team’s weak defensive link will get a stress test against the NFL’s best before the playoffs: Tennessee’s suspect pass defense (No. 28 in schedule-adjusted EPA) going against Aaron Rodgers, and Green Bay’s awful run D (No. 27) staring down Derrick Henry. Both teams are in pretty great form right now as well, with Tennessee winning four of its past five contests while Green Bay has won four straight (and six of seven). We think the Packers — playing at home — have the edge, with a 62 percent chance of grabbing the W and moving closer to the No. 1 seed in the NFC. Elo’s spread: Green Bay -3½

Neil Paine contributed research.

Check out our latest NFL predictions.