Teams Are Excelling When Their QBs Leave The Pocket. Can That Continue?

In Kansas City’s Oct. 6 game against Indianapolis, Chiefs QB Patrick Mahomes faced a third and 18 from the Colts 27-yard line. Perhaps a bit skittish after an 8-yard sack on the previous play, Mahomes vacated the pocket early despite good protection from his line. Retreating backward, Mahomes faked left, then spun around and sprinted to the right sideline. He turned upfield at the boundary, and before crossing the line of scrimmage, he threw the ball approximately 35 yards across his body to wide receiver Byron Pringle in the end zone.

Mahomes is perhaps best known for his rocket arm and improbable no-look passes, but his ability to salvage plays like this — scrambling outside the pocket when the intended play design fails — is also off the charts. Mahomes has quickly become exhibit A when coaches talk about the value of a QB who can create out of structure. Throws outside the pocket have been on the rise across the league in recent years, including plays that are explicitly designed to put the QB on the move. Chiefs head coach Andy Reid has even said that scheming movement for his quarterback is what makes his offense go.

This season, it seems to be paying dividends for teams that are proficient at it. With some notable exceptions, NFL teams that are successful on dropbacks outside the pocket have tended to win more games.

The Chiefs, 49ers and Patriots are all likely playoff teams that have taken different approaches to moving their quarterbacks.

The Chiefs have eight wins and rank fifth in the NFL in expected points added per play on dropbacks outside the pocket. I looked at all 60 plays that Kansas City has run outside, and I determined that half were improvised — plays made when the original call in the huddle broke down. On those improvised plays, Mahomes and the Chiefs averaged 0.39 EPA per play¹ — 0.27 EPA per play better than the schemed plays designed by Reid.

San Francisco’s similar success to Kansas City belies a profound difference in approach. The 49ers are tied for the league lead in wins with 10 and are first in the NFL in EPA per play on outside dropbacks (0.51). But the Niners have run the second-fewest outside dropbacks in the league (28), and while the Chiefs have been reliant on the improvisational brilliance of their superstar QB, San Francisco has leaned heavily on head coach Kyle Shanahan. Nearly two-thirds of the 49ers’ plays outside the pocket in 2019 have been schemed, and Shanahan’s offense has been brilliant on those 18 occasions, earning 1.2 EPA per play. But when quarterback Jimmy Garoppolo has been asked to improvise outside the pocket, he’s been a liability, accruing -0.43 EPA per play.

Similarly, the Patriots seldom ask Tom Brady to leave the pocket. The Patriots rank just above the 49ers in outside dropbacks on the year with 32, third-fewest in the league. Perhaps one reason is that when the 42-year-old Brady does move outside, it’s generally been a disaster: 62.5 percent of Brady’s plays outside the pocket have been improvised, and New England has an EPA per play of -0.70 on those scrambles. That’s far worse than the team’s work on schemed plays, which have generated 0.0 EPA per play.

Which approach is most likely to succeed? To answer that, we need to know if we should expect outside-the-pocket performance to continue. And if it does, which flavor of outside-the-pocket performance is most likely to persist: the improvisational approach exemplified by Mahomes and Kansas City, or the scheme-based approach of Shanahan and San Francisco?

To try to answer these questions, I tested the year-to-year stability of EPA per play both inside and outside the pocket.² The results were somewhat surprising. It turns out that a quarterback’s performance inside the pocket has a 96 percent chance of being more stable than their performance outside the pocket. That is, a QB’s performance inside the pocket is a better predictor of future success (and future struggles) than the plays he runs outside of structure. This aligns generally with the work of Eric Eager at Pro Football Focus, who found that QB play from a clean pocket is a more consistent measurement of quarterback performance, relative to QB play under pressure. Moreover, the evidence suggests that performance on schemed plays outside the pocket — specifically play-action — is almost completely unstable, further driving home that we can’t infer much about a quarterback’s success on outside the pocket.

None of this is to say that outside-the-pocket performance doesn’t matter. It’s simply unpredictable. Fumbles have an enormous impact on the outcome of a game, but they’re basically random events. Just because a team has had good or bad fumble luck throughout a season doesn’t mean future fumbles are more or less likely to occur.

We can use this information to set reasonable expectations for the rest of the season and into the postseason. Kansas City relies on Mahomes’s out-of-pocket exploits to a large degree: They’ve run nearly as many plays outside as San Francisco and New England have combined. And while outside-the-pocket performance is generally unstable across years, the type of plays Mahomes makes out of structure tend to be the most steady. Mahomes is also impressive throwing from the pocket. Among healthy, qualifying QBs, Mahomes ranks second in in-the-pocket QBR, behind only Lamar Jackson. It’s reasonable to conclude that his performance has a good chance to continue.

The performances of Garoppolo and Brady outside the pocket, however, take on less importance. Garoppolo likely won’t be as good in the future on outside plays — since most of them were schemed play-action — and there’s a chance that Brady won’t be as bad moving forward. We should probably temper our expectations on that count, though, since it’s unlikely that Brady will grow new legs at age 42. Perhaps the poor out-of-pocket numbers are here to stay for New England. More concerning is Brady’s very average in-the-pocket QBR of 54.4 through 13 weeks. If that number doesn’t improve, the Patriots’ path through the postseason could end up being one of the toughest, despite their gaudy regular-season win total.