Our New Metric Shows How Good NFL Receivers Are At Creating Separation

Is Michael Thomas elite? On its face, that seems like an outrageous question. Michael Thomas, the New Orleans wide receiver who led the NFL in receiving yards and set an NFL record for receptions in 2019¹ on the way to winning Offensive Player of the Year — that Mike Thomas might not be an elite receiver? Yet throughout the offseason, NFL analysts have debated whether Thomas’s production is best explained by his skill and talent, or if instead he’s merely a good receiver who runs a lot of slants and benefits from being in an elite offense.

Playing with Drew Brees — a former Super Bowl MVP who holds the NFL record for highest career completion percentage, most career passing yards and most career passing touchdowns — certainly doesn’t hurt. And running routes in schemes designed by Sean Payton, a coach with a .630 career win percentage (208-131, third among active NFL coaches), probably also has a strong positive effect on his production. But Thomas’s numbers are still eye-popping, and his peers in the NFL recently ranked him first among all wide receivers (and fifth overall) in the NFL 100. So which is it: elite talent, or elite situation?

One way to try to answer the question is to look at how well a receiver creates and maintains separation from a defender. Being quick and fast isn’t enough for NFL success; just ask Yamon Figurs or Darrius Heyward-Bey, both of whom ran a 4.3 40-yard dash at the NFL combine. The elements of savvy route running — footwork, head and body fakes, disguising the intent of the route, changing direction sharply without losing speed — all appear to be more reliable indicators of NFL skill and talent than speed. And the ultimate goal of every route is to create enough separation from a defender to earn a target and make a catch.

To try to capture the results of this game of cat and mouse between receiver and defender, we used NFL Next Gen Stats data that measures the distance between a receiver and the nearest defender at key moments in each play. Perhaps it’s no surprise that in the ultimate team sport, assigning credit for something even as seemingly straightforward as separation is complicated. For instance, short passes are, on average, associated with more separation than deeper passes because a defender’s top priority is to avoid getting beat deep. So we need to account for depth of target and how far the QB had to throw the ball when we apportion credit for the separation a receiver got on a given play. The type of play call matters, too. Play-action passes create more separation than non-play-action passes at nearly every depth of target on average, so we need to contextualize players who are targeted more often on play-action as well.

We also needed to decide when to measure receiver separation. Since the most interesting routes to analyze are those that earned a target, there are two obvious points in a play to focus on: the moment the ball leaves the QB’s hand and the moment the ball arrives at the receiver’s location.

But there again we’re confronted with confounders that make apportioning credit and blame difficult.² For example, some QBs throw with more anticipation than others, releasing the ball before a receiver has made his break and created the separation necessary for a successful completion. Measuring separation at the time of the throw punishes teammates with great chemistry.

So we decided to focus on separation at the moment the ball arrives, on the theory that scheme and QB play have the least influence at this crucial moment in a pass play. Ultimately, our separation model ending up including features that account for quarterback arm strength, the receiver’s separation at the time the QB targeted them, the horizontal and vertical position of the receiver on the field at the time of the throw, where the receiver lined up pre-snap, the distance to the goal line, the amount of break in the receiver’s route during the football’s journey through the air after it was released, the depth of the QB’s drop, the number of other routes that were being run on the play, if the play was a play-action pass or a screen, and the number of deep safeties.³

The final model does a decent job of predicting separation at the catch point on a given play.⁴ We used its predictions as a baseline for performance and compared each receiver’s actual separation on a given play to what the model expected an average player’s separation would have been, given similar circumstances. This result is a context-adjusted “separation over expected” (SOE) metric that we can calculate for each NFL receiver.

Since depth of target is so important in determining separation, we grouped plays into buckets — depending on whether a pass attempt was short, intermediate or deep — and created an SOE leaderboard for each. The resulting lists have reasonably strong face validity — players at the top of the leaderboards tend to be widely regarded as good route runners — suggesting SOE could be useful as a descriptive metric.

Two tight ends — Evan Engram of the New York Giants and Zach Ertz of the Philadelphia Eagles — top the list of receiver seasons with the highest separation over expected on short throws (5 air yards or less).⁵ Ertz’s targets especially were high value. While averaging more than a half-yard over expected in separation, his per-play expected points added (EPA) was worth more than 0.4 points in 2019. As a general rule, however, separation and value are decoupled on short passes. The defense is typically willing to allow an offense to throw to wide-open players short, then rally to make a tackle for a short gain.

Another interesting feature of short-target SOE is that a player’s ability to consistently get open on short throws appears to be mostly nonexistent. Having a high SOE in one season says very little about whether you will have a high SOE in the next.⁶ Short targets are also the only leaderboard where running backs make an appearance, owing to the dearth of targets they get deeper downfield.

These short passes, however, are where Michael Thomas frequently shines. Forty-eight percent of Thomas’s 185 targets came on passes 5 yards or less downfield in 2019. In the three seasons for which we have Next Gen data, Thomas has regularly posted high EPA per play values on these short targets and has been above average in creating separation at the catch point in two of the past three seasons.

Other players appear able to consistently create separation on short targets — Jarvis Landry is one example — but the value of Landry’s targets as measured by EPA are much lower than those directed at Thomas. In fact, on an EPA-per-play basis, throwing short to Landry has a worse point expectation than running the ball. That’s somewhat shocking — and perhaps it explains why the number of Landry’s short targets dropped by nearly half after he moved to the analytics-friendly Cleveland Browns.

Wide receivers make the bulk of their high-value receptions on passes at intermediate depths (between 5 and 15 air yards). The intermediate SOE leaderboard includes seasons from receivers like Davante Adams, Keenan Allen, Danny Amendola, Cooper Kupp and Adam Humphries, all of whom are generally regarded as skilled route runners. Interestingly, among the three target depths, SOE on intermediate passes appears to be the most stable year to year. So while we’d like more data, we should probably expect players who show the ability to separate above expectation on routes at these depths to continue to do so.⁷

Thomas isn’t in the same class as Adams and Allen when it comes to creating separation — Adams has averaged over a half-yard of SOE the past two years on intermediates routes — but despite the tighter windows, the expected value Thomas created on these targets ranks him among the best in the league on a per-play basis. Perhaps because of his success, Thomas has seen a steady increase in the number of targets at those depths. And compared to Atlanta’s Julio Jones — a receiver whose natural talent and skill set are rarely questioned — Thomas comes out ahead on both our separation and value metrics.

Timo Riske of Pro Football Focus has shown that the best receivers in the league earn their targets all over the field, so it’s no surprise to see familiar names in the deep-target SOE ranks. But we also see deep ball specialists like D.J. Chark and Kenny Stills at the top of the list — the “stretch X” receivers whose job is to take the top off a defense. And at the bottom of the list we find names like Robby Anderson, a free agent this offseason who reportedly drew few offers from teams; an aging Larry Fitzgerald; and the unsigned Kelvin Benjamin. Finally, we see the continued effect of depth of target on separation. Deep targets (at least 15 air yards) are the most valuable in football on a per-play basis, but they’re also the throws with the tightest windows.

Again it seems instructive to compare Thomas to Jones. Jones was targeted deep more frequently, suggesting that his skillset is better suited to the demands of beating fast humans in a footrace, but he’s also not as successful at creating separation from defenders as Thomas is. And Julio’s targets have, on average, been worth less than Thomas’s in the previous three years. (Again, no one doubts Jones’s talent or skills, and both he and Thomas are consistently in the conversation for best receiver in the league.)

When we account for the most impactful context that affects a receiver’s most important job — getting open — Thomas is routinely above average in creating that separation. And targets to him are among the most valuable plays in football across all depths. There isn’t much evidence to support the idea that Mike Thomas is anything but an elite football talent.