It’s Hard To Measure NFL Matchups, So We Built A Better Metric

The NFL is a league of matchups, and no matchup is more crucial to wins and losses than the battle between a quarterback and the pass defense. That’s important for more than just the coaches calling X’s and O’s: Each week, the media, betting markets and fantasy players all make predictions about which teams (and which players) will best take advantage of their opponent’s weaknesses.

Because football is such a complex sport, quantitative analysts haven’t found a great way to capture all the relevant information for each offensive and defensive passing unit in a way that lets us make reasonable predictions about future outcomes. Perhaps the biggest flaw of most approaches to strength-of-schedule and matchup analyses, for example, is that they assume that because a defense has or has not allowed yards, completions and points generally, we should expect that performance to continue against a future opponent. The problem: That isn’t really true. Defenses have weaknesses, but to exploit them, an offense first needs to know they exist. Then it must possess the personnel needed to actually take advantage. In the NFL, neither of those circumstances is a given.

One approach to quantifying these matchups is to look at where on the field a quarterback tends to complete more passes than average and where a defense tends to allow more completions than average. We’ve published heat maps like these before on FiveThirtyEight — both for defenses and for quarterbacks. (We think they’re pretty cool.) If we use some fancy math,¹ we can roll these heat maps into a single, convenient number that describes the similarity between where a QB’s passes are completed and where a defense allows passes in a given matchup, with higher numbers indicating a better matchup for the quarterback. These similarity scores then give us an idea of which QBs have faced a more difficult slate and who might have a challenge ahead in the upcoming week.

Like most quantitative measures, similarity scores are worthless without a decent sample size. To ensure we have enough pass attempts to make good comparisons, and to eliminate early-season variance, we chose to calculate our scores only for matchups that occurred from Week 5 onward. We then averaged the scores for each QB who played in at least three games over the seven-week period from Week 5 through Week 11 to see who’s had the toughest schedule, ranking them from the most challenging schedule to the most favorable.

Dolphins rookie quarterback Tua Tagovailoa — who was benched for poor performance on Sunday by Miami head coach Brian Flores — is at the top of our list, indicating that he’s had a tough set of matchups to begin his NFL career. Meanwhile, Derek Carr — who is having the best season of his career — has had the easiest slate of matchups since Week 5.

Tagovailoa’s situation, though, makes him difficult to score. With only two passing attempts in his pro career going into his first start, which came against the Los Angeles Rams in Week 8, his sample size was too small for us to calculate a score for that first start. In his second start, Tagovailoa faced Arizona in Week 9, leading the Dolphins to a 34-31 win while throwing for 8.9 yards per attempt. The similarity score we calculated for that matchup was low — 35.4 — but it was still based on a small sample: 24 career attempts. By Week 10, which he entered with 52 career attempts, Tagovailoa’s sim score rose to 67.2 against the Chargers, a game in which he threw for 169 yards and two touchdowns. This past week against a Denver team with only three wins, his score was 86.7, meaning the places where the defense tends to allow completions lined up well with the places where he tends to make successful throws. That might explain his coaches’ frustration with his performance — this was a game in which he probably should have played well.

Seahawks quarterback Russell Wilson has had the second-toughest midseason schedule among current starting QBs. In Week 11, Wilson’s similarity to Arizona’s pass defense was just 67.9. Wilson led the Seahawks to a win, but he ended up throwing for only 197 yards against a Cardinals team that had been allowing around 250 yards per game through the air. If Wilson’s upcoming opponents turn out to be more accommodating, his numbers — and his case for league MVP — could improve.

Lions quarterback Matthew Stafford, who has the third-lowest average sim score this season, had his toughest game in Week 9 against the Minnesota Vikings — a matchup with a sim score of 42.5. The heat map for Stafford heading into the game showed that while the Viking pass defense was weak over the middle and up the hashes, so was Stafford, making him ill-equipped to take advantage of their vulnerabilities. Meanwhile the Vikings were better than average at taking away deeper sideline passes, spots where Stafford is usually strong.

Stafford finished the Vikings game 23-of-32 for 211 yards with one touchdown and two interceptions. His adjusted yards per attempt (a stat that gives extra credit for touchdowns and penalizes for interceptions) for the game was 4.41, his worst performance of the year so far.

We’ve used our similarity scores to look back at a player’s performance, but let’s shift ahead to try to spot matchup problems in this weekend’s games.

We might have cause for concern about Aaron Rodgers against Chicago: A sim score of 59.8 will be Rodgers’s toughest matchup in the period we’re measuring. Taysom Hill and P.J. Walker also feature low sim scores, but both played decently in their first NFL starts, and sample size issues similar to Tagovailoa’s definitely apply here, so maybe we should be more bullish about their matchups. But in New York, even if Sam Darnold is healthy, the Jets should probably start Joe Flacco against Miami. Flacco’s matchup against the Dolphins is over 40 points higher than Darnold’s, which should make coach Adam Gase’s choice an easy one. Unless, of course, the Jets’ goal is to remain winless for the rest of the year, in which case starting Darnold is absolutely optimal.

Looking ahead: AFC South showdowns haven’t always been synonymous with “epic action,” but the best game of Week 12 is probably Sunday’s Indianapolis-Tennessee rematch, with the division lead on the line. Philip Rivers and the Colts are fresh off a comeback victory over the Packers and have won four of their past five contests — including one against the Titans — elevating themselves into the top five of this week’s Elo rankings. Their strong defense will look to slow down Ryan Tannehill, Derrick Henry and the rest of the Titans, who broke out of their recent skid with an impressive comeback of their own on the road against the Ravens. The more important matchup may be on the other side of the ball, however, as Tennessee struggled to stop Indy during the Colts’ 34-17 win two weeks ago. Both teams are 7-3, and the winner’s division-title odds would soar from roughly 50-50 to about 85 percent, with the loser dropping to about 15 percent. Our model gives the Colts the slight edge at home, with a 60 percent probability of moving into the AFC South driver’s seat. Elo’s spread: Indianapolis -3

Neil Paine contributed research.

Check out our latest NFL predictions.