It’s been a busy midterm week at FiveThirtyEight. Which is to say, a busy week evaluating the NFL at its mid-term point. After nine weeks of action, every team has played at least eight games, which means it’s time for the inaugural Skeptical Football Midterm Awards.
Most Valuable Player
Also known as the “Best Combination of Passing Yards and Touchdowns by a Quarterback, Unless You’re a Running Back Who Rushes for 2,000 Yards or Breaks a Touchdown Record” Award.
That MVPs are nearly always QBs is OK with me. Really, players at other positions can be extremely impressive, but the odds of the true “most valuable” player not being a QB in any given year are small — and even if some other player were theoretically more valuable, the odds that there’d be enough data to be confident of it are virtually nil. Imagine your team is lucky enough to have the best quarterback in football (whoever that might be) — is there any chance you would trade him for any other player, ever?11
Maybe it’s possible,12 but this definitely isn’t one of those years — there are probably five to 10 QBs right now whom you would never want your team to trade for any non-QB.
So let’s talk about most valuable players. The fundamental metric for evaluating players of all positions should be some version of wins produced.13 Because we can model the chances of a team winning at any point in a given game, we can determine how much those odds change on each play that a player is involved in — also known as WPA or Win Percentage Added. But this stat is extremely noisy and susceptible to huge influence from very random situations (a whole season can come down to whether the QB throws an incomplete pass or not on a single down with the game on the line).
Since we can also model how many points a team ought to score — on average — from a particular starting position (down/distance/yard line), we can measure how much a player contributes to his team’s expected points. Since this is more granular than wins, it makes for a powerful proxy. But points can be skewed, too — mostly from plays during garbage time at the end of games, but also from very high-value but unpredictable plays such as fumbles.
Then there’s the additional complication of who gets credit for what. It takes a village to complete a pass. And the running game — traditionally credited to running backs — is frequently set up by the pass. Plus, in some cases QBs even call plays themselves — so why shouldn’t they get credit for the entire offense?
Chart of the week
So to get a better look at the big picture, I’m plotting WPA against expected points added (EPA).14 You can think of it as “raw passing contribution” on the x-axis and “overall effect on the bottom line” on the y-axis:
The trend line is a quarterback’s expected effect on his team’s chances of winning based on his EPA. There are a number of reasons why a QB might be beating it, such as when the rest of his offense is strong (e.g., Alex Smith with Jamaal Charles), or he might contribute a lot with his running (Russell Wilson), or he’s just gotten lucky (or been better) at the right times.
I’ve given Aaron Rodgers a fair amount of flak around here for not “slinging” it enough. In my estimation, he hasn’t taken the necessary risks to improve his team’s chances of winning in the situations that can make such risks worthwhile. But he’s a great QB, no doubt. He’s third behind Peyton Manning and Philip Rivers for EPA/drive, and leads WPA/drive by a wide margin. In fact, his WPA/drive is nearly twice Manning’s.
But as I said, WPA can be deceiving. About 40 percent of Rodgers’s WPA for the entire season comes from a single touchdown scoring drive against the Dolphins, and about three-quarters of that comes from the last two plays of the drive alone. Dare I say? So far this season, Rodgers has been a “clutch” quarterback, coming through at the right times for the 5-3 Packers. That’s great for him, and Packers fans should be happy about it.
But I’m still going to risk the Green-and-Yellow wrath and give the award to Manning anyway.
Why? Because come on, the most valuable player in the NFL doesn’t just up and change from year to year because one QB has a few more good days than another. If Manning has been the MVP five times in the last 11 years, and there’s no evidence of him getting any worse whatsoever, what are the odds that someone has surpassed him?
It’s like instant replay15: The ruling on the field is that Manning is still the best player in the NFL, and to overturn that ruling takes (nearly) indisputable evidence. That caliber of evidence rarely even exists in the NFL, and it certainly doesn’t in this case.
Defensive Player of the Mid-Year
For all the complexity involved in statistically evaluating quarterbacks, it’s practically tic-tac-toe compared to evaluating defensive players. Sure, we record tackles, sacks, interceptions and forced fumbles (and now even more advanced things like how often a cornerback is targeted by the opposing offense), but for any given player these touch on a tiny fraction of plays he’s involved in.
And allocating responsibility on defense is even harder than it is on offense. Football is a dynamic game. An advantage may be created in one spot and realized in another. If one of a team’s interior linemen draws a lot of attention, that may lead to a defensive end getting an abnormally high number of sacks. The effect of an advantage might also be distributed diffusely, such as when a cornerback is so good that he constantly single-covers the opponent’s best receiver, freeing up other options for the other defenders.16
In other words, I’m flying pretty blind. I’m pretty sure Deion Sanders was good. But, say, Reggie White, I don’t know. Probably. So rather than try to solve the impossible, let me fall back on a classic cop-out: Who’s the best player on the league’s best defense?
Which brings me to…
Twitter question of the week
It doesn’t matter whether the Dolphins are “underestimated.” But it does matter that the Dolphins’ defense has been really, really good this year.
Dolphins opponents have scored about half a point per drive less than their expectation (and Miami has faced a tougher-than-average schedule).17 While there is still variance to deal with, we don’t have to caveat the results as much as we do with QBs: The sample sizes are relatively big, and the measurement is more direct. However, the Dolphins’ WPA still takes a pretty big hit from that one Aaron Rodgers comeback drive in Week 6 (take that out and they’d be almost 10 percentage points higher in WPA).
In other words, as good as it’s been, Miami’s D somewhat underperformed in the first half of the season. As always, I’d expect some regression to the mean in the second half of the season, but there’s good reason to expect the team not to regress as much as normal.
So who’s the best player on the Dolphins’ defense? Based on stats alone, I have no idea. But I can consult the most objective authority on player value that I know: “Madden 15.” According to the game, Cameron Wake is the best player on the Dolphins (defensive or otherwise) by a wide margin:
So there you go, Wake wins my DPOY award by virtue of being the most respected Dolphin among video-game programmers (aka the Hacker Gods of the worlds we simulate.)
Rookie of the Mid-Year
I love tracking rookie quarterbacks, but mostly because predicting their future performance is an interesting endeavor that has little to do with their quality of play. Unlike the contenders for “most valuable player” awards, rookie QBs are rarely very good — or if they are, we probably can’t tell because the best ones tend to play for crappy teams. So it is this year, where the most efficient rookie QB is still well below league average.
Rookie “of the year” is also different from “most valuable” rookie, because it opens the door for rookies who may be incredibly good at their position even though their position isn’t the most important.
One rookie is not only good for his position, but has arguably been the best in the league. My Rookie of the Year (so far) award goes to Chandler Catanzaro, placekicker for the Arizona Cardinals:
Catanzaro has yet to miss this year — one of only four perfect kickers remaining18 — and he has the highest points above expectation per attempt of all kickers. And unlike other stats in football, delivering the points expected of them is basically 95 percent of what kickers do (occasionally they have to tackle), and they’re 95 percent responsible for it (they can get bad snaps or the offensive line can let through a blocker). This is why I love kickers: They’re some of the only entities in the NFL we can evaluate with any precision.
There’s still a fair amount of variance, of course, but it’s all very measurable and predictable.19 And it’s not like it would be shocking for a young kicker to be among the best in the league.
Comeback Player of the Mid-Year
Isn’t it a bit weird that Peyton Manning has five MVP trophies, and also has a Comeback Player of the Year award? Like, his mantle wasn’t crowded enough, so they had to give him another award for having neck surgery. Other winners include Matthew Stafford, Tom Brady, Drew Brees, and … Chad Pennington? Twice?
How I could approach this award empirically was bugging me, but then it was staring me right in the face: Practically half of what I write is about QB “comebacks.”
Incidentally, I keep coming back to this issue20 not because I’m enthralled by the quasi-mystical power of the comeback clutch-osity, but because it’s where I think a lot of QBs are too conservative — and their teams’ chances suffer as a consequence. It’s sort of like the NFL statisticians’ obsession with coaches not going for it enough on fourth down.
Given my own obsessions, then, this is the award that’s most important to get right. So, to aid my analysis, I’ve created a chart to profile each quarterback’s comeback game. But instead of focusing on specific comeback thresholds (like the normal 9-point second-half deficit I often work with), I compared how each quarterback did relative to expectation for all scenarios, based on his expectation in each scenario. (E.g., if a QB has a 10 percent chance of winning, I’ll treat it the same whether that 10 percent is due to being down a small amount in the fourth quarter or a large amount in the first quarter.)21
I plotted a handful of Skeptical Football’s favorite QBs for comparison …
Experimental chart of the week
Admittedly, this is an unfair comparison: QBs who play for stronger teams are going to win more often than ones who play for weak teams, so that will certainly affect their “curves.” But what’s fascinating to me are the shapes.
To start with, Peyton Manning not only wins a larger percentage of games than he is “supposed” to, but he exceeds that rate with just about the most perfect curve of anyone in the entire data set. It’s as if his game is starting to approach a mathematical ideal of good quarterbacking.22 For other QBs, we see more variation — some of it random, and some of it following pretty clear patterns:
- Tom Brady and the Patriots have basically matched Manning in situations where they ought to have a low probability of winning, but in games they were supposed to win 80 percent of the time, they’ve only won 80 percent of the time. For shame.
- Aaron Rodgers (as expected), has a pretty unremarkable record (relative to his own high standard) in situations where his team is expected to lose, but does very well in competitive games or games in which his team is front-running.
- Philip Rivers is sort of a less extreme version of Brady. This surprised me because I’ve previously thought of him as a slightly less good but slightly less gun-shy version of Rodgers. But it looks like they’re actually quite the opposite (notice how the blue and yellow lines make an X shape).
And that brings us to Matthew Stafford and Tony Romo. Perhaps not coincidentally, both are Gunslinger of the Week recipients this year (Stafford has won twice), and both have extremely lopsided curves: Note how Stafford wins about as much when his team has a 40 percent chance of winning as when it has a 75 percent chance. This suggests, as I theorized last week regarding Tim Tebow, that Stafford’s comeback success may not be as much a conscious adjustment to sling it more when trailing. Instead, it could be that his perma-risk game is naturally calibrated to extracting as much win as possible from underdog scenarios, but also leads to a lot of blunders that cost his team wins otherwise.23 If you combined the way Stafford plays when his team is down with the way Rodgers plays with his team ahead, you’d have a heck of a QB!
Tony Romo’s curve is a little less dramatic than Stafford’s is, but it’s still pretty clear. Let’s look at it on a scale of 0 to Manning:
Romo’s practically Tony Manning when his team is way behind. But then he’s more like a normal quarterback in more even situations, and makes more mistakes when his team is ahead.
But the “comeback” award doesn’t care how a quarterback plays when he’s ahead, and this season Romo has played great when trailing. He has taken the kinds of risks I like to see (high touchdown rates and high interception rates when trailing), and those have translated into wins. His average passes go more than 10 yards downfield when Dallas is trailing by a large amount, and he has added roughly 3 percent to his team’s chances of winning every drive — good for third-best in the league. (I’ve posted a plot of air yards vs. win percentage added on Twitter). Moreover, when you factor in DeMarco Murray’s WPA troubles because of fumbles, almost all of the Cowboys’ success on offense has come from Romo.
Oh, and he came back to play in the fourth quarter with a broken back.
So Romo wins my Comeback Player of the Mid-Year Award, and hopefully he can come back from his back troubles to sling it up some more. (In which case, could he win the actual comeback trophy as well? Can you come back in the same year that you go out?)
And one more thing
With the awards handed out, let’s part ways with the win curves for some other quarterbacks as well. Again, I’d consider Manning the high baseline — Andrew Luck’s curve looks slightly better than Manning’s, though it’s over a much shorter career. I’ll leave you with the results and you can find your own narratives.
Reminder: If you tweet questions to me @skepticalsports, there is a non-zero chance that I’ll answer them here.
Charts by Reuben Fischer-Baum.