This weekend, two perennial MVP candidates and two up-and-coming stars will be under center in NFL conference championship games. Tom Brady and Aaron Rodgers are the establishment; Andrew Luck and Russell Wilson are the hopeful usurpers. They’ve all reached the conference title game, in part, because they’re among the highest-performing quarterbacks in football — all falling within a career average Total Quarterback Rating of 59.6 and 70.1.6
But that’s just an average, and sometimes averages can deceive. Turning several observations into a single metric like a Total Quarterback Rating (QBR) obscures some information. For example, post-2006 Brett Favre7 (average QBR: 52.1) and Andy Dalton (52.0) have roughly identical numbers, but would anyone think of Favre when thinking of Dalton?
A more thorough analysis makes use of the complete set of observations. The distribution of each quarterback’s QBR can offer a more thorough understanding of performance. When we look at quarterbacks this way, we find some players who consistently minimize bad games and others who can be brilliant one week and horrible the next, regularly handicapping their teams.8
Comparing each quarterback’s distribution of game-by-game QBR can be done using a density curve, which is used to estimate the fraction of a player’s performances that occurred in a given interval — in this case, the probability that a quarterback had a QBR of a specific number in any individual game.9 We looked at the 45 quarterbacks with at least 10 starts in the past two years or 50 career starts since 2006, and found that the quarterbacks stratified into 10 categories.10
Each quarterback is given his own density curve, which represents the distribution of game-by-game QBR. The dark black curve in each figure represents the average density for all quarterbacks in that group. The groups are designed such that quarterbacks within the same group are similar, but from one group to the next, there are differences in the centers, shapes and/or spreads of each quarterback’s density curves.
Some groupings weren’t a surprise. Brady, Rodgers and Peyton Manning — widely considered the three best quarterbacks in the NFL and the trio ranked Nos. 1, 2 and 3 in average QBR — make up one group with a curve that spikes dramatically as QBR rises.
Other groupings, however, offered some counterintuitive results. One cohort consists of five players: Luck, Colin Kaepernick, Matt Schaub, Teddy Bridgewater and Tony Romo. Each quarterback in this group had more bad games and fewer exceptional ones than the Brady/Manning/Rodgers set, but he posted more strong games than poor ones over his career.
This would suggest that if you’re going to call one of these quarterbacks “elite,” you need to at least consider that all of them are or were. Even though the narrative around Kaepernick, Schaub and Romo is that they’re inconsistent, their curves suggest otherwise.
Compare that with a larger group that includes Cam Newton, David Garrard, Donovan McNabb, Eli Manning, Jay Cutler, Joe Flacco and Robert Griffin III.11
That cohort has a similar number of great games but also a large bump around QBR 20, which are bad games by any measure — a bump that the Luck group avoids. Not surprisingly, the Luck group has a career win percentage of 59 percent, while the former cohort is only 53 percent. In the NFL, interceptions (a key component of a low QBR score) can drag a team, and a QBR, down.
Now compare both of those groupings to the one with Ben Roethlisberger, Nick Foles, Philip Rivers and Russell Wilson.
They, too, have a bump around QBR 25, but their peak checks in somewhere around a QBR of 85 — a score that gives a team a very good chance to win. That elevated number would explain why the group’s 64 percent win percentage is the highest of the three groups previously mentioned. The occasional bad game won’t break a quarterback as long as his good performances are strong enough.
Other quarterbacks might not like their company. Alex Smith chafes at the idea that he’s a “game manager,” and talent scout Russ Lande thinks he might not be. “Smith is often referred to as [a game manager], but physically, he doesn’t limit you,” he says. “You don’t have to make him a game manager. I think it’s just that some quarterbacks have that philosophy, ‘I’m never going to throw it where it’s a risk.’ It’s more based on their mentality than their physical skill set.” And yet the stats group Smith with E.J. Manuel, Jason Campbell, Josh Freeman, Josh McCown, Matt Hasselbeck and Mark Sanchez. Among that company, perhaps game manager is a generous term.
One of the most interesting groups involves a player who is no longer in the league. Favre’s games after 2006 landed in the same cohort as post-2006 Michael Vick and Vince Young, two quarterbacks known for making plays with their feet and not much else. We can explain this by breaking down Favre against Dalton. Although their average QBR is nearly identical, the vast majority of Dalton’s games fall between a QBR of 25 and 75. Favre, the ultimate freelancer, has a big bump around 15 QBR and another between 80 and 85. Dalton won’t win a team the game, but he probably won’t lose it. Favre, however, is likely to do either.
That difference results from playing style, according to John Westenhaver, president of Football Evaluations and a long-time quarterback talent evaluator. An average quarterback makes about half of his throws using nontraditional mechanics because he’s forced out of the pocket or rushed, but the former Green Bay Packers star made many more than that, often to his detriment. “Favre, to me, put that to the extreme. Although he’s passed for a gazillion yards, I think he leads the league in interceptions.12 Sometimes you have to make a decision: Am I going to throw from this alternative platform, or is it best to select some other alternative, which may be to take the sack, throw the ball out of bounds, run the ball?”
Rivers and Luck offer similar stories. Their career QBRs — 61.5 and 60.8, respectively — aren’t quite Manning’s 76.1 or Brady’s 70.1, but they are good enough to rank the pair in the top 10 of quarterbacks whom we examined. Both can make all the throws and post huge numbers, but there’s a general perception that Luck — a Stanford graduate who has the reputation of being a football savant — makes fewer mistakes. We see this in their game curves. Luck has fewer bad games but fewer truly exceptional ones as well, while Rivers has more bad games and great ones, with less middle ground.
This season’s playoff picture provides a look at one possible future. Whereas Rivers watched from home, Luck led the Indianapolis Colts to a win over Dalton and the Cincinnati Bengals, then an upset over Peyton Manning and the Denver Broncos. He didn’t win the games with his play, but more importantly, he didn’t lose them. In other words, a team might not need a quarterback with a huge bump on the right side of the graph to prevail in the NFL, but one with a peak on the left probably dooms it to failure.
We made an interactive tool with all the quarterback curves. Click here to graph density curves of your choosing, and look at the splits for home and away games. There’s a larger sample size of QBs in the interactive, which means players like JaMarcus Russell are involved. Because who doesn’t want to find out which QBs are most similar to JaMarcus Russell?
CORRECTION (Jan. 16, 1:46 p.m.): A previous version of this article misstated Rivers’ and Luck’s career QBRs.