During this NFL preseason, the media spotlight has shined brightest on the quarterback competition in Cleveland between incumbent Brian Hoyer and rookie Johnny Manziel. Although Browns coach Mike Pettine officially named Hoyer his starter Wednesday (a decision coming, not coincidentally, after Manziel made an obscene gesture toward the Washington Redskins’ bench during the Browns’ 24-23 loss Monday night), the attention paid to the rivalry — and to Manziel in general — has reached crazy levels.
But why?
Why are we acting like the Super Bowl is riding on the outcome of Hoyer vs. Manziel? This is, after all, a team Vegas has winning the fourth-fewest games in football. Manziel is a rookie whose bona fides are far from established; Hoyer has four career starts to his name and has been a significantly below-average passer when he has played over the past two seasons.
Montana vs. Young this ain’t.
But preseason quarterback controversies have long captivated football fans. Who could forget J.T. O’Sullivan and Alex Smith dueling in San Francisco for the 49ers job in 2008? Or Jon Kitna’s struggle with Gus Frerotte in 2002? Or even the Browns’ own 2003 skirmish between Kelly Holcomb and Tim Couch?
If you’re sensing a trend, it’s that August QB controversies usually involve a pair of equally dismal options. And, no matter who wins, he tends to produce pretty uninspiring results come the regular season.
Going back to 2000, I used LexisNexis to search news reports for variations of phrases such as “quarterback competition” or “QB controversy” during the summer months, when NFL teams open up training camp and play preseason games (aka prime time for hyped-up QB rivalries). I came across 88 cases of the media discussing a competition for an NFL starting quarterback role, from Kordell Stewart vs. Kent Graham (2000 Pittsburgh Steelers) to Terrelle Pryor vs. Matt Flynn (2013 Oakland Raiders).
And indeed, teams featuring preseason QB competitions tended to be mediocre-to-bad clubs. Since 2000, they averaged a shade under seven wins during the subsequent regular season, with an average adjusted net yards per attempt index, or ANY/A+, of 92.3 (100 represents the overall NFL average, and the replacement level is somewhere between 90 and 91). Among those who ended up getting at least 100 attempts during the regular season, only 22 percent of the quarterbacks involved in these controversies were average or better by ANY/A+. As a rule, teams are dealing with pretty bad passers in these situations, no matter who “wins” the competition.
What’s more, it doesn’t seem to matter which quarterback a coach picks as his No. 1 coming out of the preseason. Limiting our data to the 40 cases in which both of the combatants had at least 100 pass attempts (to give ourselves a reasonable sample upon which to judge the preseason decision), teams picked the QB with the higher eventual ANY/A+ only 51.3 percent of the time. And in fact, statistically, there was no significant difference in performance between the preseason-quarterback-competition winners and losers.
This suggests that picking the right horse in a preseason QB derby practically comes down to a coin flip.
Now, a potential concern in the methodology outlined above is that we could have biased our results toward evenness because of the playing-time criteria. (Barring injury, the odds are low that a clear-cut quarterback competition winner would end up sharing regular-season snaps with his backup.) But we can check our results against a control group of teams with a similar distribution of passing attempts (but no reported QB controversy). In the controversy group above, the average difference between the opening-day starter and his backup was -2.0 ANY/A+; in the non-controversy group, the difference was +3.2 ANY/A+.
That’s not quite a statistically significant difference (p=0.08), but it’s close. So the biasing effect could still be masking the true difference between the groups. But to some extent, it does suggest the underappreciated (yet obvious) truth about QB controversies: There isn’t going to be much difference in performance, no matter which option a team goes with as its starter.
And, as a corollary, it means we’ve probably spent entirely too much time worrying about which of two generally equivalent, below-average quarterbacks will get a chance to start for a bad team.