The NFL draft starts Thursday night, and during the coverage you’ll probably hear about safe and risky picks. In some cases, the risk is specific to the player. Perhaps Florida State quarterback Jameis Winston is a risk for his history of alleged off-the-field misconduct, while his counterpart at Oregon, Marcus Mariota, comes with worries that his stock was inflated by the Ducks’ spread offense. But some of the risk is also considered intrinsic to certain positions, while other positions carry a reputation for being particularly low-risk.
The trouble, though, is that there’s no way to prove which positions consistently offer good returns on draft investment — at least not with football data in its current state. And in large part, that’s because we can’t really evaluate on player performance as accurately for the so-called non-skill positions.
For instance, taking a quarterback (like Winston and Mariota) might seem like a hazardous bet to make early in the draft. But it’s possible that QBs only appear risky relative to other positions because we can more readily measure a quarterback’s performance statistically. In other words, when Ryan Leaf posts a 39.0 quarterback rating as a rookie, he’s easy to identify as a bust, but his linebacking equivalent might be harder to recognize.
Here’s another example. You could use Pro-Football-Reference’s Approximate Value (AV) to estimate the odds that a player’s performance15 will live up to the expectations of where he was drafted, and break those numbers down by position:
You’d find that the average offensive lineman16 meets or exceeds the median AV expected of his draft slot about 59 percent of the time — the NFL-wide average across all positions is, by definition, 50 percent — while the average receiver17 provides positive value for his pick slot only 41 percent of the time. And quarterbacks wouldn’t be far behind receivers on the “risky” list, beating expectations at a rate of only 43 percent.
But there’s another clear pattern in the AV data: The positions that seem like the riskiest picks are also the ones where we have the most data to differentiate between good and bad performances. Coincidence? Probably not.
An offensive lineman’s AV is based solely on his team’s offensive performance, his own playing time and any accolades he receives (like Pro Bowl or All-Pro nods). So, short of being outright benched, there’s very little he could do to distinguish himself negatively under the structure of the study above (or those like it). A skill-position player struggling to meet expectations, on the other hand, can be identified via his inferior yardage, touchdowns, turnovers and the like. (To the extent that those metrics are even good descriptors of player performance.)
It’s similar to the phenomenon that causes linemen to boast a far higher Pro Bowl “retention rate” than other offensive positions, particularly quarterbacks. The less information voters have to go on, the more they rely on a player’s priors (and perhaps rightly so). And the same can go for the draft, where certain positions can feel safer simply because we don’t have glaring, easily-quantifiable evidence to the contrary.
The answer, of course, is more data. When Chase Stuart conducted a more granular (if anecdotal) look at offensive tackles using ProFootballFocus grades, for instance, he found what seems to be a healthy bust rate even for a position often tagged as one of the most reliable. Then again, PFF grades aren’t universally accepted as gospel, either; besides, they only go back to the 2007 season (as opposed to 1950 for AV).
That’s why, for now at least, it may not be possible to truly say whether certain positions are more or less likely to live up to their draft-day expectations. It’s another unanswerable question to throw onto the pile of things about the NFL draft that we just don’t know.