Spring is a season of tumult for the NFL. Team rosters are in flux as free agency reshuffles veteran talent across the league. Schemes are torn down and rebuilt as coaches try to shore up weaknesses and better exploit their playmakers’ strengths. 

Even the NFL combine in Indianapolis — a constant on the NFL calendar for decades — has been buffeted by the winds of change. The Super Bowl champion Los Angeles Rams didn’t send their GM or head coach to Indy this year— and not for the first time. The Denver Broncos top brass joined L.A. in shunning the combine back in 2020. Yet amid all the upheaval, at least one institution has managed to maintain its cachet: the Senior Bowl in Mobile, Alabama.

Popular perception holds that the Senior Bowl is the first draft-related event of the year (its motto is “The Draft Starts in Mobile”).²⁹ It’s also the first offseason convention where GMs and other decision makers can begin evaluating prospects. And unlike Indianapolis, teams don’t appear to be skipping Mobile.

“Most every team sends all their coaches, front office and all components’ of player personnel as part of the week’s evaluation process,” Andy Dengler, the former assistant director of player personnel for the Jacksonville Jaguars who is now with the Las Vegas Raiders, told SB Nation in 2021.

The Senior Bowl’s privileged position in the draft hierarchy is a little strange, though, and the reason is right there in its name: It’s limited to older players.³⁰ Research has shown that younger players who have early success against older competition are better bets for NFL success. Prioritizing older players runs counter to that finding. The calculus is different for each team, but it seems likely that GMs aren’t ignorant of the value of young players — they simply see Senior Bowl players as lower-risk bets.

“We’ve taken our share of juniors like anybody has,” Colts general manager Chris Ballard told ESPN recently. “But guys that have [long] productive careers in college and then have a good week here usually end up being pretty good players in the league.”

Since we love testable claims here at FiveThirtyEight, Ballard’s comment sent us down a Senior Bowl rabbit hole in an attempt to find out if his impression is correct. Do players who get invited to Mobile tend to end up being productive NFL players?

To answer the question, we turned to Meaningful Snaps Over Expected (MSOE), a metric we introduced during the 2021 season. MSOE measures the number of snaps a player plays over (or under) expected in a season after adjusting for draft slot, position, the year (2021 had 17 regular-season games, for instance) and combine testing data.³¹ For our purposes, “meaningful” snaps are defined as those that occur when the game is still in doubt — between 10 and 90 percent win probability. After training and testing, we applied the model to NFL players drafted from 2012 through 2021 who also attended the Senior Bowl the year they were selected.

To help ground the analysis, let’s first take a quick step back and give a sense of MSOE’s shortcomings: It measures opportunity — the volume of important snaps played — not the value of a snap.³² This means injuries — which are not evenly distributed among position groups — have a significant impact on the metric. Also, players drafted onto teams with deep benches are disproportionately hurt by the metric, since they have to leapfrog a skilled veteran or two in order to earn meaningful snaps. Teams lacking bench depth are more likely to start their young players early and allow them to accrue meaningful snaps.

A final factor to keep in mind is that MSOE rewards teams that find starters from later rounds of the draft more than it does players taken early. This isn’t a flaw — players selected early should be expected to contribute more meaningful snaps to their team — but it’s worth noting.

So, do Senior Bowl players get more meaningful snaps than we would expect? The answer is yes. As a group, players drafted from 2012 through 2021 generated 35,542 MSOE, indicating that Ballard’s impression about Senior Bowl players developing into good players is supported by some evidence.

The effect remains when we break out MSOE by position. Since 2012, the NFL has picked Senior Bowl players who have returned aggregate positive MSOE at every position but quarterback and running back. And among the position groups with aggregate positive MSOE, no position had a higher player-season hit rate³³ or total surplus value than defensive back.

The NFL used 143 picks on Senior Bowl defensive backs since 2012, and the selections have paid dividends. Senior Bowl DBs played 16,697 meaningful snaps over expected, and 49 percent of the player-seasons produced by those picks returned surplus snaps to their teams. 

Interestingly, when we look at MSOE by round, the model suggests that the NFL is finding most of the surplus meaningful snaps from Senior Bowl players in the first two rounds of the draft. Round 3 provides positive MSOE in the aggregate, but more Senior Bowl players have been picked in the third round than any other since 2012, lowering its per-player average.

This distribution of snaps should probably give us pause. No one is immune to the effects of confirmation bias, and it’s reasonable to assume that early round picks will get more chances to succeed than late-round picks. Former NFL player Ryan Riddle has argued that first-round draft picks must prove time and again that they can’t play, while late-round prospects are forced to prove that they can. The model attempts to control for this, but it’s possible that coaches and GMs whose futures are tied to the success of early draft picks lavish those players with opportunity in a way that continues to bias the data.³⁴

It’s also true that if a team is interested enough to take a player early in the draft, there is more to its interest than the fact that the player attended the Senior Bowl. Getting an invite to Mobile is probably worth a bump in the rankings, but it’s unlikely that teams would move a player up from a mid or late-round pick to a first- or second-round selection on that factor alone.

None of this appears to be lost on Ballard: “Look, some of it’s probably random and timing of need and us liking a player,” he said. Like many advantages in the NFL, a Senior Bowl edge may exist, but it’s small. And in a league where even an award-winning GM’s career is short, you don’t get many shots at getting lucky.