Seven years ago, I started reporting a piece for Sports Illustrated on the financial health of NFL players — a whole lot of them went broke soon after leaving the league. Ever since then, I’ve been seeking reliable statistics to help define the scope of a complicated problem. Just how many athletes emerge financially imperiled once their pro salaries dry up?
And so my brain’s wonkiest pleasure centers lit up recently when an economist friend emailed me a PDF that bills itself as a new, “comprehensive” study of that very issue. What I wasn’t expecting was for the study — in the course of selling itself to the public — to train its sights on me.
Presenting the National Bureau of Economic Research’s Working Paper No. 21085, “Bankruptcy Rates Among NFL Players with Short-Lived Income Spikes.” The researchers who put it together used Pro-Football-Reference.com to compile a list of 2,016 players who were drafted by NFL teams between 1996 and 2003. They then gave that list to a third-party service, asking it to search for recorded addresses that matched those names, and had another service scan for bankruptcy filings that fit both the names and the addresses.
The study’s big finding: Of the 471 draftees who had been retired for at least 12 years, 74, or 15.7 percent, had filed for bankruptcy by year 12.
Listen: Pablo Torre Discusses His Piece On Our Sports Podcast
Torre’s segment begins at 42:00. You can subscribe to “Hot Takedown” on iTunes.
It’s worth mentioning, for the uninitiated, that NBER working papers have not yet been peer reviewed. As the preamble to FiveThirtyEight’s In the Papers series perpetually explains, the papers’ “conclusions are preliminary (and occasionally flat-out wrong).” But as crude as the methodology of this particular study may be, it does seem to be a useful analysis of a defined subset of NFL draftees. Had the paper existed back when I reported that 2009 SI article on how and why athletes lose their money, I would’ve added it — caveats included — to the depressingly brief roll call of academic research in this field.
And that’s why one of the paper’s other prominent findings, as announced in a note from the authors, was so jarring to read. “The result of our comprehensive research on bankruptcy risk among NFL players,” they wrote, “is quite different from a widely-cited Sports Illustrated article, which reported that 78 percent of former NFL players are bankrupt or under ‘financial stress’ within two years of retirement (Torre, 2009). After 2 years of retirement, only about 1.9 percent of players in our sample have filed for bankruptcy.”
One blog used that first sentence as the basis of a headline. In terms of economist trash talk, this was basically the authors flexing.
But their summary of the 78 percent statistic was incomplete. Here is what my SI story said: “Reports from a host of sources (athletes, players’ associations, agents and financial advisers) indicate that: By the time they have been retired for two years, 78 percent of former NFL players have gone bankrupt or are under financial stress because of joblessness or divorce.”
I called up the California Institute of Technology’s Kyle Carlson, one of the authors of the working paper, and mentioned this discrepancy. I pointed out that excluding the details about joblessness and divorce helped mask the difficulty of measuring a personal, complex issue. Financial health cannot be comprehensively captured by whether a person filed for bankruptcy or not. That their results are “quite different” from, well, an entirely different study should be surprising to approximately zero people.
Bankruptcy, by definition, is an elective legal proceeding wherein a person publicly admits an inability to pay outstanding debts. It is also a sufficient but absolutely not necessary condition for defining financial stress; for various reasons, bankruptcy and financial stress can even be mutually exclusive. In my reporting, I have met athlete after athlete who knew to avoid the headline-generating shame of a bankruptcy filing. Even if their finances remained a shambles. Even if they were, by any reasonable standard, broke.
Consider former NBA guard Allen Iverson, or ex-NFL receiver Raghib “Rocket” Ismail — whose case, by no coincidence, leads off that SI story — both of whom never filed for bankruptcy despite squandering their pro fortunes.
“Bankruptcy is only one measure of a person’s status,” Carlson admitted to me. “It’s what we could get data on. There are many other ways in which a person could be in financial trouble. You might think of our number as maybe a lower bound for the number of guys who are actually in trouble.”
And yet what Carlson called a “narrow” academic paper was promising a definitive analysis of a larger, albeit related, problem. Due to the academic marketplace — “Publish or perish” is an idiom for a reason — working papers tend to be hungry and ambitious. The goal is to make it into a journal and contribute knowledge to the world. But it can be tough to find the kind of dramatic angle that makes for an eye-catching study.
Measuring financial health just through bankruptcy would be the equivalent of, say, assessing the mental health of a population by just tracking suicide rates, ignoring any other indicators of psychological stress. Pain isn’t exclusively about disaster.
You often hear data people say that sample size matters. It’s not just size, though; it’s also the quality of the sample itself. Yes, the authors tell us about 2,016 players who were drafted by NFL teams between 1996 and 2003. But thanks to the relative inaccessibility of data on undrafted players, they also left out every player who didn’t begin his career by shaking hands with an NFL executive on a televised stage. In 2013, according to the Elias Sports Bureau, 638 undrafted players were on active rosters for at least one game — a staggering 31.4 percent of the entire NFL. And that does not even include the drafted or undrafted players who never make it to an active roster, toiling instead on the practice squad or injured reserve.
My 78 percent number from 2009 is limited in its own ways, admittedly. I did not conduct the study myself, and, as I told Carlson, I wish I had access to all its component parts. But the statistic was vetted by multiple NFL and NFL Players Association sources who asked not to be quoted or only be quoted anonymously. Several of them shared with me that the stat had been presented at confidential meetings they attended. It was the last, best estimate anyone in this industry had seen; in the six years since SI published the article, neither the PR-obsessed NFL nor the Players Association has disputed the number’s validity in public.
What the NFL did instead was eventually market its own study, published months after my SI article in 2009, with findings that were more pessimistic than the NBER working paper’s but more optimistic than the ones in my article. The league supplied University of Michigan researchers with an even more rarified sample of players: pension-eligible retirees, meaning those who had played a minimum of three years. The average career length among those interviewed was 7.3 seasons, far longer than the NFL average. “We had no way to include players with shorter careers,” one of the Michigan authors, David Weir, wrote to me in 2012, “and I would certainly agree that they would be an interesting group to know more about.” That same year, I received an email from an NFL PR person with the following results for me to chew on: “45% (age 50+) and 48% (age 30-49) of retired players said that they have at some point ‘experienced significant losses in business or financial investments.’”
None of this was a direct comparison to the number cited in my article, not even close. But said PR person nevertheless included his own note, colored in bright red font: “a far cry from 78 percent.”
The NFL has already convinced thousands of men to devote themselves to the pursuit of a lifestyle that is unsustainable at best and fictional at worst. Some of the enablers of this dream include, but are not limited to: the league’s financial literacy programs, which have historically failed to instill basic principles; the NFL Players Association’s certification program for financial advisers, which supposedly vetted a number of moneymen who reportedly allowed players to lose more than $300 million in recent years; and the players themselves, who are pressured to exaggerate the opulence of their existence.
Even academics are susceptible. “I don’t think there’s really a problem with either of our numbers,” Carlson told me. “They’re really measuring different things.”
That last sentence isn’t a great headline, no. But it’s 100 percent true.
