Last week, The New York Times published an investigation that said an NFL committee assembled in 1994 to research head injuries had omitted crucial data from its analyses, which were published in a series of scientific journal articles starting in 2003. It’s the latest evidence that the NFL’s scientific endeavors on concussions have been tilted in the league’s favor. These revelations undermine years of research into the effect that playing in the NFL has on the brain.
They don’t necessarily provide evidence that football is dangerous for the brain, though. Science is hard like that.
Confidential data obtained by Times reporters Alan Schwarz, Walt Bogdanich and Jacqueline Williams showed that “more than 100 diagnosed concussions were omitted” from concussion studies — “including some severe injuries to stars like quarterbacks Steve Young and Troy Aikman.” The reporters found that, “Over all, at least 10 percent of head injuries diagnosed by team doctors were missing from the study.”
Despite public sniping with the Times, the NFL does not dispute that its committee’s data set did not include all concussions. That’s one example of the NFL’s biased science. I’ll get to the others in a minute, but let’s first consider why this one undermines the NFL’s findings.
If you want to know how common concussions are or how often they lead to lingering problems, you can’t just count total concussions. You need to know the denominator — the number of players the sample was drawn from — to calculate how often the problem occurs. If concussions were missing from the database, that could make them seem less common in the league than they really are, and it would also mean that rather than painting a complete picture of the league, the 16 published studies (which nodded to concerns about safety but generally downplayed the potential risks) included just a particular subset.
The shady business doesn’t end there. When then-NFL Commissioner Paul Tagliabue created the league’s Mild Traumatic Brain Injury Committee in 1994, he appointed Dr. Elliot Pellman, a rheumatologist with no apparent neurology experience, as its chairman. The committee had strong ties to the NFL — many of its members were employed as team physicians or trainers — so they were essentially tasked with studying and assessing their own work managing concussions. There’s a powerful financial conflict of interest here, but there’s also a philosophical conflict. One of the things the committee was researching was whether their return-to-play protocols were harming players — who wants to discover that they’ve been making bad decisions?
And then there’s the publication process. It’s hard to know whether the papers were submitted anywhere else, but all of them were published in Neurosurgery, a journal then edited by Michael Apuzzo, a neurosurgical consultant to the New York Giants. It’s a legitimate, peer-reviewed journal, but the Times reported that “more than a dozen pages of anonymous back-and-forth between reviewers and the committee show some reviewers almost desperate to stop the papers’ publication while the authors brushed aside criticism.”
Researchers primed to believe that the NFL has concussions under control, a data set that’s missing important information, and publication in a journal edited by a consultant to the NFL — it looks more like an attempt to create evidence for a predetermined message than good science. But even if we throw out these studies, we can’t yet conclude that football inevitably leads to lasting brain damage.
We’ve learned a lot about concussions, brain trauma and chronic traumatic encephalopathy in the last decade. It’s common sense that brain trauma is best avoided, and we know that some people who get concussions go on to have lasting problems. But there’s still so much we don’t know. For example, we still don’t know exactly what a concussion is. We still don’t know that football leads to CTE. And finding answers to the lingering questions will require a lot more research. No one has proven that concussions aren’t worrisome, but neither do we have the science to determine whether or in which cases concussions lead to long-lasting neurocognitive problems. If we want to draw scientific conclusions about the risks associated with head trauma, we need to acknowledge the lingering uncertainties and avoid jumping to unwarranted conclusions.
