This afternoon the NFL released the results of an investigation into whether or not the New England Patriots intentionally deflated footballs below league standards. The report was so sensational that we had to gather some of our best sports minds to talk about it. Here’s an edited transcript of our Slack conversation.
cwick (Chadwick Matlin, senior editor): The NFL finally has its own version of the Starr Report. After a months-long investigation and hundreds of pages, a law firm hired by the NFL has found that the Patriots most likely did tamper with their footballs, and that Tom Brady was likely “generally aware of the inappropriate activities.” The report itself is sensational, full of amazing details about the bathroom habits of low-level assistants, furtive trash talk about Tom Brady, and a statistical appendix that seems designed for FiveThirtyEight to dissect. Let’s all assemble and talk about it! What stuck out most?
andrewflowers (Andrew Flowers, quantitative editor): The report — especially the stat-sy appendix — went to great lengths to show that the difference in pressure between the Pats’ and Colts’ footballs was not due to chance.
benm (Benjamin Morris, sportswriter): I would say that the report hammers home the stats and science sides of this extremely well so as to head off any skepticism on those fronts, but they’re relatively easy cases to make. You don’t need a stats degree to look at that table and see that something is amiss.
The salacious parts to me are the surrounding facts: Who was responsible, how long has this been going on, what did Tom Brady know and when did he know it, etc? It has a lot more to say on those subjects than I would have guessed.
neil_paine (Neil Paine, senior sportswriter): Yeah @benm – I was surprised they had the texts, which referred specifically to Brady being on the guy’s case about ball pressure.
benm: And they present pretty good evidence that this was a deliberate rules violation for the purposes of gaining an advantage for Tom Brady and the Patriots organization, AND has likely been going on for some time.
benc (Ben Casselman, chief economics writer): I am an unapologetic Pats fan, and would name my firstborn Brady if my wife would allow it. But one question I had was how clear it is that Brady wanted the balls UNDER 12.5 psi vs. as close to the minimum as possible. He seems to have complained about the balls in the Jets game, but those seem to have been inflated well over 12.5.
neil_paine: @benc – Right, they did complain the balls were at 16 one game.
benm: Agreed the evidence about Brady (at least from 30 minutes of dissecting it) seems a little more thin than evidence against McNally and Jastremski. [Jim McNally is the Officials Locker Room attendant for the Patriots, and John Jastremski is an equipment assistant for the team.]
neil_paine: Another interesting note is that it was the first time in 19 years Walt Anderson [the head NFL referee for the game] couldn’t account for the balls. In the other games this was happening, who was the referee and what did he notice?
benc: That feels to me a bit like the kind of thing you notice only after the fact though. Like, how sure are we Anderson would have noticed if he couldn’t find the balls on other occasions where no one raised questions?
cwick: Left unaddressed is whether deflated balls had been helping the Patriots win — do we have any better sense for that now than we did before? Because before we thought this was isolated to just the Colts game, which they won handily. But now it seems like this was about Tom Brady’s preferences in more than just that game.
benm: I think the much-maligned study by Warren Sharp about the Patriots having a low fumble rate should be taken more seriously, for sure. I mean, though it had flaws, at a very minimum that author correctly identified that the Patriots fumble rate has been absurdly small. I did my own calculations using binomial and Poisson models and found the same.
benc: NOW this is a 538 chat!
benm: But the fun part is when you get all Bayesian about it. As I said at the time, the existence of the Patriots’ extremely low fumble rate, as a Bayesian matter, makes it much more likely that the Patriots were intentionally cheating – even though the link between fumble rates and inflation levels is only speculative. That’s the beauty of Bayesianism. But it gets better: Now that it seems likely that the Patriots were violating the rules to gain an advantage, the fact that they also had an extremely low fumble rate makes it more likely that the relationship between inflation levels and fumbling is real – and more likely that the Patriots have materially benefited from their cheating.
cwick: @benm “That’s the beauty of…” is a phrase that’s been written so many times. That may be the first time the sentence ended with “Bayesianism.”
neil_paine: Agreed that some of those outlier Patriots stats deserve a second look – although it bears noting that the report specifically exonerates [head coach Bill] Belichick, so this doesn’t appear to be part of a grander conspiracy involving the entire team. (Certainly the team beyond Brady could benefit, though.)
One question regarding the fumbling is whether non-QB players coming to the Patriots from other teams noticed a change in ball feel.
benm: Agreed, the report seems to be focused primarily on these two Pats personnel, with a looser link to Brady.
neil_paine: If I recall correctly, the WOWY (with or without you) evidence with players going to/from New England wasn’t very convincing after removing special teams plays.
benm: But what do you do in a situation where the team benefits from one person in the organization bending rules? USC has lost national championships for less.
neil_paine: Right, was going to say, that’s the kind of moral question that often haunts college teams forced to vacate wins/titles. (And unrelated players are faced with future bowl bans.)
benm: And again, just to be clear. I’m not saying the Pats did gain some huge advantage from this. But the odds of that have gone up substantially.
cwick: ok — can we talk about the data appendix for a minute, because I was floored by the detail.
benm: Indeed, the science-y stuff “floored” me more than the stats.
cwick: To your eyes, is this the most high-profile statistical model of the year? and were you impressed by it?
andrewflowers: What I love about this report is the various extents the researchers went to make their analysis iron-clad: the natural experiment of comparing the Colts (control group) and Pats’ footballs; the statistical inference (p-values!) in examining the differences in the pressure changes; and the hard-science engineering to test the environmental effects. I mean, just look at this photo…
andrewflowers: Someone also apparently “simulated” whether it’s possible to deflate 13 footballs in under 1 minute and 40 seconds. Apparently it is.
benm: Honestly, it’s probably overkill. But that is, in a sense, a product of the incredibly strong statistical case. So I kind of give them credit for going so far to address alternate possibilities. A model a lot of stats people could probably learn from. I mean, once it’s clear that this didn’t happen by chance, the discussion moves to: So what explains it.
benc: What do we think the odds are these guys were getting paid by the hour?
andrewflowers: Very high.
benm: Law firm: Yes — paid by the hour.
benc: Because this has a definite, “Hey, is there another test we could run?” element to this.
benm: And you think FiveThirtyEight is nerdy.
andrewflowers: Honestly, today’s report is the best thing for the “Ideal Gas Law” since Bill Nye the Science Guy aired.
cwick: Final question: What stat analysis does this report make you want to run out and do?
andrewflowers: @benm’s point about the relationship between fumble rates and ball pressure.
benc: Chart of “humor of references to Tom Brady’s balls” over time.
benm: There’s definitely more to be done on the Patriots fumbling to isolate for the fact that they were the most consistently winning team, the types of plays they ran, their personnel, how that period fits in historical context (have other dominant teams had similarly bizarre quirks), etc.
neil_paine: And perhaps another look at Brady from 2007 onward in general. That season was very much out of step with his previous performance. And what followed was very different as well. Do other QBs suddenly get that much better, at that age?
benm: WOWD: With or Without Deflation.
neil_paine: Lol
benm: Even if org isn’t culpable, it’s a pretty massive shift in the conversation about Pats. Suddenly we’re faced with the possibility of having to discount some amazing sports accomplishment. Again.
neil_paine: “Behind every great fortune is a great crime.”