It’s Black Monday — the day after the NFL’s regular season concludes — when 20 to 25 percent of teams (usually of the non-playoff-bound variety) have historically begun their offseason by firing (or otherwise parting ways with) their head coaches. This year, the New York Jets have fired Rex Ryan, Mike Smith is out in Atlanta and Jim Harbaugh left the San Francisco 49ers in a mutual split. Further changes may be coming.
Teams don’t take these coaching changes lightly, but for all the focus on the coaching carousel, it’s been difficult for researchers to figure out how much who’s wearing the headset matters.
Teams that change coaches have a strong tendency to improve the following season, which could be taken as prima facie evidence that swapping in a new coach makes a profound difference. But it also could simply be the residue of regression to the mean. A poor record is generally required for a team to consider dismissing its coach, but much of the differences in NFL team records is due to luck and not the comparative skill levels of the teams themselves. When that luck evens out, the team appears to improve, even if its underlying skill didn’t change all that much.
And this phenomena is essentially what the research on NFL coaching changes has found. Although the average team to change coaches since 1994 has seen its winning percentage improve from .383 to .428 the next season, that’s mostly regression to the mean at work. In fact, once we account for the teams’ previous Elo ratings and the inexorable pull that a .500 record exerts on NFL teams from year to year, there’s little evidence that changing coaches helps teams at all.
The aforementioned sample of teams had an average Elo rating of 1437 at the end of the regular season with their old coach, which tends to translate to a .463 winning percentage the following year whether a team changes coaches or not. But the season after making the change, those teams averaged a .428 winning percentage — about 35 points lower than we’d have expected based on their previous Elo ratings. This may speak to broader institutional issues that are correlated with coaching changes but beyond the influence of the coach himself, such as dysfunctional ownership, a poor general manager or players who consistently win less than point-differential-based metrics would predict.
These types of findings lend credence to the theory that NFL coaching changes offer franchises little more than the illusion of control over their future. While it may feel satisfying to fans and owners to fire a coach after a disappointing season, it’s tough to quantify the real benefits of such a move — if any even exist.