Last Saturday, the Mississippi State Bulldogs — ranked No. 1 in the AP Poll, the Coaches Poll and the College Football Playoff committee’s rankings — lost to the Alabama Crimson Tide 25-20. The loss ruined the Bulldogs’ quest for their first undefeated season since 1940, when they went 10-0-1.
But under college football’s new playoff system, which will select four teams to contend for the national championship, a late-season loss by a No. 1 team isn’t as devastating as in years past. Mississippi State, for instance, fell only to No. 4 in the committee’s rankings after it lost to Alabama.
It’s less clear whether the Bulldogs control their own destiny. They’ll have another chance to impress the committee when they travel to Oxford, Miss., on Nov. 29 to play No. 8 ranked Mississippi. But they probably won’t play in the SEC Championship and could be leapfrogged by a team (such as Baylor or TCU from the Big 12) that runs the table.
The permutations can get intense. To forecast a team’s likelihood of making the playoff, you not only have to account for all plausible outcomes of upcoming football games but also how the set of humans who make up the selection committee might react to those different outcomes. We’re in the midst of investigating these questions and will have some results for you soon.
But we’ll start, in this article, with a simple cut of the data: What’s happened, historically, to the ranking of No. 1 teams like Mississippi State when they lose? Could Alabama, the new No. 1, remain in the top four if it loses one of its remaining games?
Because the playoff committee is new, we’ll be looking at the historical results from the Coaches Poll for guidance. (We figure the Coaches Poll might be a better proxy for the playoff committee than the AP poll since the committee leans heavily on athletic directors and former coaches.) Here are all the teams since 2002 to have lost a game while ranked No. 1 in the Coaches Poll:
No. 1s that lost a regular-season game wound up anywhere from No. 4 to No. 10 after their defeats. That shouldn’t give much comfort to Alabama. Mississippi State’s case was unusual; most regular-season losses knock a No. 1 team out of the top four.
What accounts for the different outcomes? Well, it’s complicated. The humans voting in the polls are a little forgiving if the loss comes against another ranked opponent. (No. 1s that lost to a ranked opponent fell to No. 6 on average, versus No. 8 for those that lost to unranked teams.) The margin of defeat may matter some as well. The No. 1s that lost regular-season games by the largest margin — Alabama and Ohio State on consecutive weekends in 2010 — fell to No. 8 and No. 10, respectively.
Another factor is whether the year features a deep field of contenders. Part of Ohio State’s steep drop in 2010 may have been because there were an unusually high number of undefeated teams ranked just behind them at the time.
Human voters have historically been more forgiving of losses in the conference championship: No. 1s to lose there have fallen to only No. 4 on average. That could be good news for Alabama. Still, this is a small sample with just three examples. Furthermore, the playoff committee claims it will put a particular emphasis on conference championship results.
As I said, we’ll be making an effort to sort all this out. It will necessarily come with a lot of probabilities and approximations — this is the first year of the new system and we’re not expecting to identify hard-and-fast rules.
There is one rule, however, that has almost always been true. It’s a simple one: A team can’t gain ground by losing.
The chart, below, shows what happened to teams ranked throughout the top 25 in the Coaches Poll after they lost a game. (In the chart, read the vertical axis to find a team’s original ranking, then scan across to find the red line, which shows the team’s expected ranking after a loss. The dashed line represents what would happen if a team held its previous position.) Of the 1,133 ranked teams to lose since 2002, only 22 retained their original spot in the rankings. And even fewer — just five teams — improved their position in the poll.
This may not be a surprising result, but it tells us something about how human voters react to college football outcomes. As I mentioned, the evidence suggests that voters pay some attention to margin of victory. A definitive win might get a little more credit than a narrow one.
But what voters almost never do is reward a team because it loses by less than expected. Say, hypothetically, that No. 24 Gotham Tech travels to No. 3 Gotham State’s home stadium and loses on a last-minute field goal despite having been a 17-point underdog. Your esteem for Gotham Tech should probably improve: That’s a much better showing than you really had the right to expect. But a team’s standing in the polls has almost never improved after an outcome like this; the team has just been punished less. (In this respect, human voters seem to behave a lot like our version of NFL Elo ratings, which account for margin of victory but always prioritize a win over a loss by any margin.)
Nor do teams seem to be demoted after narrower-than-expected wins. Take the non-hypothetical case of the 1995 Texas A&M Aggies. On Oct. 14 of that year, they won by just 3 points at home against the SMU Mustangs, an awful team that would go 1-10 that year. If there were ever a time to punish a team for a bad win, this was it. But the Aggies held their position at No. 22 in the polls.
The next chart shows the data for ranked teams after wins since 2002. There are a few cases where a team lost ground in the Coaches Poll despite winning — but from what we can tell these were mostly cases where a team was leapfrogged by another that won in more impressive fashion. (That happened to Florida State earlier this year, for example.)
There are some other interesting characteristics in these charts. After a loss, teams fall more positions in the poll if they are ranked lower to begin with. While No. 1-ranked teams fall five spots, on average, after a loss, teams ranked No. 15 fall eight positions. This may reflect the fact that teams are more closely bunched together toward the bottom of the Top 25 than toward the top, which is what you’d expect if team skill levels abided by a normal distribution.
And a ranked team’s position falls more after a loss than it improves after a win. For example, when the No. 10 team wins a game, it improves only to No. 9, on average. But the same team falls to No. 17 after it loses.
This is mainly a reflection of the fact that ranked teams are expected to win. A successful college football season is mostly a matter of running the gauntlet and avoiding upsets. The playoff system gives teams more slack, but not much.