Major League Baseball’s playoffs begin Tuesday with the American League wild card game between the Kansas City Royals and Oakland Athletics, and chances are you’ve heard or read some version of this aphorism over the past few days:
“Baseball’s playoffs are a crapshoot.”
It’s kind of true. MLB’s postseason — some call it a “gauntlet of randomness” — tempts with a million narratives that seem to legitimately explain why some teams rise and others fall in October. But most attempts to solve the playoff puzzle have failed. (To wit: It once looked like Nate Silver, FiveThirtyEight’s editor in chief, had figured out the “secret sauce” that determines postseason success, but recent results have led Baseball Prospectus to retire his metric.)
The only thing practically every study of the postseason has in common is that a team’s overall regular-season performance, and little else, matters when predicting its playoff fortunes.
However, despite having 162 games per team, baseball’s regular season is quite short relative to other sports’ in terms of the amount of information conveyed by its standings. (Several years ago, I determined that MLB teams would have to play 610 games apiece for their win-loss records to offer as much certainty as the NBA’s 82-game schedule.) This is why baseball records must still be regressed about 30 percent of the way toward the mean to best approximate team talent.
In predicting the postseason of another sport with a chaotic regular season — NCAA men’s basketball — FiveThirtyEight found that a team’s preseason reputation (as measured by its preseason AP and Coaches’ Poll rankings) carries some vestigial predictive power even after accounting for regular-season performance. All else being equal, surprising teams that rise from preseason obscurity to enjoy successful seasons tend to have shorter tournament runs than teams with similar records that were regarded more highly before the season. Is it possible the same effect exists in baseball?
To test this, I gathered preseason Vegas over/unders and Baseball Prospectus’s PECOTA projected records back to 2005, then plugged those projections — and regressed versions of each team’s regular-season record — into a logit model attempting to predict the outcomes of every playoff game from 2005 to 2013. In accordance with every other failed study looking for magic postseason bullets, it appears that preseason projections don’t matter a bit when predicting the playoffs. I tested PECOTA and Vegas separately and together as a composite prediction, and neither was a significant predictor after controlling for a team’s regular-season performance.
That’s good news for the Royals and Baltimore Orioles. Before the season began, neither was expected to crack .500 by the bookmakers or computers. And yet the Orioles ran away with the AL East and the Royals clinched a hard-fought wild card berth thanks to their best win total in 25 years. Despite the lack of confidence shown by prognosticators before the season, we should take their surprisingly great seasons at face value. (The same goes for the Pittsburgh Pirates, who made the playoffs after PECOTA looked down on them before the season.)
All told, it’s just another case of the postseason defying simple explanations. That’s why the modern statheads’ best approach might be to embrace the chaos and appreciate the playoffs for what they are: an entertaining tournament among baseball’s top teams, not a scientific experiment designed to definitively identify the best team in a given season.