Billion-Dollar Billy Beane

The film version of “Moneyball” depicts many establishment baseball types as ignorant of where wins in baseball come from and clueless about how to properly value talent.

Take, for example, the scene when John Henry — the billionaire owner of the Boston Red Sox — tries to recruit the Oakland Athletics’ general manager Billy Beane. Henry tells Beane that any managers not rebuilding their teams with Beane’s system in mind are “dinosaurs,” and then hands him a slip of paper. On it, there’s an offer for Beane to become the new Red Sox general manager for the insane amount of $12.5 million over five years. His fictional colleague tells us that the offer would make Beane “the highest-paid GM in the history of sports.” Despite appearing tempted, Beane ultimately declines the deal, claiming, “I made one decision in my life based on money and I swore I’d never do it again.”¹⁷

Beane may not be the highest-paid GM in the history of sports, but he may be the most famous. An outfielder originally drafted 23rd overall by the New York Mets in 1980, Beane made his MLB debut in 1984, but was never successful against top competition. After getting washed out of the league, he became a scout for the A’s and eventually worked his way up to GM in 1997. As GM, he has used Bill James-style advanced statistics to inform his decisions, and taken a strictly economic approach to valuing and acquiring players. Under his leadership, the A’s have been a very successful franchise despite routinely carrying one of baseball’s smallest payrolls. Beane’s story caught the attention of author Michael Lewis, who made him the central character in his 2003 bestseller “Moneyball” and something of a cultural icon for sports analytics.

Beane’s methods continue to be analyzed and celebrated by sabermetricians, and the A’s continue to massively exceed expectations given the amount they spend. They own the best record in baseball so far this season, and have the fifth-lowest payroll.¹⁸ It’s the best 100-game start of Beane’s career, and the best for the organization since its 1990 pennant-winning squad. Over the last 15 seasons,¹⁹ the A’s under Beane have had the fifth-best winning percentage in baseball, with the fourth-lowest total payroll. (The data used here is current through Monday, July 21.)

Beane has been a godsend to the frugal A’s, enabling them to achieve top-tier performance at bottom-tier prices. For this, the A’s have paid him fairly modestly²⁰ — but since we don’t know how much winning is worth to the A’s organization, it’s hard to say exactly how much Beane has been worth to them.

For a team like the Red Sox, however, the picture is much more clear. Over the last 15 years, they’ve happily spent over $2 billion in the pursuit of wins — and because they’re one of baseball’s most successful franchises, no one in Beantown is complaining.

From a strictly economic perspective, not offering Beane however much money it took to get him may have been one of the Red Sox’s poorest decisions since letting Babe Ruth go to the Yankees for next to nothing. And I mean that literally: Over the past 15 years, Billy Beane has been nothing less than the Babe Ruth of baseball GMs. The Red Sox offered Beane $2.5 million per year,²¹ but even $25 million would have been a bargain.

Finding Beane’s potential dollar value to the Red Sox is relatively simple: It’s the amount the team spent under general managers Theo Epstein and Ben Cherington, minus the amount it would have had to spend for the same performance with Beane as GM.²²

To show this, we first we need to figure out just how many A’s wins Beane has been responsible for, and how much those wins would cost on the open market.

Let’s start by comparing the A’s performance under Beane’s leadership to the performance we would expect from a typical GM with the same payroll.²³ I created a logistic regression model²⁴ that predicts a team’s win percentage by season based on the team’s relative payroll (excluding Oakland from the data), as measured by how many standard deviations it was above or below the average MLB payroll for each season. Below, I’ve plotted the non-Oakland team-seasons from 2000 to 2013 (on which the model is based) in groups of 15 by payroll (so, the dot farthest to the right represents the 15 team-seasons with the highest relative payrolls), and plotted the model’s prediction as a red line. I then plotted Oakland’s 15 seasons through 2014 as a single green point:

The point on the upper right represents the 15 team-seasons with the highest relative payrolls. These teams were 2.68 standard deviations above the mean payroll on average and won 58.5 percent of their regular-season games on average.²⁵ Oakland, on the other hand, averaged .81 standard deviations below the mean payroll and won 54.8 percent of its games on average.

From this we can take each team’s expected wins per season based on payroll,²⁶ and then see how many games above or below average it ran. Here’s Oakland, broken down by year (Note: 2014 is through the season’s first 98 games only):

This comes out to 180.2 wins above expectation given the A’s payroll (165.5 prior to this year). That’s 12.0 wins above expectation per season (and there’s a good chance of that per-season average rising).

“Wins above expectation” may sound familiar to you. It’s conceptually very similar to wins above replacement (WAR), the stat we use to evaluate how many wins a player earns a team versus how many games that team would expect to win without him.²⁷

Beane’s 12 wins per season above what we would expect of an average general manager is slightly more wins than Barry Bonds earned when he hit 73 home runs in 2001 (11.9 WAR). The most WAR earned by any batter over his entire career was 163 by Babe Ruth.²⁸ In fact, if you assemble the top 15 position player seasons of all time, they still trail Beane’s 15 seasons as GM, with 180.1 WAR combined versus Beane’s 180.2 wins above expectation.

No one can get that lucky. If you’re expected to win 1,116 out of 2,364 games, winning 1,296 games instead may not look impossible, but that’s because our intuitions about these things are terrible. Excel’s binomial distribution function makes calculating such odds pretty easy:²⁹ In this case they’re somewhere around one in 13 trillion — effectively zero.³⁰ Of course, we can’t know to what degree Beane alone is responsible for the A’s success. But as GM, Beane is formally responsible for the A’s performance, and there aren’t any other obvious causes that would suggest he isn’t responsible (there have been several different managers and 100 percent turnover of players during Beane’s tenure).

Imagine the A’s wanted to have exactly this level of success and were willing to pay whatever it cost. With Billy Beane, the A’s have paid $839,902,108 to their players from 2000 up to and including the start of the 2014 season (but prior to recent acquisitions). How much do other teams normally have to pay for this level of success?

There are a lot of estimates for the price of wins out there, ranging from ESPN’s Dan Szymborski’s $5.5 million per marginal win and FanGraphs $6 million on the lower end to Lewie Pollis’s $7 million and up to Hardball Times’ $7.6 million on the high end. To make things a little more complicated, the price of wins has also risen substantially with the growth of payrolls in the last decade³¹:

If we use these values to price wins above or below expectation on a year-by-year basis for every team as we did for Oakland above, and then sum up by team, it would look like this:

FanGraphs’ value for Oakland’s performance adds up to $812 million since 2000, while the Hardball Times’ value adds up to $891 million. Over three-quarters of a billion dollars — that’s huge! We can smell-check these numbers by looking at the overall picture. Leaving aside standard deviations and year-by-year breakdowns for a moment, we can see how each team’s total payroll over the last 15 years has compared to its performance:

That trend line shows us how well teams have performed relative to how much they’ve paid, but we can also use it for the reverse:³² The Oakland Athletics have won 54.8 percent of their games, so the corresponding 15-year payroll (the amount we would expect a team to have paid for that win rate) is about $2.02 billion — about $1.18 billion higher than the Athletics actually paid.

So the smell-check turned out a higher number than the estimates based on the normal price of wins, when that normal price already seemed absurd.

This isn’t broken down year by year, so it could just be that the A’s won a lot more in years when wins were cheaper. To correct for this, we need a more empirical method for pricing wins. On a year-by-year basis, how big would each team’s payroll have to have been to buy its performance? Using the regression above (and some fancywork in R³³), we can model this and see that wins may be harder to buy than standard win-valuation models (FanGraphs, Hardball Times, etc.) would suggest. Valuing each team’s relative season-by-season performance this way leads to a very different accounting from above:

Over the past 15 years, the A’s have exceeded expectations by close to $1.38 billion — even better than our smell-check estimate of $1.18 billion. This suggests that they’ve performed slightly better in years when they were at a bigger payroll disadvantage (at 2013 market value, those A’s wins would cost closer to $1.78 billion).³⁴

Yes, that’s “billion” with a B. (Or two.)

* * * * *

Now that we have a sense of Beane’s performance and how much it would cost to replicate it, let’s turn back to the Boston Red Sox and their failure to sign him (or even to offer him anywhere near his worth).

The situations in Oakland and Boston aren’t directly comparable. Exploiting market inefficiencies is probably easier for Beane than it is for a successful big-money team, because he has never had to face the winner’s curse or the diminishing returns of spending. On the other hand, the A’s have been way above average, not just a little above average. Aside from the Red Sox’s post-season successes,³⁵ the team has only performed 0.6 percent better than the A’s over the 15-year period — for which they’ve paid an extra $1.2 billion in salaries.

But some of that money was spent and some of those wins came before the Red Sox attempted to hire Beane. To be conservative, let’s just look at the period since Henry made Beane his offer: In the last 12 years, the Red Sox spent $1.714 billion on payroll, while the A’s spent $736 million. We can then break down what it could have looked like if Beane had worked for the Red Sox like so:

• Let’s say it would have cost Boston the same $736 million that it cost Oakland to get the A’s performance with Beane.
• At the hypothetical $25 million-per-year salary I suggested earlier, Beane would have cost the Red Sox another $300 million. (It’s possible that Beane would have wanted more, but it’s even more possible that they could have gotten him for less.)
• The difference in performance between the A’s and the Red Sox over that period (where the Sox were as successful as at any point in the franchise’s history, and the A’s were supposedly stagnating after Beane’s early success) has been about 50 games for Boston. Since we don’t know exactly how good Beane would be at procuring additional wins above his Oakland performance, let’s assume that the Red Sox would have had to pay the typical amount teams have paid for wins in the period to make up the difference. According to the year-by-year price of wins from my calculations above, those 50 wins (taking when they happened into account) would have a market value of about $370 million (though this might have been lower with Beane in charge).

If we combine these — the price of the A’s performance ($736 million) plus Super-Expensive-Billy-Beane’s salary ($300 million) plus the additional 50 Red Sox wins at high market estimates ($370 million) — merely duplicating their previous level of success still would have saved the Red Sox more than $300 million relative to what they actually spent, and that’s with reasonably conservative assumptions. That’s money they could have pocketed, or spent making themselves even better.

In other words, failing to understand Beane’s true value may have cost the Red Sox hundreds of millions of dollars or more. “Moneyball” isn’t just some nerdy obsession that helps a few teams save a bit of money. It’s about more than nickels and dimes; it’s about millions and billions.

CORRECTION (July 24, 7:10 p.m.): A footnote in an earlier version of this story misstated the most recent year the Oakland A’s played in the World Series; it was 1990, not 1991.