With the NHL playoffs starting this week, all eyes will be on the goaltenders, hockey’s masked men. A goalie’s time on the ice is a solitary existence, with flurries of activity punctuating long periods of inaction. Hockey has been called the ultimate team sport, but all too often the goaltender finds himself alone as the puck approaches.
Because the goalie is a team’s last line of defense, it’s no surprise that strong performance in net is incredibly important to winning a hockey game. During the regular season, save percentage (the generally accepted shorthand measure of goaltending effectiveness) explains a higher proportion of team performance than any other fundamental factor in hockey. In the playoffs, the emphasis on goaltending only intensifies; save percentage is easily the most important determinant of a team’s goals-per-game differential in the postseason. A hot goalie really is the key to a successful playoff run.
But herein lies a great paradox: Despite goaltending’s outsize impact on the outcomes of hockey games, it’s extremely hard to say exactly which goalies are truly good or bad at their jobs.
This perplexing point was raised by the authors of the 2010 book “Stumbling on Wins,” and it still stands today. Using Hockey-Reference.com’s adjusted version of the save percentage statistic, adjusted Goals Against Percentage (GA%-), the correlation of goalie performance from year to year is so low that, in practical terms, only 30 percent of the difference we see between a goalie and the league average in any given season actually “belongs” to the goalie himself. The rest is just random.
The poor correlation of save percentage from one year to the next also indicates that goalies are extremely volatile commodities. For instance, if a goaltender is above average in a given season, there’s only a 59.2 percent chance he’ll be above average again the following year. And if he’s below average now, don’t worry: There’s a 47.2 percent probability that he’ll be above average next season.
Take Brian Elliott of the St. Louis Blues. During the 2010-11 season, Elliott was the NHL’s second-worst qualified goalie — only Nikolai Khabibulin was less effective at stopping pucks — and in 142 career games he had a lifetime GA%- of 111 (which translates to 11 percent worse than league average). If any goalie seemed unlikely to play well in the future, it was Elliott, but the very next year he led the NHL with a 69 GA%- (31 percent better than league average), at the time the third-best single-season performance by any goalie since the NHL started tracking save percentage in 1984. And how did Elliott follow that brilliant campaign? By posting a below-average 106 GA%- last season, and a 90 GA%- this year.
It’s the kind of thing that will spin you around faster than a Pavel Datsyuk deke.
So is Elliott a good goaltender or not? We can say he is probably a slightly below-average netminder who happened to have an all-time outlier of a career year in 2012. But that’s just because we have a relatively large amount of data on him by now. His career GA%- is 103 after nearly 250 games and 6,000 shots faced. A goalie’s save percentage only begins to stabilize after facing around 3,000 shots, at which point we would expect half of his observed performance to be talent (the rest is still luck). The busiest goaltenders each year face roughly 2,000 shots, so it takes about a season and a half for GA%- to offer insight on even the biggest goaltending workhorses.
This does not mean that there is no difference in talent among goalies. It just means there’s a great deal of uncertainty around how any one goalie compares to another, and that the distribution of talent among NHL-caliber goaltenders is significantly more narrow than would be expected from looking at season-level save percentages alone. As a consequence, the “replacement-level” save percentage for goalies (to borrow a term from baseball’s sabermetrics, referring to the production a team could expect from a minimum-salary player freely available on the waiver wire) is remarkably close to league average. This, too, is a product of the uncertainty surrounding the true talent level of any given goalie — with such high levels of volatility, teams don’t need to accept bad goaltending performances for long. Given what little information we have about any goalie’s actual talent, a backup is almost as likely to give above-replacement production as a struggling starter is.
If chance overwhelms skill in an entire regular season’s worth of goaltending statistics, imagine what can happen in the playoffs, when the leading goalies face but 800 shots at most. The Vancouver Canucks have experience with this: Kirk McLean, a nine-year veteran with a perfectly average 100 career GA%- going into the 1994 playoffs, backstopped the team to within a win of the Stanley Cup on the strength of a stellar 78 GA%- in the postseason. So do Capitals die-hards: Olaf Kolzig led Washington to the Finals in 1998 with a playoff GA%- that was 27 points lower than his career average. And Mike Smith nearly did the same for the 2012 Phoenix Coyotes. Playoff history is littered with seemingly nondescript goaltenders who suddenly became incredible puck-stoppers come springtime.
But history can also cut the other way. In 2001, the great Patrick Roy had a regular-season GA%- of 90 — 13th in the league, if slightly down from his peak numbers of a few years earlier — and during the playoffs he had one of the best performances of his career with a 75 GA%-, leading the Colorado Avalanche to the championship. The following year, Roy was quite a bit better during the regular season (81 GA%-), ostensibly setting himself up for another strong playoff bid. So what happened next? Roy put up a terrible 110 GA%- in the playoffs, capped off by an embarrassing, season-ending loss to Detroit in which he allowed six goals before being pulled from the game in the second period. In the minuscule sample of the playoffs, even Hall of Famers are at the capricious whims of variance.
It’s something to keep in mind during this year’s playoffs. Just as we found the correlation for regular-season GA%- to be quite low from year to year, the correlation between a goalie’s regular-season and his playoff GA%- is even smaller, as is the correlation between his previous career GA%- and playoff GA%-. We can’t predict who will fluctuate, just that somebody likely will.
It’s not just goalies who are unpredictable; hockey’s stats holy war over shot quantity versus shot quality has shown us that an offense’s shooting percentage is just as inconsistent. The whole sport is especially vulnerable to random fluctuations, something that shows up most once a puck starts moving towards the net.
In 2008, an Edmonton fan named Brian King left a comment at a now-defunct Oilers blog suggesting that the best way to understand luck in hockey was to look at a team’s shooting percentage on offense and the collective save percentage of its goalies. If you combined those stats for a team, and compared them to the league average, you could tell whether a team had been lucky or unlucky — and how far it had to go to regress to the league’s mean. The metric became known as PDO (its namesake was King’s online pseudonym), and the more it strays from its baseline of 1.00 (above 1 means lucky; below, unlucky), the more likely the team’s record, and even its goal differential, has been tainted by randomness.
A great deal of recent hockey research has shown that, if given a large enough sample, every team’s PDO will more or less regress toward the league average of 1.00. The big implication of PDO is that a team has virtually no long-term control over shooting percentage, just like it can’t predict the efficacy of its goalie. The key to good defense, then, is simply to keep the opposition from shooting, because it’s impossible for a goalie to maintain a consistently high save percentage.
PDO can now tell us the extent of hockey’s chaos, but goaltenders have always grappled with the randomness of their position. In his classic book, “The Game,” the Montreal Canadiens’ Hall of Fame goalie Ken Dryden wrote that a goaltender’s mental focus is key:
If you were to ask a coach or a player what he would most like to see in a goalie, he would, after some rambling out-loud thoughts, probably settle on something like: consistency, dependability, and the ability to make the big save. [ … ] Because the demands on a goalie are mostly mental, it means that for a goalie the biggest enemy is himself. Not a puck, not an opponent, not a quirk of size or style. Him. The stress and anxiety he feels when he plays, the fear of failing, the fear of being embarrassed, the fear of being physically hurt, all are symptoms of his position, in constant ebb and flow, but never disappearing. The successful goalie understands these neuroses, accepts them, and puts them under control. The unsuccessful goalie is distracted by them, his mind in knots, his body quickly following.
Stats portray the goalie’s job as a nihilistic one. Chaos mounts, pucks fly, muscles react. What happens beyond that is so random that, as Dryden writes, the only way for a goalie to cope is to focus on what’s immediately in front of him: a stretch of ice with an ever-changing landscape of variables.
