Don’t Blame The Refs For All Of These Replay Reviews

The first thing to know is that you’re not imagining it. Instances of replay reviews have, indeed, spiked significantly in the NBA bubble at Walt Disney World.¹ Reviews jumped about 7 percent from the pre-hiatus portion of the regular season to the seeding games, and they’re up another 20 percent from there during the playoffs.²

The second thing to know about the spike is that it’s not being orchestrated, according to Monty McCutchen, a longtime referee who is now the NBA’s vice president of referee development and training. “It’s not from a directive from me,” McCutchen told FiveThirtyEight. In other words, the league has not instructed referees to review plays more often to ensure a greater degree of call accuracy inside the bubble.

The third thing to know, of course, is why.

Currently, NBA play-by-play logs categorize reviews three different ways. There’s “altercation,” which is self-explanatory. When, say, Marvin Williams of the Milwaukee Bucks and James Ennis of the Orlando Magic get into a tussle and have to be separated by some combination of players, coaches and officials, the refs head to the replay monitor to see if any punches were thrown or if anyone made contact with a referee, and determine whether to assess technical fouls.

Then there’s “challenge,” a new option for coaches this season. For example, if Pascal Siakam picks up his third foul at the 7:44 mark of the second quarter of Game 3 in a series where the Toronto Raptors are already down 2-0, and Raptors coach Nick Nurse believes the foul should be on Boston Celtics forward Grant Williams instead, he can call a timeout, then light up the little green light at the end of the bench to trigger a review. (In this particular case, an unsuccessful one.)

The third classification is tagged as “request,” meaning the game officials themselves requested the review, and that type of review can be triggered in 15 different ways. Notably, any flagrant foul called on the floor triggers an automatic review in which the referees can either confirm the call on the floor, upgrade a flagrant one to a flagrant two (or downgrade from two to one) or downgrade a flagrant to a common foul. Referees can also review any common foul that they feel could be upgraded to a flagrant.³ Any of the other 14 triggers requires what McCutchen calls “doubt on the part of the referee.”

To explain the concept, McCutchen flashed back to Game 1 of the 2018 NBA Finals between the Golden State Warriors and Cleveland Cavaliers, and specifically to a contested block-charge call involving Kevin Durant and LeBron James.

“When [the viewer] is watching this, and now he knows he is supposed to be looking at LeBron’s feet, and now he’s only concentrating on that when he’s watching the replay, and LeBron’s feet are 8 inches outside the restricted area, everyone goes, ‘Oh my God! That’s so obvious that he’s out of the restricted area,’” McCutchen said. “But when you’re also worried about a travel on Kevin Durant, a blocking foul, or whether he’s gonna hit his arm, or whether someone in the post is holding another player, you’re not exactly always looking at what we end up looking at on TV and replay. And therein lies how doubt is created in a referee. Once we know what to look for, things become obvious. When you’re concerned with all things at one time, things become less obvious — and that’s what constitutes doubt.”

It’s those doubt-inspired reviews that tend to agitate viewers — or announcers, in the case of ESPN analyst Jeff Van Gundy. The league is cognizant of the fact that constant reviews (particularly late in games) can be bothersome, and they are studying the issue. But according to McCutchen, that issue doesn’t affect a referee’s decision on whether to review a specific play.

“We need to get those plays right,” he said. “If [we go to replay review] six times and you don’t go on the seventh, and the seventh is the one that you screw up the game, all anyone is gonna talk about is the seventh time. They’re not gonna talk about the 15 minutes it took to play the [last minute of] the game.”

The decision to review a play is also not subject to the whims of what ESPN’s Kevin Arnovitz recently called the “small claims court,” where players and coaches continuously lobby referees to change calls and implore them to review a call they believe is incorrect. “A player or a coach might genuinely create doubt in you. But in and of itself, a player or a coach trying to create doubt or passionately believing they are correct does not necessarily mean you should go to replay,” McCutchen said. “That being said, if you have the potential for a trigger, and you’re not honest with yourself about there being doubt, and you end up being wrong — then you do run the risk of being held accountable to that.”

While officials are held accountable for their calls on the floor, they’re not fined for judgment, according to McCutchen — only for “misapplication of rules or behavior.” So referees face no additional incentive review a call just to make sure it is correct, and thus improve their grade, which can affect their playoff game assignments.

So, about the playoffs. Reviews are up approximately 20 percent from the seeding games to the playoffs, and they’re up around 28 percent in the playoffs compared to the pre-hiatus regular season. But that’s actually not abnormal. In each of the past two seasons, the league saw a similar spike in reviews during the postseason — an average of about 20.8 percent.⁴ But this year’s spike started from a higher level: Pre-hiatus reviews this year were already up from the previous two regular seasons, and they jumped even more in the seeding games.

Explaining why the rate of reviews tends to rise in the playoffs is fairly easy. “The importance of the moment creates more doubt,” McCutchen said. “And I think that’s just a human nature thing. You can understand if you’re in Game 6, and someone’s going home, you better be right on the play.”

Part of the reason the baseline review rate was higher this year, though, is because of the introduction of the coach’s challenge. Non-challenge replay reviews per game were actually down during the pre-hiatus regular season compared to the past two regular seasons, and that continued through the seeding games. Just about all of the seeding-game bump can be explained by increased use of the coach’s challenge.⁵

In other words, that spike you’re seeing in replay reviews isn’t the fault of the refs. It’s being caused by the coaches. (In particular, it’s being caused by Eastern Conference coaches. Four of the top five teams in postseason reviews per game are the four teams that made the Eastern Conference semifinals.)

It’s worth asking, then: Is it worth it? Are coaches even particularly good at challenges? Do challenges result in overturned calls any more often than requests from the referees themselves? It depends on what your definition of “worth it” is, but coach’s challenges have certainly resulted in more instances of overturned calls than have requests.

The NBA has three classifications for the result of a coach’s challenge or non-altercation replay request): “Overturn” means the call on the floor was incorrect, and the ruling is reversed; “support” means the call on the floor was correct; and “stands” means there was insufficient evidence to overturn the call on the floor. In the seeding games, the rate of calls receiving each of those designations was pretty similar to where it was during the regular season. In the playoffs, calls have been more likely to get the “support” ruling than “stands,” indicating slightly more confident review decisions.

All of this is on the NBA’s radar. The league is studying what’s happening in the bubble just as much as those outside of it are, and that includes looking at the replay system.

“We’re using this as an opportunity to really understand our game at a higher level,” McCutchen said.

Check out our latest NBA predictions.