While baseball is a statistician’s dream with its series of discrete actions across the field, the development of comparable analytics in football is hindered by the complexity of formations and events — what the president of EdjSports has called “a probability atom bomb” — not to mention the scoring of points in varying increments, yet the ultimate goal is binary: win or lose.
That’s the simple principle underlying the sophisticated simulation computer models that EdjSports provides to more than 10 NFL and college football clients.
With the Tennessee Titans trailing the Kansas City Chiefs 21-3 early in the third quarter of last weekend’s wild-card game, for instance, the EdjSports system pegged the underdog Titans as having a paltry, single-digit win probability. Three unanswered touchdowns turned the tides, but the Chiefs regained the upper hand as favorites after converting a third-and-4 play with a nine-yard pass to cross midfield with about four minutes remaining. Kansas City, however, managed just one yard on the next four downs to hand the ball over and secure the Tennessee win.
“This is a good way of illustrating the way we look at the world and the way a lot of other games are modeled — it’s this tug of war of win probability,” Frank Frigo, co-founder of the Louisville-based parent company, EdjAnalytics, said, adding: “If you think about what we produce in these modules, it’s essentially a price ticker on the game, just like a stock chart. It’s how that win probability is fluctuating in every increment of the game.”
Some of the value EdjSports offers its clients is in these postmortem autopsies of each game, helping coaches revisit decisions from the previous week and, using the company’s carefully calibrated EPI scores (Edj Power Index), prepare for the next opponent. EdjSports projects Tennessee has only a 29.6 percent chance of winning in New England on Saturday night, with none of its four units — offensive pass and rush, defensive pass and rush — having any appreciable advantage over its Patriots counterpart.
EdjSports’ model is built using nearly two decades of NFL play-by-play data, running about 7 billion simulations during the season. Frigo said the company is “very eager” to incorporate the in-game tracking data gleaned from the Zebra Technologies chips placed on every player’s shoulder pads and under the laces of every game ball, but their initial conversations seeking access have not yet succeeded.
“I think that we’re uniquely situated because what we inherently do is build tools around giant and massive data sets,” said EdjSports president Tony DeFeo, who worked in San Francisco 49ers front office before becoming the University of Michigan’s head of football operations and analytics. “Let’s face it, what is a team going to do with that information? It’s massive amounts of information, but it’s information, not insights. It’s not actionable intel.”
The genesis of the EdjSports model came from a collaboration of Frigo and Chuck Bower, an Indiana University astrophysicist. The two are both world-ranked backgammon players; Frigo won the 1994 world championship, and Bower sits on the U.S. Backgammon Federation’s board of governors. In 2006, they built a football model on a lark and named it Zeus, even co-authoring several strategic analyses for the New York Times. After Frigo helped start EdjAnalytics — which also does work in health care and with hedge funds, among other sectors — the team updated Zeus and began consulting for teams in the last few years. Prior to the 2017 NFL season, EdjSports launched the full EdjFootball product and is developing individual player evaluation modules.
Frigo said, at its core, EdjFootball is “a risk-management tool” born from modeling and programming other strategy board games, such as backgammon, chess and Go. While in-game access to simulators is prohibited by the NFL, clients will run dozens or even hundreds of simulations to create “benchmark scenarios” and have access in the upstairs stadium booth to these printed charts as a guide.
“Ours is a forward-looking tool,” Frigo said. “While it’s based in the history of play-by-play data — and there’s a foundation of that that helped us build the model — the reality is we are assessing unique situations by simulating them going forward in a very sound way.”
Get The Latest Sports Tech News In Your Inbox!
EdjSports guided SportTechie through one such scenario: the Patriots and Titans are tied in the fourth quarter, and the Patriots are facing a fourth-and-3 at Tennessee’s 45-yard line with eight minutes remaining. While probably every NFL coach would elect to punt in this situation, the model, drawing on 100,000 simulations of the ending, favored the New England offense attempting a short pass — they do have Tom Brady, after all — over a punt by a 7.1 percent difference in win probability. Football coaches are known for their meticulous attention to detail, yet 7.1 percent is far from trivial and is a “chunky” difference, Frigo said, noting that making the wrong choice on such a decision once every week would cost the team a win (and maybe a playoff berth) in a 16-game season.
“We can help flesh out and shine a spotlight on areas that are critical junctures,” DeFeo said.
With three minutes remaining instead of eight in the hypothetical scenario, the discrepancy grows more pronounced, up to a 9.4 percent difference favoring a fourth-down pass. (Given New England’s overall talent superiority, a punt still gives the team a 60 percent chance of winning but a 69.4 percent chance for throwing a short pass.) After running a batch of simulations to account for pinning the Titans deep on a punt — or risking a touchback — the model still gave an edge to going for it, even if the punt was downed at Tennessee’s own 1-yard-line.
When the scenario is reversed and the Titans are facing that same fourth-and-3 at New England’s 45, the choice is less clear-cut, although opting to attempt either a pass or run is still sightly preferable to a punt.
“It gives context to the decision,” DeFeo said. “If we know that, in order for that to be a profitable decision that we have to pin them between the goal line and the three-yard line, anything else we’re giving away [game-winning chance]. Does that color that decision in real-time?”
While the motivation for a baseball team is more clear-cut — score as many runs as possible and allow as few — the matter isn’t always so straightforward in football, given the different point values and the decaying clock that limits opportunity for comebacks. Because the model only cares about winning or losing and not, say, saving face at a press conference or mitigating the margin of victory or defeat, its answers can be surprising.
“The model is not emotional in any way about what the final score is, what the statistics generated are — it’s only caring about the course of action the produces the highest win probability, which is where it can sometimes offer up some counterintuitive types of recommendations because it’s recognizing that certain times when you buck conventional wisdom actually produce more wins on average,” Frigo said. “It might produce losses that are losses by greater deficits, but it’s only focused on win probability.”
The Patriots, of course, are responsible for one of the most counterintuitive plays in recent history. In a Sunday night game against the Peyton Manning-led Indianapolis Colts in 2009, New England faced a fourth-and-two at its own 28-yard line with a six-point lead and 2:08 left in the game. Coach Bill Belichick elected to go for it rather than the safe option, a punt. When Brady’s pass only went for one yard, the Colts took over on down and ultimately scored the winning touchdown.
“We actually concurred with Belichick’s decision,” Frigo said of the contemporary Zeus model. “It was a pretty brave decision, and even Bill Belichick got barbecued over that one. That’s the reality of it, right? Good decision-making isn’t based on short-term results, it’s based on expected outcomes.”