:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/61147073/usa_today_10637260.0.jpg)
“Do you dare?”
That’s the question that Al Michaels asked his broadcast partner, Cris Collinsworth, as the clocked ticked down from 5:51 to 5:50 remaining in the 4th quarter of Super Bowl LII. The Eagles were trailing 33-32, facing a 4th and 1 at their own 45-yard line.
Collinsworth responded with no hesitation, “you have to”.
Eagles coach Doug Pederson agreed. He left his offense on the field and called for a mesh concept with his tight ends crossing underneath, allowing Nick Foles to find an open Zach Ertz for the first down.
Seven plays later Foles found Ertz again, this time in the endzone for what would turn out to be the game-winning score. Confetti falls, parades are planned, Pederson becomes a a folk-hero in Philadelphia and author.
That decision wasn’t an outlier though, it was a microcosm of what Pederson had been doing all season. During the 2017 season, Pederson “went for it” on 4th down 26 times per ESPN data, second most in the NFL behind Mike McCarthy and the Packers’ 28 attempts. When you take in to account the fact that the Eagles spent the least time trailing — just 12:15 per game according to Football Outsiders — that amount of attempts stands out even further. A big chunk of all 4th down conversion attempts occur late in games when a team is playing from behind in desperation mode.
Desperation wasn’t the driving factor behind Pederson’s 4th down calls though, it was data. Pederson and the Eagles organization have been working with EdjSports — a data intelligence firm based out of Louisville, Kentucky — for the last couple seasons. Edj’s services include providing data analysis that helps coaches determine what the “right” call is for a given game situation based on a Monte Carlo simulation model that they have developed.
EdjSports co-founder and Chief Information Officer, Frank Frigo, says that their model suggests that “the average team in the NFL gives away something in the vicinity of two-thirds to three-quarters of a game [per season] based solely on sub-optimal fourth down decision making”. That may not sound like a lot, but when you consider the lengths to which NFL teams will go to gain even the most ridiculously small edge — hiding injury information, spying on opponent’s practices, and even slightly deflating footballs (allegedly) — gaining nearly a win per season simply by making better 4th down decisions seems like it should be a no-brainer.
But it’s not a no-brainer for most of the NFL right now as teams continue to play 4th down far too conservatively. A completely rational coach would go for it on 4th and 1 nearly every time when the ball is between the 40’s, and yet teams still choose to punt in this situation about 60% of the time. The Eagles faced 4th and 1 between the 40’s 10 times in 2017 and went for it 7 times, converting on 6 of those 7 attempts.
On the 4th and 1 play from the Super Bowl that we looked at above, the Eagles had two options: go for it and try to extend the drive, but risk giving the ball back to the Patriots on the edge of field goal range with less than six minutes remaining or punt and try to pin the Patriots deep and rely on their defense to get a stop. Frigo says that EdjSports’ model would have recommended going for it even if you could guarantee that the Eagles punt would have gone out of bounds at the 1-yard line — the best possible result of punting. Obviously, the probability of the punt actually being downed on the 1 is extremely low, but that goes to show what a landslide that decision was according to their analysis.
Not only does being aggressive on 4th and short situations tend to be rewarding to teams as a stand-alone decision, but it can help a team’s broader offensive strategy too. Frigo added that, “If you know 4th down is a viable option if you come up short on 3rd, that really makes you a bit less predictable on 3rd down and opens up some playbook options. I think there is a lot of residual value there as well.” How many times do you see defenses set up on 3rd down with their linebackers and defensive backs sitting on routes right at the sticks? Traditional coaching tells receivers to make sure you run your route to the first down marker on 3rd downs, but it’s infinitely easier for defenders to do their job when they know where their man is going to end up. It’s a big part of the reason that completion percentage on 3rd downs is always lower than 1st and 2nd down completion percentage. However, if you’re open to using 4th down, you can be more comfortable throwing short of the sticks since 4th and 1 or 4th and 2 could be a positive outcome rather than the end of the drive. That forces defenses to defend more space on the field which is a primary goal of any good offense. When you factor in the residual value gained from being less predictable on 3rd downs with optimal 4th down and kicking game decision-making, EdjSports firmly believes that an average NFL team would improve their record by 1.5 games over the course of a season. That is a huge amount of value in a 16 game season.
One thing that sets Edj’s model apart from other analytical tools is the fact that they look at all decisions from the viewpoint of how much it helps or hurts the team’s chance to win the game. They call it GWC — or Game Winning Chance — and it’s the basis off which their recommendations are made rather than an expected points model. The basic example that Frigo gave was a 4th and goal call at the 1 yard line with very little time remaining and your team down by 2. Edj’s GWC would tell you to kick the field goal and take the lead, while an expected points model would tell you to go for it because the expected points for attempting to score a touchdown is higher than the expected points from a field goal attempt. Obviously, coaches don’t really need a Monte Carlo simulation to figure that one out, but this becomes important when looking at more complex decisions. By using GWC instead of expected points, Edj aligns the goal of their model with the goals of the team, which is simply to win the game.
The NFL does not allow teams to run actual simulations during the game, so Edj helps it’s teams prepare “game books” for each week which includes tabbed and organized “go or no-go” sheets for 4th down calls for various downs, distances, field positions, and clock situations. Those calls may vary slightly from week to week depending on the opponent. An internally developed rating system allows them to model the optimum decisions for that specific opponent, not just an average NFL team. After all, 4th down attempts against the Vikings stout defense are a very different proposition than a 4th down against the Colts.
In addition to what they call “critical decisions” — 4th downs, kickoffs, and PATs — Edj also is able to offer strategy decisions. Does a particular corner struggle against a certain route? Is there a play call that the opponent’s defensive scheme gives up big yardage to? New developments are allowing their model to reveal further insights in to roster building as well. They can take an individual player at a position and replace him with a better or worse player and see how that impacts a GWC against an average NFL or even specific teams. Want to know what kind of roster would be most effective against AFC South opponents? Edj could tell you. What positions make the largest difference when it comes to win-loss records at the end of the season? They can help with that as well.
Most NFL teams use analytics in some form or fashion, but some are far more advanced than others. Frigo believes many only pay lip service to their use of analytics, noting that “...lots of teams have analytics staffs and say they do analytics, but there is a difference between having an awareness of the tools that are out there and actually implementing those ideas and having the courage to do that on the field.”
For NFL teams to really take full advantage of the value that an analytics firm like EdjSports can provide, it takes buy in from all levels of the organization. Fans are quick to judge coaches based on the results of their decisions in critical situations regardless of whether or not the process to arrive at the decision was sound. Frigo and Edj take a different view, saying that “good decisions are independent of short term results”. However, a coach needs to feel confident that their owner and GM aren’t going to hang them out to dry if a good 4th down call produces a bad result. Coaches generally don’t have to stand at the podium and answer questions about their decision to punt on 4th down after a game — though they should — but if a 4th and 1 attempt gets stopped short they’ll be second guessed and criticized by the media all week. Frigo calls the Eagles “a special organization” because of their willingness to embrace analytics at all levels from Doug Pederson and the coaching staff to GM Howie Roseman and owner Jeffrey Lurie.
The Eagles have been investing heavily in analytics since the 90s and have one of the larger dedicated analytics staffs in the NFL. That’s something that Frigo says actually makes their pitch easier because the Eagles staff can really appreciate how robust the tool that Edj provides really is. Philadelphia has been on the cutting edge of analytics for some time, but there seem to be more NFL teams beginning to head in that direction.
Since Jon Robinson arrived in Tennessee, he has pushed the Titans towards analytics. He hired two scouting assistants with analytics backgrounds in 2017 and the team is now using some sort of player tracking data from practices as evidenced by Mike Vrabel’s comments about Corey Davis’ high work rate in practice. That kind of data can be highly useful. We know the team is using it from an injury prevention standpoint — though the current laundry list of nicked up players isn’t exactly a ringing endorsement of its effectiveness right now — and it likely is also monitoring speed, acceleration, and other physical attributes using the same data. Being able to know actual playing speed is valuable for many applications, including comparing players battling for roster spots, knowing what kind of condition a player is in, and knowing when a player recovering from injury is back at full strength.
In 2017, the Titans weren’t terrible at in-game critical decision making, but they still left 0.6 wins on the table according to Edj’s model. Considering Tennessee missed winning the division by just 1 game, it’s fair to say that being more aggressive on 4th downs could have made the difference in the AFC South race and brought a home playoff game to Nissan Stadium. We have no idea what Vrabel will be like when it comes to these decisions, but the hope for Titans fans should be that he’s willing to listen to the data and bold enough to follow it.
The analytics movement in the NFL is gaining steam and the teams that figure out how to harness data and apply it first will continue to gain advantages like the Eagles did last season. The average NFL team is leaving 1.5 wins on the table per year simply because they are irrationally conservative with in-game decision-making. Obviously, if every team made optimal decisions this advantage would be washed out, but right now there is still opportunity for smart teams to differentiate themselves and EdjSports is here to help them do just that.