The Problem With Sabermetrics

Today, Grantland posted one of the most poorly formed opinion pieces I’ve ever read from someone I normally respect. It was a column by science writer Jonah Lehrer about the dangers of sabermetrics. Lehrer, whose previous work includes a book in which he uses neuroscience to describe how humans make the decisions that they do, suggests that:

Sabermetrics comes with an important drawback. Because it translates sports into a list of statistics, the tool can also lead coaches and executives to neglect those variables that can’t be quantified. They become so obsessed with the power of base runs that they undervalue the importance of not being an asshole, or having playoff experience, or listening to the coach.

At no point in the rest of his piece does Lehrer offer any evidence that i) sabermetrics has led to coaches or executives neglecting intangibles; ii) neglecting intangibles leads to a negative performance, or iii) embracing intangibles leads to a positive performance.

The one anecdote he does use is that of the Dallas Mavericks who managed to beat what analysts would consider to be better teams in order to win the NBA Championship this season. Of course, what Lehrer fails to mention is that the Mavericks employ more advanced basketball metrics in player acquisitions, deciding matchups and making substitutions than almost any other team in the NBA.

To put it in the parlance of our times, his column is a massive fail. It takes a strawman, and condemns the poor scarecrow based on his misunderstandings.

However, something good has come out of the piece, and it can be found in the responses by some of those who embrace sabermetrics as a way of better understanding and promote it online.

From Tom Tango:

Lehrer’s objection is not with sabermetrics, even though he blames sabermetrics.  His objection is with the misuse of statistics or with the non-use of non-statistics.  That’s not the problem with sabermetrics.  That’s the problem with the user.

From Joe Pawlikowski:

Yes, we argue mostly from a statistical standpoint, and oftentimes we make no mention of intangibles or even scouting aspects of the game. That does not mean that we devalue or neglect them. Rather, we’re focusing on one aspect of the argument. No one — no one worth reading, at least — discounts the immeasurable side of the game. Since it is immeasurable, though, it is difficult to find evidence to back claims. Therefore, many statistically minded writers opt not to deal with it altogether.

From Colin Wyers:

The syllogism at work here seems to be:

  • Some people misuse numbers related to baseball.
  • Sabermetricians are people who use numbers related to baseball.
  • Therefore, sabermetricians are people who misuse numbers related to baseball.

If knowing more about baseball makes it harder for you to enjoy the game, then I’m really not seeing your case that you’re the better fan than someone like me. I know those things and I still love baseball. Love love love it. And you can have whatever opinion you want to of people like me and the work we do. But stop, please, just stop questioning whether or not we love baseball. It’s demeaning, it’s insulting, and it’s been a hoary old cliché for longer than I’ve been alive. Let it rest in peace.

From Craig Calcaterra:

If you’re going to accuse sabermetrics of leading organizations astray, shouldn’t you be obligated to cite a single example?

In summation, we get it. No one who uses reasonable analysis metrics to measure or predict performances thinks that there isn’t other stuff at play beyond the number crunching of past performances. We simply admit to knowing nothing about it, and so we focus on what we do know about, and what areas we can offer value.

Coaches and executives have access to far more information than we do, and like even Billy Beane says in Moneyball, sometimes you’re going to want a player, but he robs a bank, and then he doesn’t seem like quite the catch that he did before you found that out about him. You would be very hard pressed to find a single executive in any sport who believes that intangibles have no role in team construction or play implementation.

Having a deeper or more critical understanding of the game doesn’t ruin anything, least of all, one’s enjoyment of a sport. I love baseball. Every part of it. The unpredictability, the predictability, the short hops, the home runs. And you know what else I love. I love trying to figure out what factors were and weren’t at play in causing the short hops and the home runs. And I love that sometimes there isn’t a clear answer to the cause of certain plays.