The Problem With Sabermetrics

Today, Grantland posted one of the most poorly formed opinion pieces I’ve ever read from someone I normally respect. It was a column by science writer Jonah Lehrer about the dangers of sabermetrics. Lehrer, whose previous work includes a book in which he uses neuroscience to describe how humans make the decisions that they do, suggests that:

Sabermetrics comes with an important drawback. Because it translates sports into a list of statistics, the tool can also lead coaches and executives to neglect those variables that can’t be quantified. They become so obsessed with the power of base runs that they undervalue the importance of not being an asshole, or having playoff experience, or listening to the coach.

At no point in the rest of his piece does Lehrer offer any evidence that i) sabermetrics has led to coaches or executives neglecting intangibles; ii) neglecting intangibles leads to a negative performance, or iii) embracing intangibles leads to a positive performance.

The one anecdote he does use is that of the Dallas Mavericks who managed to beat what analysts would consider to be better teams in order to win the NBA Championship this season. Of course, what Lehrer fails to mention is that the Mavericks employ more advanced basketball metrics in player acquisitions, deciding matchups and making substitutions than almost any other team in the NBA.

To put it in the parlance of our times, his column is a massive fail. It takes a strawman, and condemns the poor scarecrow based on his misunderstandings.

However, something good has come out of the piece, and it can be found in the responses by some of those who embrace sabermetrics as a way of better understanding and promote it online.

From Tom Tango:

Lehrer’s objection is not with sabermetrics, even though he blames sabermetrics.  His objection is with the misuse of statistics or with the non-use of non-statistics.  That’s not the problem with sabermetrics.  That’s the problem with the user.

From Joe Pawlikowski:

Yes, we argue mostly from a statistical standpoint, and oftentimes we make no mention of intangibles or even scouting aspects of the game. That does not mean that we devalue or neglect them. Rather, we’re focusing on one aspect of the argument. No one — no one worth reading, at least — discounts the immeasurable side of the game. Since it is immeasurable, though, it is difficult to find evidence to back claims. Therefore, many statistically minded writers opt not to deal with it altogether.

From Colin Wyers:

The syllogism at work here seems to be:

  • Some people misuse numbers related to baseball.
  • Sabermetricians are people who use numbers related to baseball.
  • Therefore, sabermetricians are people who misuse numbers related to baseball.

If knowing more about baseball makes it harder for you to enjoy the game, then I’m really not seeing your case that you’re the better fan than someone like me. I know those things and I still love baseball. Love love love it. And you can have whatever opinion you want to of people like me and the work we do. But stop, please, just stop questioning whether or not we love baseball. It’s demeaning, it’s insulting, and it’s been a hoary old cliché for longer than I’ve been alive. Let it rest in peace.

From Craig Calcaterra:

If you’re going to accuse sabermetrics of leading organizations astray, shouldn’t you be obligated to cite a single example?

In summation, we get it. No one who uses reasonable analysis metrics to measure or predict performances thinks that there isn’t other stuff at play beyond the number crunching of past performances. We simply admit to knowing nothing about it, and so we focus on what we do know about, and what areas we can offer value.

Coaches and executives have access to far more information than we do, and like even Billy Beane says in Moneyball, sometimes you’re going to want a player, but he robs a bank, and then he doesn’t seem like quite the catch that he did before you found that out about him. You would be very hard pressed to find a single executive in any sport who believes that intangibles have no role in team construction or play implementation.

Having a deeper or more critical understanding of the game doesn’t ruin anything, least of all, one’s enjoyment of a sport. I love baseball. Every part of it. The unpredictability, the predictability, the short hops, the home runs. And you know what else I love. I love trying to figure out what factors were and weren’t at play in causing the short hops and the home runs. And I love that sometimes there isn’t a clear answer to the cause of certain plays.

Comments (12)

  1. Did you know that Billy Beane doesn’t even watch baseball?

  2. So what you’re saying is intangibles matter!

  3. My issue with WAR – it’s a simplistic attempt at making an argument but it isn’t showing what WAR represents.

    My issue with most defensive metrics – I wouldn’t be the first sabermetric lover to talk about how we have a long way to go with them.

    To dismiss them is ridiculous, but to cite them as the only way to measure the game would also be ridiculous. They are not definitive, they are constantly changing. Science advances, as does logic, and I’m sure in 20 years we won’t even be talking about WAR and UZR.

  4. @Kelekin: I’ve always thought WAR does a perfect job of saying what it represents.

  5. You guys often scoff at intangibles.

  6. @Travis: Not saying it does a poor job, but I’m calling it the “AVG” of sabermetrics.

    But my point is, there is rarely a primer on what WAR is, to most. Would you say two players with 3.5 WAR have given the exact same value, because of WAR?

    It’s like two players who have an .850 OPS being of equal value. But would you prefer a .300/.350/.500 hitter or a .250/.350/.500 hitter?

  7. but OPS compares two things…the ability to get on base and how many total bases the player has per at bat…and they’re not weighted evenly. WAR takes a weighted version of that along with pretty much every other thing a player can do. Is it perfect? Not at all. But it’s easily the most comprehensive single stat out there.

    And there are plenty of primers out there. Yes, it’s complicated, but that doesn’t make it a bad stat.

  8. Q: When has a GM used sabermetrics and ignored intangibles, leading to detrimental side effects for performance?


    Geez was it tough to write this article while you were cupping the balls of the statistical community? I read the cited article and it was pretty mind-numbingly stupid, but its not like Moses came down from Sinai with WAR and UZR on the stone tablets. Maybe I was just so happy it wasn’t about gold cup soccer or tennis that I want to defend it. Does Bill Simmons not know anyone who watches North American sports?

  9. That’s a silly argument, Ray.

  10. The Jays could use Milton Bradley right now. Seriously. They’re playing Juan Rivera every day.

  11. @Kelekin

    I’d take either. as my little league coach used to say “a walk is as good as a single” obviously the .250AVG hitter is getting more xbh to make up for the fact that a walk doesn’t allow for a runner currently on base to advance more than 0/1 base on the walk

  12. Sabermetric wingnuts.

Leave a Reply

Your email address will not be published. Required fields are marked *