When umpiring goes wrong in baseball, it’s hard to accept. When it goes really wrong, when power is abused, helmets thrown and expensive beer senselessly wasted, the fans demand robots. But robots will not make you feel better.

They might make the right calls. It won’t matter. Not when the right call goes against you.

Two new studies out of The Human Interaction With Nature and Technological Systems (HINTS) Lab in Seattle show that humans view robots as moral entities. The resulting paper is entitled Do People Hold a Humanoid Robot Morally Accountable for the Harm It Causes.

Yes. They do.

Now, granted, this is a humanoid robot and it’s doing purposeful wrong. But the study assesses the reaction to harm caused and umpires, by the very nature of their existence, cause harm.

As far as it being a humanoid goes, similar studies about Roombas have shown much the same. People react to machines much like they are humans.

Robot Vacuum Cleaner Personality and Behavior published in the International Journal of Social Robotics,Volume 3, notes that:

People tend to behave towards artifacts in a social way, particularly if artifacts exhibit some degree of autonomy such as robot vacuum cleaners. Aspects of anthropomorphism—the attribution of human qualities to non-humans—are reported in studies on the experience of robot vacuum cleaners.

To get away from the actual research for a moment, just think on your own experiences.

When your computer does not react as you wish, do you react logically? Or do you call it names and think about smashing it? Though the machine is rational, are you? You ever hear your dad working on a car? Did he sound calm? Or was his language making the neighbours blush?

In a high stress situation, like a Major League Baseball game, this will only get worse. Fans and players will not only treat the robot umpire as human but will assign it morality and motivation.

Just ask Curt Schilling, who in 2003 smashed a QuesTec camera installed to provide mechanical oversight to umpires. He said:

The QuesTec system in this ballpark is a joke. The umpires have admitted it. They hate it. In the last three starts I’ve made here, multiple times umpires have said to the catcher, “It’s a pitch I want to call a strike but the machine won’t let me.”

Robots might make the right calls but that won’t stop arguments and you won’t feel any better about these calls. You may have less empathy towards a robot even while you assign it morality. This translates into smashing and ballparks littered with the robot dead.

Machines are not the answer.

Baseball should have better oversight of its umpires. While they shouldn’t answer to reporters –as facing the partisan New York and Boston media will only exert more pressure on their objectivity– baseball should have a visible disciplinary and public review process. It should hold the umpires to a higher standard of behaviour.

But make no mistake, robots will answer to no one. And they won’t make you feel any better.

Comments (61)

  1. The robot would have gotten those calls right though

  2. Getting calls right isn’t about the feelings of those involved. The question at hand is: “Could robots make more accurate calls than humans?” You’ve presented a third-person anecdote from 2003 as evidence against that proposition.

    • No, it’s deeper than merely right or wrong calls. It’s the feeling that a call is right or wrong.

      • So… we shouldn’t get the calls right, because when people incorrectly disagree with a robot, all hell breaks loose?

        I think with automatons, there is a much higher expected value of accuracy than with humans. Humans are supposed to be fallible and make mistakes; computers are not. So if a human makes a costly mistake, people get upset, but can counter-point with “he’s only human.” But if a computer makes a mistake, people get furious with its fallibility, questioning why it’s in the position in the first place.

        However, the computer should be far less fallible. It’s mistakes should be infinitesimally more rare than a human’s. So the combined cost of the computer’s total mistakes must be far less than the combined cost of the human’s total mistakes.

        Much of the anger of a blown call today comes from the fact that there is computer evidence that the call was blown. I.e. the ump says it’s a strike, the replay & pitchFX say it’s a ball, people freak out. With a computer (based on pitchFX & replay )calls a strike, and a human says “I believe it was a ball,” there’s no evidence — just the human belief. All hell will not break out over that.

        Having said all that, I like the human element. There’s gotta be a happy medium.

    • That’s not the question presented. The question presented is “Will robot umpires make us feel better about the calls they make?” And I don’t know that that’s the right question to ask.

    • I’m sorry if I wasn’t clear but the Shilling story is not meant to illustrate the accuracy of the 2003 calls but the emotional reaction to them. He held a machine accountable.

      The research I’ve cited indicates that he would have done so even if the human umpire was not involved. The important part is, he felt they were strikes.

      A more recent example was during last years World Series, I recall Francona complaining from the broadcast booth that PitchFX was missing calls. He did so as a more or less neutral observer about a more or less neutral observer.

      In a situation where PitchFX or a similar system was the final call, a highly partisan and emotionally charged fanbase would quickly make the same claims, challenge the robot’s strikezone and then assign morality to the machine.

      They’d feel just as bad about the borderline calls and suspect the machine of all the same things that they suspect human umpires of.

      At least, that’s what the research indicates.

      • But is that the “problem” that replacing umpires with robots seeks to remedy?

        • Exactly.

          Just to be clear, I’m not convinced that robots could do better than humans. I simply don’t want my feelings, or the feelings of the umps, players, managers, or owners to matter when addressing the issue of fairness.

          • Just to be clear. With proper configuration a robot would have near 100% accuracy. It without question would do a better job of calling balls and strikes.

            The question is whether that would actually make fans/players happier. We have biases and we would disagree with the calls whether they were correct or not.

          • “With proper configuration…”

            It all comes back to how much one trusts MLB to implement and oversee the technology. I’m convinced that robots, correctly configured, would be far more accurate. I’m not convinced that the best system that MLB could muster would offer a significant upgrade without introducing its own negative externalities.

            Just to be clear. :)

        • In part, I believe it is.

          The inaccuracy of the strikezone calls are perceived as an injustice. (In some cases, they may well be.) But it is a perception of injustice.

          It’s that perception that robots cannot solve. Even if they are accurate, they will still be felt to be inaccurate. And people will *know* that the robot was wrong.

          • Assuming that’s true, then what’s the point of preserving the status quo? If the “perception of injustice” will remain anyway, regardless of who’s making the call, why not have the more competent actor make it?

      • “In a situation where PitchFX or a similar system was the final call, a highly partisan and emotionally charged fanbase would quickly make the same claims, challenge the robot’s strikezone and then assign morality to the machine.”

        It just seems that this is far, far less likely. The fan base now has replay & pitchFX to make the case that they should be angry. The likelihood of uproar would have to be lower with the automated system — it’d be isolated to people akin to conspiracy theorists (“they’ve fixed the machine and cameras, against my team”).

      • Also, people seem pretty content with the Hawk-Eye replay system in Tennis. I’ve never heard of fans getting in an uproar over disagreement with that.

        • Cricket has a system that is similar to tennis’ Hawk-Eye and it lays down a definitive answer on replay challenges. IPL fans seem to be more disappointed than incensed when a call doesn’t go their team’s way.

          Cricket also uses infrared cameras to tell whether a ball struck the bat, or the batter’s body (or in cases where both were struck, which order). Most of the infrastructure is already in place if MLB is willing to outsource the technology.

      • Francona thinks the system was missing calls because he is watching the catcher’s mitt as well as the path of the ball, hence the importance of framing by a catcher. Tracking the ball itself frame by frame and making ball or strike calls based on where it actually crosses the plate not where the catcher catches it is what makes pitchFX et al. a better system.

      • Francona complaining that the PitchFX data was wrong does not make the pitchFX data wrong. You need some supporting evidence to make your argument. Relying on something a talking head said during a broadcast does not really support your point.

        tldnr: anecdotes make for horrible arguments.

  3. Will Smith wouldn’t agree.

    • Isaac Asimov would shoot Will Smith dead for what he did to his story.

    • Scene: interrogation room, downtown.

      Det. Spooner: He didn’t agree with your Strike 2 call. He also didn’t agree with your Strike 3 call. And now he’s dead.

      Umpire Robot: I did not murder him.

      Det. Spooner: He flipped his helmet in disgust and it hit you. You didn’t like that.

      Umpire Robot: I did not murder him.

      Det .Spooner: And then someone threw a beer…

      Umpire Robot: I DID NOT MURDER HIM!!

  4. You mean Curt Shilling didn’t like a system that wasn’t allowing him to throw pitches out of the strike zone and have them called strikes? I AM SHOCKED.

    You might not like the calls that a robot makes but they will be *consistent*. Which is all that both hitters and fans want. It’s hard enough to be successful as a MLB hitter just trying to compete with the pitcher, never mind having to remember that Bill Miller expands the strike zone up and away to right handed batters with two strikes.

    The current environment that allows the same pitch to be called a ball for a RHB and a strike for a LHB, or having the same pitch called differently depending on the count is horseshit.

    • Exactly! The problem is consistency in umpiring not whether or not I get pissed off at my computer when it doesn’t work.

      • And inconsistency on the part of umpires is largely intentional, and thus could be controlled and mitigated witht he proper oversight.

        Umpires don’t give Halladay a big strike zone because they’re fallible, they give it to him on purpose because he’s “earned it.” We don’t need robots to solve that, we need better umpiral (huzzah! new word) oversight.

        I say, keep human umpires, but take away their freedom to do whatever the hell they want without caring about the consequences.

  5. If robots are 99.9% accurate and humans are 98%, who cares? What does human perception have anything to do with accuracy? Nothing.

    And at the least, there should be an assessment of umpire accuracy post hoc, with shitty umpires sent to the minors.

    • I’d also like to point out that the area of the strike zone is deterministic, which is not the case for Roombas, which need to adapt their behaviour based on new data and machine learning as they go. There is no pseudo-randomness in a strike call.

      Extrapolating those studies does not necessarily make sense here.

  6. You don’t need robot umpires, but you need umpires to take more responsibility in their actions. Why does know one hate Jim Joyce? Cuz he fuckin admitted he was wrong, like a man.

    Umpires get to hide behind their dickheadness, just like cops, do what they want, for whatever reason they want and have no accountability.

    The umpire yesterday should apologize for his actions, he quite clearly made a decision he was not going to allow brett lawrie to walk, he ruined the integrity of the game with a personal vendetta.

    i dont condone what lawrie did, I wish he handled it a bit better. but the fact that lawrie gets suspended, which he should, and miller gets to go to work the next day is complete bullshit

    • Umpires do get suspended and such but it is never revealed to the press. When they are taking days off sometimes it is probably legitimate vacation but other times it is because they’ve been suspended a few games for shitty calls.

  7. Robots need love too.

  8. I think the headlne is the problem. It should me something more like:
    “Why, even if they are the answer, you still aren’t gonna like it”

  9. You are objectively wrong.

  10. An actual robot standing there acting like a dickhead isn’t exactly the same as a pitchfx system that has a strikezone overlaid on video showing the ball passing through the zone or missing it.

    This is article just shows that people can be annoyed by machines programmed to annoy them by acting like assholes.

  11. I’m SO ANGRY at my computer for calculating my final average in school and telling me I failed. Stupid computer. It’s computer’s fault.

    (The strike problem is more akin to this than a Roomba’s algorithm).

  12. The question here is, how will we react when the robot calls something against us? My thought is, it’ll be the same as it is now. It’ll be better in fact, because at least we’ll know “I’m pissed but it was right call”, as opposed to “the ump has it in for us”.

    Last night, Toronto fans were pissed but Tampa fans were happy. If the calls went the other way, I doubt Tampa fans would have been pissed (because they were obviously balls). Some might have been upset, but a minority I think – therefore you can argue that in the Lawrie scenario, robot umps would have yielded a more positive result.

    • Well said!! A computer calling balls and strikes would have made more people happy.

    • Exactly. When Morrow walked 2 in a row to start the other game, I was frustrated at him for making bad pitched, not the ump for calling them balls.

      The author seems to think that people are the real robots, automatically disagreeing with anything that doesn’t go our way and automatically agreeing with calls that do. That’s not a fair assessment – people have at least some capacity for objectivity.

  13. It was said earlier but bears repeating..

    If the assertion in this article were in some way true – that people would still feel rage regardless of whether the umpire was machine or human: why is there no record of human on machine rage in the sport of Cricket – where rowdy fans riot at the drop of a hat?

    The technology already exists in two sports (tennis and cricket) that have traditionally relied on the judgement of humans for longer than baseball.

  14. A home-plate umpire would still be there to make other calls … so would (likely) be the one to simply voice what the electronic eye would record and indicate. It could be done, easily. And will be, one of these centuries.

    • Or … they could keep it as it is and just use what Ernie (above) is talking about and give each team a certain number of ‘challenges‘ the way they do it in tennis. That could work.

      • How does that apply for balls and strikes though? How would you challenge a decision that stems from human element to begin with?

        the challenges would be more about safe/outs calls

        • I don’t know … of the top of my head, maybe a strike 3 challenge rule could be instituted … ? I’d have to think about it. But I’m sure some sort of combination of Umpire/Electronic Eye could be hammered out and made use of, effectively.

          • but the problem with that is consistency;..

            if an umpire has been calling outside a strike all game and then the computer, after a challenge, calls it a ball, thats unfair.

            Umpires are allowed to be imperfect. its expected actually, one thing that isn’t tolerated is inconsistency. The 3-1 pitch to lawrie was a ball all game, and then became a strike. thats the issue. if he had been calling it a strike all game, perhaps lawrie swings at it, or atleast is more prepared for it

          • Well, I suppose there’s that. So for it to work, maybe then, the ‘eye’ would tell the ump if it’s a ball or strike every time (and the ump would signal the call). With the ump still having to be there anyway, the ‘appearance’ of the ump making the call – for the esthetics of it more than anything – there might be less resistance to the major change in the game that it would unquestionably be.

          • when i look at the pitchfx, it appears to me that miller called one pitch by someone from toronto a strike that was FAR more outside than strike two to lawrie. he also called 3 TB pitches strikes in the same area so close together i can’t guess which was the one in question.


  15. This is beyond silly. Two studies, one with a sample size of 40 and another with a sample size of 6, with research questions that aren’t even close to generalizable to the situation you are trying to generalize to. This is pretty much a textbook example of how not to responsibly use peer-reviewed literature.

    I don’t know if getting rid of the human element is the right way to go or not, but two things. One, there needn’t be a physical symbol of an electronic umping system (like a Roomba or a humanoid robot behind the plate). I find it somewhat improbable that a player would seek out one specific camera to smash during play, but I suppose it’s not outside the realm. Two, regardless of human irrationality, human umpires can exacerbate situations by being vindictive, unprofessional, and adhering to some bullshit unwritten code of ideal masculinity. If Lawrie had swung and missed at strike two last night, he may have argued the 3rd strike call and maybe gotten tossed, but it certainly wouldn’t have escalated to the point that it did. It was the probably valid perception that he was being deliberately screwed that made him lose it and got the crowd riled up. Without the “human element”, there would very rarely be a valid reason for someone to feel that way. An automated system might get calls wrong, but it would be totally irrational for someone to feel they were being deliberately screwed.

    • “This is pretty much a textbook example of how not to responsibly use peer-reviewed literature.”

      Yep, and also: burn.

    • it’s also totally irrational for lawrie to behave as he did. he knows the rles, and he knows that the ump would not reverse the call, so his reaction could do nothing other than serve as a(n identity) performance.

      • I agree with you, but my point is that it was the perception that the ump was being purposefully vindictive (I certainly think he was) that made the whole thing ugly. Fans probably aren’t chanting “bullshit” and tossing beers onto the field if they don’t believe there was some malice aforethought on the part of the umpire. Obviously there are exceptional situations where this might happen (Jim Joyce’s missed call in Galarraga’s perfect game comes to mind), though.

        Also, those “identity performances” would probably be a lot less sympathetic from a fan perspective if there were an accurate system with a defined error rate instead of a more fallible human. This doesn’t account for the folks who ignore Occam’s Razor in favor of conspiracy theories though, and they’ll never go away.

    • Exactly. It may be true that robot umpires are a bad idea, but this piece is exactly an example of how not to apply science literature. Once image recognition has taken place, there is no articial intelligence anymore in this problem. It’s totally deterministic. As I said above, one should be no more angry at this classication than the calculation of an average, which is similarly completely deterministic. Of course, humans will get upset at machine learning algorithms that don’t learn as well as humans do.

      If one wanted to show the author’s hypothesis, what should be done instead is a comparison of different types of algorithms vs the feelings of humans to the result. Nobody gets upset at a computer for its result of a numerical problem that’s deterministic. They may however get upset at an algorithm that needs to adapt its own algorithm on the fly based on new information that it learns (machine learning). So, this piece says nothing.

      Taking a result for one algorithm and saying it applies to every algorithm is beyond ridiculous.

  16. We need to move past The Human Element excuse. It’s dated and really just an easy cop out for those who are against using the technologies available to officiate more accurately. Just ask football(soccer) fans the world over. There’s so much invested by people iin sports today. From the owner’s and cities that invest millions into their teams for stadiums and players, to the players who spend most of their lives just trying to get to the level of performance required, to the fans who make it all possible in the first place by pouring their hard earned money and emotions into the sports they love. Take away the human element and you take away a lot of the anger and frustration that comes because of the differences we perceive of the same incident.(When a ref calls a hook in hockey that leaves you scratching your head thinking “What’s he watching?”) Professional sports should be using every possible available technology to make sure the games are officiated as accurately and consistently as possible.

    In regards to the Lawrie incident my biggest beef is with Umpires calling strikes on blatant balls because the batter starts to walk to first before the Ump calls the pitch and feels the batter is showing them up! Pretty fucking childish but they been doing it for as long as I’ve watched baseball.

    • re: Lawrie leaving the box before the call.. This is also difficult because a lot of UMPs won’t call anything on that play. The only call strikes – which means often times a player is “showing up” and ump, when in actuality they simply think they’ve earned ball 4.

  17. Any sane and rational sports fan that has ever favored the expansion of replay in baseball cannot possibly deride the advancement of technology to most accurately assess the on-field action. Yes, I get mad at my computer, call it names, suppress my urge to grow it on the ground. And a moment later – as a relatively sane and level-headed adult – I acknowledge my silly reaction and dismiss it as a poor reaction, and get on with it.

    As a mathie, I cannot argue with fact. I can, however, judge a person paid to be of the top of his profession to work to he best of his abilities and call a ball a ball, and refuse to change the call based on the batter’s assumption. These “old tyme unwritten rules” have gone the way of drinking and driving after games, segregation, and spitballs.

  18. So, you’re saying people have emotional reactions to robots and robots make better calls and people have reactions to human beings and human being make mistakes. So, don’t use robots because both they still create emotional reactions. How about a human with robot capabilities?

    I’ll re-post my thoughts from DJF:
    BlueWingedTeal says: 05.16.12 @ 2:28 PM EDT
    re: Robot Umps

    I don’t want to see the ump behind the plate disappear, but, that doesn’t stop me from thinking that I can call a better game from my bloody laptop. We ARE in the 21 millionth or whatever century.

    Can’t we give the umps a little helping hand? I think some brilliant engineer at RIM or Apple or Samsung (or HTC or Microsoft, etc.) should be able to design a wireless device that gives the ump a vibrating indicator if the pitch is in the zone. I’m sure they could work out a system like this that would give the ump the final call (and all the power he craves) but give him an indication that nobody else sees. This would just be an extra tool the ump can use to determine the location of the ball besides his (old-fashioned) eye-balls (and in this case, his emotional reaction to Lawrie trotting off to first before the call was made).

    Lawrie’s reaction was embarassing, but, so was the umps call. I really believe that both calls were deliberately provocative. Because Lawrie is a young, hot-tempered rising star, because he’s on a team with Bautista who never let’s a bad call go without chirping the ump, because he started his walk before the call, Miller made the call against him. If it wasn’t for those three factors, Lawrie would have gotten his base with no controversy.

    I say, give the umps a tool to help remove this emotional aspect out of these potentially crucial calls.

    Every night, they can review their calls that went against the sensor. If an ump is out of sync more than the rest, MLB officials can sit them down and give them a talking to.

    I was thinking about the micro sensor idea too

  19. Ryan, nice article. I think it would be interesting to conduct a poll of all umpires, including those toiling in the minors, to see how many of them think that they are “part of the game.” I’m of the opinion that in certain situations umpires, whether they they acknowledge it or not, initiate hostility with the players for one reason or another. Of course I could be wrong, however, shouldn’t we evaluate the emotional reactions of umpires themselves and how they impact the game? This would not only encompass verbal communication but body language as well.

    Another thought: while I can accept human umpires, given that the technology exists to show the exact accuarcy of an umpire’s calls (balls and strikes/ basepath activity/etc.) sholdn’t this be implemented as part of the performance reviews umpires are subjected to? Again, it could just be me, but it seems that turnover among the ranks of umpires is extremely limited. Perhaps an accuracy threshold could be instituted in order to ensure that only the most accurate umpires are allowed in the MLB.

  20. This article presents the most ludicrous argument I could possibly imagine in defense of not moving towards a foolproof system of calling strikes accurately. Simple counterpoint, exactly how has this “robots will be seen as moral entities” phenomenon exhibited itself in tennis since it instituted the electronic eye? Answer, it hasn’t. It has provided a clearcut basis for what was once a highly contentious issue. Anyone else remember John McEnroe?

    • Because no one ever argues tennis calls anymore.

      • Doesn’t this contradict your thoughts on expanded replay?

      • You can’t argue with a robot. And they can challenge the call of an umpire. In these cases the replay is immediately shown to everyone as seen by the electronic eye, and the issue is settled quickly and cleanly. And there are still no “moral” judgements being thrown about. Again, it was an interesting concept for an article perhaps, but at the end of the day to argue it as a valid point is just too ridiculous to be believed.

  21. This article to makes one good point but misses the next logical one:

    It’s true that fans will dislike any call that goes against their team, generally speaking. Although I would say those who really know the game will be upset, but when they see a replay showing that it was the right call to make will quickly move on.

    But if people are going to be upset at who/whatever is making the calls, isn’t the duty of the MLB to do everything they can to get the right call as much as possible? Is it not, or rather should it not be, their duty regardless of what some portion of fans want for their team?

  22. I really don’t understand why the emotional gratification of fans, players, or umpires is relevant to whether or not improved call accuracy is worth pursuing. People may well still “feel” the call was wrong, but if it’s not, why does that matter?

  23. machines ARE the answer in baseball. Computers measure in on or off 0 or 1. There are no borderline strikes with them. And they make no errors and there will be no need to arguing the call. It’s working in tennis very well and it is only a matter of time balls and strikes will be called correctly by computers. And it is long overdue.

Leave a Reply

Your email address will not be published. Required fields are marked *