If you happened to catch our fearless leader Dustin Parkes on theScore’s Stat Attack baseball season preview, you know there isn’t a lot of love lost over saves here at Getting Blanked. The only thing worse than rewarding the wrong contributions is basing an entire managerial strategy around it.

Luckily, a cadre of smart people devised a new way to evaluate and value the contributions of relievers. Better yet, the good people at Fangraphs now track them for all of us to enjoy. Shutdowns and meltdowns are here to save us from the tyranny of the save stat.

These two numbers are based on win probability added – any time a reliever increases (or decreases) his team’s WPA by .06 or more. Put another way, it measures relievers who increase or decrease his team’s probability of winning by more than 6%.

Take yesterday’s Yankees – Tigers game, for example. Both Joba Chamberlain and Rafael Soriano picked up shutdowns for pitching effectively when the game was still very much in doubt. Tigers relievers Phil Coke gets a big ole meltdown as the leadoff home run he surrendered to Curtis Granderson took the Tigers probability of winning from 41.5% to a mere 22.3%.

This stat (should) appeal to all types of fans as it only cares about getting outs. It doesn’t care if you used strikeouts, groundballs, pickoffs, or whatever. Get the outs at the right time – you get the credit. Another appealing aspect of shutdowns/meltdowns is more than one is up for grabs per game. The game isn’t always won or lost in the 9th inning, why only reward the guy that pitches at that time?

The list of shutdown leaders looks pretty much as you would expect. Carlos Marmol leads the way with Brian Wilson, Mariano Rivera, Joakim Soria, and Matt Thornton among the top compliers. The list of most blow-upable relievers is a rogue’s gallery of crappy pitchers with Aaron Heilman, Rafael Perez, Matt Guerrier, Jon Rauch, and Kevin Gregg in the top 5.

Like any counting stat, it is important to try and add some context. I calculated the leaders in “batters faced per shutdown” in an attempt to determine who successfully converts shutdowns with the most regularity. A lower number is better, in this instance, as it indicates more usage in higher leveraged situations.

I often find differentials can be more telling that straight-up rates. As such, I created the differential shutdown rate: subtracting total meltdowns from total shutdowns then divided by batters faced (expressed as a percentage for, let’s say, ease of use.) The results are dumped in chunks below, with the occasional comment.

Name Batter Faced/Shutdown Shutdown differential rate
Joe Nathan 7.29 11.65%
Joakim Soria 7.60 11.30%
Mariano Rivera 7.10 11.26%
Brian Fuentes 7.31 10.66%
Phil Hughes 7.96 10.55%
Rafael Soriano 7.81 10.32%
Neftali Feliz 7.88 9.59%
Francisco Rodriguez 8.19 9.28%
Brian Wilson 8.30 8.90%
Arthur Rhodes 8.14 8.65%
Jonathan Papelbon 8.99 8.52%

The top ten is all closers except for….Arthur Rhodes? Arthur Rhodes! Never underestimate the value of a good LOOGY. Dusty Baker might not know when to make the call to the bullpen but when he does, he calls on the best guy.

Name Batter Faced/Shutdown Shutdown differential rate
Hong-Chih Kuo 9.36 8.45%
Heath Bell 9.07 8.44%
Carlos Marmol 9.06 8.37%
Francisco Cordero 8.81 8.34%
John Axford 9.38 8.09%
Huston Street 8.81 7.98%
David Aardsma 9.43 7.85%
Brad Lidge 9.14 7.81%
Mike Adams 10.05 7.69%
Ryan Franklin 9.89 7.67%
Jose Valverde 10.28 7.55%
Troy Percival 9.11 7.32%
B.J. Ryan 9.30 7.27%
Scott Eyre 10.64 7.26%
Billy Wagner 9.36 7.18%
Matt Thornton 9.28 7.14%
Bobby Jenks 10.17 6.84%
Jonathan Broxton 9.51 6.78%

Justify my love Matt Thornton! Wait, does that say B.J. Ryan? There is something to be said for getting by on spit and will, isn’t there? Scott Eyre, Mike Adams, and Carlos Marmol are all guys who spent more time as setup men than closers. Hong-Chi Kuo ranks ahead of the man who follows him, Jonathon Broxton.

Let me know what you think of these numbers – do they have value? Would you use them in a fantasy league, if available? I plan to do more with shutdowns and meltdowns as the season progresses, hopefully using more leverage data to measure which pitchers deployed most effectively.

Hat tip to Fangraphs for the meltdown/shutdown data.

Comments (12)

  1. Interesting idea, I like I a lot. Any idea how/why .06 WPA was chosen as the limit?

  2. ‘Cause Tango says it’s so.

  3. Question about this: Would relievers on really good or really bad offensive teams potentially be penalized? If the Yankees consistently score a gazillion runs per game, it would be tough for a one-inning reliever to boost the WPA by 6%. Conversely, if the Pirates allow a gazillion runs (while scoring none!), it would be tough to boost the WPA by 6% (or potentially, tough to make things any worse).

    I like the stat, way better than saves, I’m just curious about its limits.

    • WPA is free of context in that it doesn’t estimate the relative quality of the two teams. I assumes all teams are equal and each batter is just as likely to help or hurt his team as any other. It looks exclusively at the state of the game (outs, baserunners, inning.) It’s egalitarian like that.

  4. I think I may have poorly phrased my example. I’m more thinking of pitchers on teams who enter games when the game has already been decided (though the more I think about it, the more I realize that these types of situations won’t happen enough for it to really matter).

    Let’s say that a pitcher only gets brought into games when things are already out of hand and the opposing team is winning by 7 runs or whatever. Even if he allows another run, he likely won’t get credit for a meltdown. Or if a reliever gets brought into games and throws a 1-2-3 inning but his team was already up by 8 (or whatever), he likely won’t get credit for a shutdown (as the game is pretty much already decided). Does that make sense, or have I somehow totally managed to misunderstand WPA?

    • That’s the whole point of WPA and this stat – you can’t add (or subtract) too much from your team’s probability of winning when it is already close to 100 (or zero.) WPA is determined by the leverage of the situation.

  5. Or maybe .06 was picked as a threshold as it will count just about any meaningful/useful outing?

  6. Wow…epic meltdown by Brandon Lyon. Why the hell did they leave him in there?

  7. How I’m only coming across this now, I don’t know. But I like it and I’m excited to see what else you can do with it. It reminds me a bit of what I was doing last year when the bid debate was “should Kevin Gregg be the Jays’ closer?” I was digging into WPA and leverage (I went with gmLI) and all that shit.

    The problem is that I didn’t really have a clue what I was doing.

    Have at it!

  8. Daniel Bard discussed meltdowns and shutdowns with the Boston Globe last week: http://hardballtalk.nbcsports.com/2011/04/21/stat-head-setup-man-daniel-bard-reads-fan-graphs/

Leave a Reply

Your email address will not be published. Required fields are marked *