We Are the Superpens

Always a good plan, no matter the score.
In this, the age of Baseball Enlightenment, few things are as divisive as the role of relievers. Many believe relief pitching is nearly fungible, that just about any big league quality pitcher can step in and be an effective reliever.

Others, usually with stinging losses still at the front of their minds, firmly believe the difference between winning and losing lands squarely on the shoulders of relief pitchers with enough regularity that the job is best left to proven professionals. If any Joe could step in and be a good reliever, any old Joe would do just that.

The obvious truth is a good bullpen is integral to being a good team. Many bad teams have great bullpens, but few good teams have bad bullpens. Does building a super bullpen full of high octane arms give teams a legitimate leg up?

Just as it is obvious that a good bullpen is key to a good team, the makeup of a quality bullpen seems obvious, too. A bunch of guys who don’t give up runs sounds good enough to me. When looking back and trying to “evaluate” such claims, it gets dicey. Do we use the standard go-to stat Wins Above Replacement? It certainly helps lay the “how much does a good pen add?” question bare. But WAR and bullpens don’t often mesh, not as well as we’d like.

Win Probability Added is another strong measure of a collective bullpen’s worth. WPA is good because it asks not “how?” but “how many?” WPA cuts all the mumbo-jumbo out of the equation and just focuses on the game situation. Did you get the outs? Cool. That’s it.

Via Fangraphs, here are the top 20 bullpens (by WPA) since the strike with the WAR and team record & outcome for fun.

Season Team WPA WAR Team Wins
2002 Braves 11.22 6.6 101 (Lost, NLDS)
1995 Indians 10.73 6.7 100* (Lost, WS)
2003 Dodgers 10.59 8.8 85
2001 Yankees 10.47 8 95 (Lost, WS)
2006 Twins 10.36 9.7 96 (Lost, ALDS)
2001 Mariners 10.23 5.9 116 (Lost, ALCS)
1997 Orioles 10.22 5 98 (Lost, ALCS)
2002 Twins 10.1 6.8 94 (Lost, ALCS)
2004 Rangers 10.04 8.8 89
2003 Astros 9.9 5.2 87
2009 Yankees 9.6 5.2 103 (Won WS)
2008 Rays 9.3 3.2 97 (Lost, WS)
2006 Mets 9.26 6.7 97 (Lost, NLCS)
2011 Yankees 9.11 7 97 (Lost, ALDS)
2001 Astros 9.02 5.2 93 (Lost, NLDS)
2007 Red Sox 8.73 5.9 96 (Won WS)
2011 Red Sox 8.59 7.7 90 (LOL)
1999 Braves 8.58 5.3 103 (Lost, WS)
1999 Rangers 8.58 7.7 95 (Lost, ALDS)
2003 Diamondbacks 8.53 5.2 84

Again, this is helpful but not but particularly instructive for many reasons. Number one being the fluid nature of bullpens. Take the 2011 St. Louis Cardinals, for example. Like many teams before them, the defending champs bolstered their ‘pen at the trade deadline. They also received a solid boost from injured hurlers returning to health late in the season.

WAR suggests a good pen can be as impactful as a superstar player. Only three bullpens (of the more than 500 team bullpen seasons since 1995) rated above 9 fWAR, or the value of Matt Kemp/Jacoby Ellsbury in 2011. The average relief corps posts a shade under 3 Wins Above Replacement.

Which results in more regular season wins? For fun, I ran a regression (I shoved some numbers into Excel which, as Ben Lindbergh notes, doesn’t sound as smarty pants as running a regression) and here are the results. Which collective number correlates strongest with team wins?

Win Probability Added comes out with the strongest correlation to team wins, nearly doubling Wins Above Replacement’s R-squared (.41 to versus .22, if that means anything to you.) For argument’s sake, both ERA and FIP relate negatively with total team wins. WPA is, of course, not without its shortcomings. As with any assessment of pitching we must account for defensive input and considerable contributions of a capable starting staff.

Using my considerable excel skills, I know now 34 teams won more games than they lost after their bullpen posted a collective negative WPA. The 1999 Diamondbacks won 100 games with a negative WPA from their ‘pen. The 2000 Snakes won 85. The 2001 team won the World Series. How did they manage that? I’m not 100% sure but I think the 40 fWAR over those three seasons from Randy Johnson and Curt Schilling helped.

Two playoff teams from 2011 posted negative WPA from their bullpens: for two very different reasons. The Cardinals posted -2.09 (and just 1.2 WAR) but, as already mentioned, what that bullpen was at the end of the season differed wildly from the group that posted brutal numbers for much of the year.

The 2011 Tampa Bay Rays similarly snuck into the playoffs with a below-average pen. The Rays not only notched a -0.32 WPA but a meagre 0.7 WAR. One good reason for that low number? They pitched an incredibly small number of innings. Less than 400 in total, the fewest innings pitched by a club since 2002 and the 11th lowest number in this entire sample. Good starting pitching is clearly your best reliever.

Having a great bullpen is a nice luxury for teams. It isn’t exactly the first place you start building if winning titles is your endgame. Like the Cardinals in 2011, you can always add effective pieces as the season wears on to fit the particular needs of your club. Or, like the Blue Jays, you load your bullpen with high quality pieces with another endgame in mind: eagerly extracting value from those glory-hunting clubs with championship aspirations.

Like many teams successfully demonstrate on a yearly basis, if you excel in one area of the game you are free to “borrow” from other, weaker areas of your team. Please to enjoy this Google Doc of all World Series winners since the strike on one sheet and the full dataset on another if you’re interested. You can’t win a title without a bullpen but you can’t ride one exclusively to glory.

Comments (22)

  1. I would say that it’s true that all Pen players are failed Rotation guys but not all former Rotation players can work in the pen.

  2. More proof(?) that the Cardinals had no business winning last years World Series.

    Still sounds better than blowhard Michael Kay reciting “defending World Series Champions New York Yankees” incessantly, though.

  3. WordPress is a smarmy mistress sometimes…good post though, I’ve always kind of wondered about this.

  4. What’s the estimated WPA for the Jays’ new pen?

  5. Very good analysis Drew. I think the Jays need a good bullpen to help out the young starters.

    Also, I hope Farrell gets better at managing the bullpen.

    He didn’t use Dotel properly.

  6. so if the jays pen can win the 2012 WPA award (brought to you by carl’s jr.) and manage not to defy the last 20 years of baseball history we can reasonably expect anywhere from 84 to 116 wins. in addition to that there will be a 75% chance of a post season birth.

    not bad.

    • due to the frigid temperatures, canadians are more likely to spend jan-feb indoors trying to keep warm. with that in mind, i think the odds of a post season birth are much higher than 75%.

      • With Texas and LA battling it out for 1st and a WC berth, and the Yanks, Soxs, AND, Rays doing the same, I wouldn’t realistically put our chances at a post season birth this year any higher than 20%. And I feel that is generous. Only 5 teams will make the playoffs in the AL…I can almost guarentee it will be a mix of Texas, Anaheim, NY, Boston, Tampa, and Detroit. I just don’t see the Jays finishing higher than any of those teams.

  7. Nice article.

    2011 WPA 2010 FIP 2011 FIP
    Santos 0.19 3.10 2.87
    Oliver -0.24 2.64 2.77
    Frasor 0.85 3.31 4.09
    Janssen 2.05 3.85 2.45
    Cordero 2.05 3.92 4.02
    Perez -0.39 DNP 4.64
    Villanueva 0.23 3.74 4.10

    Total for 2011 4.74

    Should be noted that’s just for 7 guys. I am not sure if the totals from the list above include more than 7 guys or not. I also could not separate Perez or Villanueva’s starts from their WPA. Both had some pretty brutal numbers as starters. Villanueva had 1.60 ERA as a reliever and Perez had a 4.27 ERA as reliever.

    Interesting to see Santos’ WPA so low and Cordero who has been catching flack did rather well.

    You’ve got an interesting conundrum with your list. WPA isn’t great for looking forward as a predictor so what do we expect going forward for the Jays pen?

    FIP didn’t correlate well to your list of success stories because it’s a predictor and doesn’t reflect all that well on what’s actually taken place in the past which is kind of a pet peeve imo when it comes to fWAR and FIP in general.

    fWAR or FIP on the other hand might be the best tool to use if you want to see how the Jays pen will do in 2012. If you’re dying for something to do it would be interesting to see where each team on your list ranked in FIP the year before and then correlate the FIP to WPA and see how good the correlation was.

    There’s a refresher on WPA
    http://www.fangraphs.com/library/index.php/misc/wpa/

  8. “Good starting pitching is your best reliever.” Nice one. That said, you get better where you can, and the pen is way better than that bag of clueless shite from last year.

  9. you should run another regression with offensive WAR, starting pitching WAR and WPA to see how much impact to the bullpen has relative to the whole. Maybe get that r squared co-efficient up to a more respectable level.

  10. as good a read as this was, you could have started talking about RBIs and grit and the 2001 Red Sox result of “LOL” would have made it the best thing I’ve read all day.

  11. It’s nice that the endgame is about buying back future value…I don’t see Coco being here in August unless we’re a half game behind for division. But even then it’s probably better just bring up another reliever from AAA or [insert McGowan to bullpen speculation here]

    • Dunno is McGowan is pen material. I’ve always heard that the pen can be more damaging on an arm as the pitcher has to call upon it more often and has less time to heal between appearances.

  12. Good post as always Drew.

    My favorite part was this from the chart:

    2011 Red Sox 8.59 7.7 90 (LOL)

    The (LOL) was a nice touch

  13. Any reason you didn’t use shutdown/meltdown ratio to analyze the pens? WPA is a great story-telling stat, but quite misleading as if a reliever throws a shut down inning in a one run game the WPA is quite substantial, but if a reliever throws a shutdown inning in a 3-4 run game the WPA is very miniscule, and since relievers can’t control the score when they enter a game it seems like WPA is an odd choice to evaluate them on.

Leave a Reply

Your email address will not be published. Required fields are marked *