“Not very successfully right now,” suggested Brian Burke to a few laughs when he was asked at the MIT Sloan Sports Conference about how he evaluates players.

I mentioned on Friday the notable differences between the style of management between Brian Burke and Peter Chiarelli, the other NHL general manager who appeared on the panel alongside Toronto’s Burke. Chiarelli, Boston’s guy, not only won a Stanley Cup, but did so with a team that was supposedly going extinct, brash, bruising brawlers and bullies.

There is a wealth of free information available online that has the capability to give managers and coaches some insight into what goes on when a player is on the ice. It isn’t perfect, and never will be, but it’s useful to know where the puck will be when Greg Zanon is on the ice as a supplement to “well he skates like so-and-so and he hits like so-and-so” which, until only a short while ago, was the norm around the National Hockey League.

Do successful teams keep track of data points that will help their team succeed and properly evaluate their players? Absolutely. Critics tend to see statistics as an attempt of controlling the game with a calculator, and argue that mathematics can’t account for intangible factors present in sports. But those critics fail to understand exactly what a statistic is. It’s just another form of taking notes.

(There’s the old anecdote of Walter Gretzky walking in the living room and seeing his son Wayne trace the location of the puck across the ice on a sheet of paper, so Wayne would know at which spots on the ice he was more likely to grab the puck. This is an early example of advanced analysis, simple note-taking for data that is otherwise unavailable anywhere else.)

Chiarelli was far less dismissive of advanced stats, according to the Edmonton Journal’s David Staples of the Cult of Hockey blog, who noted that Chiarelli uses “different kind of plus/minus numbers” and characterize different situations internally. There’s no specific example listed there, but different kinds of plus/minus numbers could range anywhere from scoring chance differential to Corsi numbers to both.

Corsi, a metric named for its creator, Jim Corsi, a goaltender coach with the Buffalo Sabres, is the simple measurement of all shot attempts, including missed shots and blocked shots, directed at the opponent’s net when a player was on the ice minus the ones directed at its own. Alone, Corsi isn’t perfect, or anywhere close to being perfect. There are so many factors that can influence a player’s Corsi number, but as a standalone stat, it is a surprisingly effective proxy for determining in which zone the puck spent the majority of the time when a player is on the ice.

Now, there are some players whose job is to score, so a good Corsi number won’t satisfy the fans who demand more, but for a player such as Mikhail Grabovski, his strong play, as a two-way guy, will be reflected in a strong Corsi number, which it has for the last two seasons.

Even in Toronto, Mikhail Grabovski is one of the league's most underrated players.

Whether or not Burke re-signs Grabovski in this summer will be an excellent notice as to whether or not Burke uses any sort of advanced data. Whoever does, in fact, sign the underrated Belorussian, will probably be a team that uses numbers to some extent, numbers that show off hidden defensive attributes for the centreman who also has 45 points this season. These are attributes that may not be visible to everybody who watches hockey regularly.

It’s all about process, not necessarily production. I blog for about eight or nine different Vancouver Canucks blogs and am usually recording something of some effect when watching a game. As a result, I would not be able to tell you which players have been scoring goals or are in slumps as of late because I’m so wrapped up in the process of the game. What are the plays that players are making? Are they creating scoring chances, are they moving the puck forward, are they restricting opponent possession, are they eating up tough minutes? It caught me off guard Sunday morning when I suddenly found myself having to defend Henrik Sedin’s play on Twitter because, who knew, apparently he hasn’t been getting many points lately. I’d been so wrapped up in the plays he’s made that usually lead to goals.

You often hear about how baseball lends itself better to advanced statistics because the situations are mostly one-on-ones. I think that baseball lends itself better to acceptance of advanced statistics because the general fan is more willing to accept that sports are played on an individual basis. A player can make a fine play and do everything expected of him, but his teammates may turn over the puck, or not capitalize on a very fine outlet pass to set up a scoring chance.

This was discussed at some length on a terrific radio segment between a couple of Edmonton Oilers’ blogosophere mainstays Allan Mitchell and Tyler Dellow Saturday afternoon:

MITCHELL – “For example, I want to know how many successful passes out of his own zone Tom Gilbert made in a season. We might be able to quantify at least a little bit of the difference between him and [Nick] Schultz.”

DELLOW – “Oh I agree with that, and you’re on Twitter and I don’t know, there’s a guy Derek Blasutti, he’s got a good eye for the game and he commented last week that he could see why Nick Schultz had such a low turnover number compared to Tom Gilbert: because Schultz never passes the puck more than six feet. You know, having that data and understand that data would be fantastic, and [inaudible] that would help us tell what the difference between those two players is.”

Staples, who attended Sloan and wrote much of the hockey stuff that I read this week, is notoriously critical of using team total events for Corsi or scoring chances or +/- to determine a player’s value. Certainly there needs to be some individual aspect involved. Staples counts something called “Neilson numbers” named after Roger Neilson, who modernized video analysis and went through the video tapes after games to count which players made plays that led up to scoring chances, or made mistakes that led to chances against. I’ve hypothesized, however, that Neilson kept the count to only one or two players for each scoring chances due to time constraints.

But when you hear that Chiarelli has internalized some data that he hasn’t released to the public, and Burke goes off pretty convincingly against the statistics, (calling much of it “horseshit” and falsely states that “nobody has ever won a championship with Moneyball”) and looking at the position of those two teams in the standings, it’s apparent that there’s at least some value to this.

As a fan, perhaps it takes a little away from a certain magic of the game, but a grasp of a few of the concepts that are explored in hockey’s numerical blogosphere can still create insight on why certain teams make certain trades, or why your favourite team keeps giving ice time to that guy who sucks.

Comments (11)

  1. The Sabres staff listing has Hockey Analytics Assistant Graham Beamish listed.

    I wonder how many other NHL teams have someone working on analytics?

  2. Intangibles only go so far, Burke. Speed never slumps. Well yes, but if you don’t generate that speed effectively and resort to playing the dump and chase game, guys like Kulemin suffer. The kid has great hands and work ethic but he’s forced to chip the puck deep and fish it out. Much of what Grabo does is because his line-mates are doing the dirty work. Maybe a stat showing how Kuly wins pucks, battles and beats guys with skill and strength won’t go unnoticed and you’d put him in a position where his skill set would be best used.

  3. It’s all about process, not necessarily production. I blog for about eight or nine different Vancouver Canucks blogs and am usually recording something of some effect when watching a game. As a result, I would not be able to tell you which players have been scoring goals or are in slumps as of late because I’m so wrapped up in the process of the game.

    That precisely describe what I have to deal with after 2+ years of recording habs scoring chances. The cognitive dissonance is sometimes stunning; what people are saying of Grabovski is being uttered about Plekanec right now in Montreal.

    I think the Canucks are doing a lot to advance the cause of a more cogent understanding of advanced stats recording and analysis in the general public. Why? Because even tough they never tell what the “secret sauce” is, they openly and vehemently refuse to stray from “the process”, they beat the living tar out of the league and they are doing so in a big canadian market. I wish the habs were half as smart (the demolition derby we’ve seen this season in MTL is sickening).

    Terrific piece of writing.

  4. The first paragraph should have been quoted, being lifted from your article :).

  5. Mr. Charron, I wish you could be at the Sloan conference to report live.

    I’m bringing up a different sport, but I just finished listening to the B.S. Report where Jeff Van Gundy mentions that there is a smattering of luck when it comes to success that statistics cannot measure. Daryl Morey also mentioned that you cannot use statistics alone in a game like basketball (which is similar to hockey imho) because you can analyze a jump shot or a shooting motion of a guard but the situation leading up to the shot is just as important ( I paraphrase).

    I disagree with your statement about baseball somewhat. Baseball is a series of static plays that often affect each other without time taken into consideration. Hear me out:

    A baseball pitcher is only pitching to one hitter at a time, that much is obvious. How he pitches to that hitter depends on the past and future, which is where statistics really come into play with baseball.

    The past can devise probabilities related to the situation. First, what does the hitter traditionally hit well, versus not well. Is there a pitch this hitter will pop up more often than not? Is there a pitch this person will hit into a double-play? These are the superficial statistics at play in the first portion of the analysis.

    Then the pitcher must consider who is on base and where. How does this batter do with runners on base. What do these specific runners tend to do while on base?

    Then the future is considered: who is coming up in the lineup. Am I more likely to have success against another batter?

    Again, these are the superficial stats going into each at bat in baseball. Baseball is a sport that considers a series of one-on-one matchups simultaneously at play.

    Hockey and Basketball are dynamic sports because the play doesn’t necessarily stop to move from offence to defence. Likewise, there is also transitional play that makes up a good part of success in hockey. Players may have one-to-one matchups but each play and the success of that play are dependent on all 12 people on the ice (5-on-5 of course). This is why I don’t like the comparison of baseball to any other sport. Football is similar but there are many variables involved with Football since it is a timed sport. Baseball is the only major sport in the world, save Cricket, that is not timed. There are technically no limitations to the play.

    As for Burke, I think he believes in statistics, but he doesn’t let them run his team. There are many factors involved, especially in the salary-cap system. When Boston acquired Nathan Horton, why was it that Boston was the one to get him over everyone else? Boston had an offer that the Panthers were willing to take and Boston had the pieces in place to make the deal. A part of it just came down to luck: cap space, the right package, and a destination the player wanted to go (expecially with the NTC/NMC we see today).

    Could Burke have made the same offer for Horton, and if so, would Horton agree to go to the Leafs?

  6. There seems to be a battle lately between those who “just keep stats” and those who “just watch the game”, as though stats take the magic out of the game or watching the game without looking at underlying numbers is naive. I think some are forgetting that both sides are still fans of the same game. People are allowed to like the game for different reasons, which is why a 21 year old ex-hockey player can like the same team as a 40 year old mother who’s never touched the puck. In order to get the stats you have to watch the game, so it’s foolish to say one is a real fan and the other isn’t.

    I watch the game without recording advanced stats, but I still like reading about these numbers. I think it gives some context to how teams and players are really faring out there. I couldn’t figure out how to calculate Corsi or PDO for the life of me, but that doesn’t mean I don’t know what it is and what it means for the team. Anyway, keep up the good work Mr. Charron, I always enjoy your articles even if sometimes I’m afraid of what the numbers will tell me.

  7. My fourth year counting chances and I am terrible when it come to Oilers boxcar numbers.

  8. The problem with a stat like Corsi numbers in hockey is that not all “scoring chances” are equal. A weak shot from the point is less likely to lead to a goal compared to a breakaway. So while a guy like Grabovski may generate higher Corsi numbers, that doesn’t mean he’s more likely to generate good scoring chances.

    Also, a good checking forward who blocks alot of shots won’t have good Corsi numbers, but does that make him less valuable than a forward who only knows how to play with the puck?

  9. I was at the conference and attended the Hockey Analytics Panel. While Brian said some pretty Burke-type (I believe my horseshit count was around 9) things i was somewhat surprised by his lack of appreciation for analytics. One would think a GM would be seeking any advantage possible.

    That being said, he went on to explain that analytics, as of right now, did not have the same real-life implications as one might see in baseball or basketball. It just hasnt advanced far enough yet. He went on to say that he must receive at least 20-30 research papers from MBA students every year. He made a point of mentioning that while he does not read them “someone does”, and his team has yet to find an idea of “moneyball” proportions.

    In terms of recruitment analytics, Burke also cited the difficulties associated with the fact that in the NHL we are often scouting teenagers who have yet to reach their potential both in terms of skill and physicality.

    Most importantly, Burke cited the importance of values of work ethic offering a great Trevor Linden anecdote wherein Linden had to call and tell Burke that he would be unable to make and upcoming combine due to a family farm obligation wherein his primary job was to “hold the calf down so someone can brand it and cut it’s nuts off”. Burke recognized the level of dedication Linden had as well as his deeply entrench family values and genuine personality.

    While there are many other factors to be considered, one cannot ignore heart and work ethic among the most important ‘statistical’ elements to be considered.

  10. First let me start off by saying that this was a great article. Really a fantastic read.

    As far as advanced analytics, I agree with Sandwiches that its application to baseball is unique because it is a game made up of many individual plays which, though they are part of one game, are quite seperate and stand alone.

    In regards to their application to hockey, I think we are just beginning to scratch the surface of their usefulness in the game. They certainly have a place in hockey but as many of the commenters have pointed out, they alone do not tell the story and there are many other important aspects that must be looked at in regards to players, certain “intangibles” for lack of a better word. In the end, to truly know a player’s worth, teams will still have to use scouts. (I think there are many similarities to soccer’s use of these types of stats)

    Lastly, though Burke doesn’t seem like the biggest proponent of advanced analysis, I doubt he totally ignores them. He’s a smart hockey guy and a smart guy in general (HLS grad) so I’m sure he’s keeping an eye on them (his signing of Grabovski might be proof of that).

    Sorry for the long comment but please keep these great articles coming!

Leave a Reply

Your email address will not be published. Required fields are marked *