Hockey’s counting problem

alberts orr

So the analytics movement has gone mainstream. I’ve seen references to the website Behind the Net in several major daily newspapers over the last two weeks. And now, Elliotte Friedman wrote a fairly popular blog post on the subject at CBC. His work itself has a line or two that I’d quibble with, but it’s pretty enlightening about the state of analytics in hockey:

The biggest problem for the NHL is the sport just doesn’t have the statistical bent of others.

“We are third, behind baseball and basketball,” [Washington assistant GM Don] Fishman said.

So teams are creating their own. Because there is no consensus, they are notoriously secretive. One thing I believe some teams do is remove “second assists” from players and see how many points are left over. But good luck trying to confirm that.

I find a lot of useful information in hockey has made its way through the Internet. A lot of the statistical bloggers who have been hired to do some consulting work for NHL teams still post quality information on their websites and it’s not like the whole project is going dark.

That said, there are some teams that I know dabble in analytics who come up with some confusing decisions. The Vancouver Canucks, for instance, who have attended the Sloan conference for the last four seasons have tried to make it through the first half of the short season with a single NHL-tested top nine centreman. The Calgary Flames, one of four teams that lists an analytical director as a staff member on the team page, have made an unending number of bizarre offseason decisions resulting in consecutive playoff misses, and on their way for a third. The Buffalo Sabres are the second worst team in the Eastern Conference and fired their coach for the first time in a generation this season.

This isn’t to say that analytics make teams perform poorly, but part of the advantage with having peer-reviewed research in the public sphere is that other people can check your work for inaccuracies. I can imagine a huddled room of Montreal Canadiens executives coming up with the basis for the “plus-minus” statistic, and not realizing there was nothing to devise from the information until years later.

Or not. I’ll admit that being pretty young, I have no memory of anything before the free agency era, and every player acquisition I paid attention to had some sort of number attached to it. X goals, Y assists, Z penalty minutes, whatever it was. I’ve written before about for how much information released by the NHL, so little of it is actually useful. A lot of people suggest that you can’t build a team with just numbers and, I guess until about 10 years ago, that was because there was so little reputable publicly-available information that was useful. I think times have changed to the point where even if a computer couldn’t build a team for you, it could give you some damn good ideas.

One of the things I’ve seen is that it can take a couple of years for a player’s true talent to finally begin to be noted. Players like David Booth, Scott Gomez, Daniel Winnik, Michael Frolik or Patrick Dwyer are useful players if you’re patient enough to wait for pucks to go into the net. Part of the problem with hedging bets is that it can take years for a bet to really pay out, because so much of what happens over the short-term can be attributed to variance.

Mark Cuban said this at Sloan:

“We take input from everybody, but in the end, the GM and owner have to make the final decisions. A lot of it is how you evaluate players organizationally. It’s not about any one individual, it’s what the organization is thinking.

“A lot of GMs measure their own mortality relative to their job. If they feel they’re at risk, they’ll make different decisions than if they feel safe. That’s typical in any job. People want to keep their jobs. Man loves hierarchy. GMs want to feel safe and have longevity, and hopefully they also want to win championships. If he feels there is a risk of losing his job, he’ll behave differently than if there’s no chance he’ll lose his job.”

Basically what you have is teams struggling with concepts they can’t prove, all while working within a very narrow window to succeed. I can imagine if I were a team owner and for two years I was paying money for an analytics department to wrestle with the concept of zone entries while the team lost games on the ice and nobody was paying for tickets, I wouldn’t have too much patience with the direction of the club. The work done by Eric, Geoff, Robert and Corey on entries took a lot of hard work and dedication, and they made a pretty good breakthrough in hockey research. What helped, though, was being able to reach out to the online community for help with tracking individual games and coming up with accurate results.

Finally, baseball guy Voros McCracken, who created “defence-independent pitching statistics” which judged pitchers based on things they could control: homeruns, walks and strikeouts:

“Just because everyone knows OBP is important doesn’t mean OBP isn’t important. Just because we learned something a long time ago doesn’t mean we should unlearn it. We should keep it and add to it. There are a lot of people who are itching to do the next new thing. That’s great, it’s just that mindset can cause you to forget some of the basics.

“You might say to yourself, ‘I want a stat that can measure this.’ Then video technology comes out and gives you the stat you wanted to measure. There is a tendency to think, “Ooh, I’ve been waiting for this, and now I’ve got it, and it’s the greatest stat in the world.” But you haven’t even looked at it yet. You haven’t looked at what it actually says — what its weaknesses are. There’s a hazard there. You want to know more things than your competition. What you don’t want is to know something your competition doesn’t, and it’s wrong. If everybody is wrong about something it doesn’t hurt you too bad, but if you’re the only one, you have 29 teams taking advantage of your mistake.

(That’s from the same Fangraphs link as the Cuban quote)

There’s a reference in the Friedman piece to Craig MacTavish walking around looking for the “Aha!” moment when it comes to hockey analytics. I don’t think MacTavish has realized that half the hockey world is a step ahead of him in that regard. The “Aha!” moment comes when you realize that shots are a hell of a lot more predictive than goals for determining future events. As soon as you realize that hockey is a game between two teams trying to take shots on goal, I think the rest of it falls into place. Shooting percentages, PDO, “clutch” performances in the playoffs… once you get over the hurdle by recognizing that shots are a more sustainable currency than goals, lots of new concepts make sense.

To apply that to McCraken’s point, say you’re a coach who is looking to play the defenceman in his lineup who is the best at exiting the zone, and he has to pick between Andrew Alberts and Albert Andrews. The data shows that Andrew Alberts is the best at getting the puck out of his zone, so the coach sends Albert Andrews to the press box.

What the coach doesn’t see in zone exit data is that his team can’t get shots on goal when Alberts is on the ice. Regardless of how good he is clearing the zone, the team generates far fewer shots relative to the opposition. Shot differential statistics are good because they paint a good picture of what happened to a player when he was on the ice, and microstats are good because they can break up the ‘team’ components of differential stats and show an individual contribution, but I don’t think you want to get carried away by any one particular micro statistic. Some team, for instance, is going to sign the face-off proficient Tyler Bozak to a big money contract this offseason and get burned on the return.

The limitations of numbers as they apply to hockey aren’t so much the “you can’t track a fast-moving game” or “you can’t track leadership” strawmen arguments. It’s moreso that in the rush to obtain information, teams are preventing others from seeing what they’re doing, but at the same time preventing others from adding to their work. I’m unfamiliar with soccer analytics, but I’m thinking along the lines of Manchester City releasing a season’s worth of data to the public. The longer information stays behind imaginary lines, I think the less chance there is of something useful coming of it. It takes patience though, and presumably a lot of people, not only to accumulate the data, but to make something of it, and to have smart people in the background making sure you’re doing it right.

Comments (13)

  1. “The Calgary Flames….consecutive playoff misses, and on their way for a third”

    Bite your tongue Elliott. Let’s see how we do with Kipper back. 3 games in the next 4 nights should be interesting.

    • No credit is given to the Flames for anything ever. Everyone is raving about Leafs and Canucks, both of which we have games in hand on, they each have a few more wins (bfd 20 games in to the season). Flames also had to play over half their games without Kipper, no one talked about how well they managed that. I love the fact the Flames are considered underdogs on a nightly basis, but if you look down the depth chart they are actually much better than people want to give them credit for. This is the best team in CGY for years, including the years they were favoured to win the division before the Canucks dove to the top. Says here Flames are playoff bound and I would not want to play them one bit in the dance. Hoping up and down the red mile the boys can prove me right!

    • This game isn’t like other sports. It’s a sport relied on grit and dirty play. Sure you have your showboats and danglers, but this isn’t roller hockey. And even if it was, the sport wouldn’t last on TV. Remember that roller hockey league back in the day that was on ESPN? It was all about dangling and scoring…then it turned to gritty play like the NHL, lost ratings then folded.

      In my opinion the problem is that hockey is one of those sports where you can play a slow boring style game “trapping” the other team and can win. It’s effective and it works, just look at the Coyotes for example. Me personally, I love hockey and can watch any kind of hockey, but that doesn’t fly with the public, which create ratings and revenue.

      There’s a reason why the Penguins are on TV as much as they are. Crosby and Malkin. Used to be Ovechkin but since dropping out of the All star game Ovi has played miserably. Making a statement? maybe…

      The sport should be back on ESPN (which will help ratings and revenue) but someone in that office is holding a grudge for no apparent reason. And when the Blackhawks were on ESPN the past week for their streak, did any of you guys take notice of how jealous all the NHL “insiders” got on Twitter? For example Dan Rosen..

      I enjoyed your article, but trying to bring up more stats to increase ratings is not the answer. It’s getting back on the biggest sports channel there is and that’s ESPN and SportsCenter.

  2. “The Vancouver Canucks, for instance, who have attended the Sloan conference for the last four seasons have tried to make it through the first half of the short season with a single NHL-tested top nine centreman.”

    …Henrik Sedin?

    • I guess so. FWIW Kesler missed 16 games or 1/3 of the season to be exact.

      • Think that’s the fancy-stats way of saying “Kesler got hurt, now they have no center depth.”

        Also, there’s a bit of an issue here in that it’s difficult to get the answers sometimes. Overwhelming statistical data would reinforce the notion that yes, Vancouver should acquire another center boasting quality PDO/FenClose/Whatever, but how does the club go about getting one? You can’t just snap your fingers and make one appear — if there’s not a market for said player, there’s not a market for said player.

        • “You can’t just snap your fingers and make one appear ”

          If you snap your fingers 3 times, Olli Jokinen will appear and do a little dance for you, then he’ll disappear off into the night with your money. Of course, if you still have money left over, it’s possible to get him to come back.

  3. “The biggest problem for the NHL is the sport just doesn’t have the statistical bent of others.”

    Is that really the biggest problem for the NHL?

  4. “So the analytics movement has gone mainstream.”

    Movement? You’re talking about number geeks trying to look at events a little different, not women organizing for the right to vote.

    The rest of this article is very poorly considered as well.

    NHL clubs do not dabble in “analytics” for posterity sake. They’ve dipped their toes in the water to try and get a competitive edge. Making their findings available to their competitors is such a counterproductive suggestion that I’m still dumfounded 20 minutes after reading it.

    May as well suggest Paul Holmgren give Ray Shero access to his e-mail. Or that Claude Julien should text Michel Therrien photos of his whiteboard the next time the Bruins play the Habs.

    Hell, even some independent analysts don’t publicize the formulas involved in their work for review by other analysts. Yet TEAMS should make it available to COMPETITORS so they can all get something “useful out of it” together?

    Maybe Coke should mail their corporate strategic initiatives to Pepsi, while they’re at it. After all, there’s a bunch of smart people working at Pepsi, and the more smart people there are to review your work, the better the chance that something useful comes out of it.

    I can’t decide if you’re as obtuse as your articles make it seem or just trolling for page hits.

    • I secon this comment in it’s entirety,

    • You’re accusing Cam of trolling? Get stuffed! The truth is hockey would be a better sport if the stats could prove the mike browns and to sesitios out of the game. It could become about goals and scoring plays instead of truculence and grit.

      If the teams bothered collectively releasing collective data (instead of hits and takeaways). they could all benefit, ala baseball. The present revolution in hockey stats could make the product better if it were realised, what with the desire for goals n all.

      Why not make the nhl better with verified data. Dejardins and co have started. Others will follow.

  5. My goodness Larry Dallas is an angry one isn’t he!?

    But really I just wanted to express my amazement at the incredible depths of self delusion Flames fans are capable of. It’s almost like any of them could be the GM and nothing would change. There’s a reason 92% of people who watch or care about the NHL believe the flames have no chance of post season success (or appearance, for that matter); the only reason it’s just 92% is because there are 8% delusional flames fans out there.

Leave a Reply

Your email address will not be published. Required fields are marked *