You can use numbers to prove pretty much anything.
The NHL publishes five types of RTSS, or “real-time scoring statistics”. These are “hits” “blocks” “missed shots” “giveaways” and “takeaways”. They produce a mountain of data and quite oftentimes, there’s so much numbers available that people are constantly adding and subtracting and dividing these numbers to paint a rational picture of why teams are succeeding as they are.
This tweet from Sault-Ste. Marie Greyhounds head coach Sheldon Keefe caught my eye:
Hard to hit when you have the puck all the time.“@robertjftc: Chicago is last in the NHL in hits. Their record is 15-0-3.”
— Sheldon Keefe (@SheldonKeefe) February 25, 2013
I’ve never met Sheldon, but I have heard that he has “a real affinity for advanced stats”. I’ve seen the RTSS used so many different ways. I saw a blogger two years back mention if Detroit is such a good hockey team, why are they always ranked so low in takeaways and blocked shots? (Her conclusion was that the Red Wings were flawed, not the numbers) I’ve seen people credit Toronto being first in hits for why they’ve improved this season (side note: after 19 games this season, the Leafs have 22 points. After 19 games last season, the Leafs had 22 points). I’ve seen defencemen judged by their “giveaway:takeaway ratio”.
The most egregious over-analysis of these numbers was CBC during last playoffs who added blocked shots and hits together to form some sort of catch-all “grit” rating that had the Rangers ranked very high. PJ Stock alluded to both numbers in the pre-game show for the Leafs game against Ottawa on Saturday. (Don Cherry Saturday mentioned toughness as a reason the Leafs are improved by, again, zero points)
The problem with these numbers is that they lie and that they really mean the exact opposite of what you’d think they mean. In the real world, a “giveaway” is actually preferable to a “takeaway”. In the real world, “blocked shots” correlate so highly with losing you may as well just be counting goals against. In the real world, the importance of “hits” is imaginary.
Let’s start with the obvious problem of these RTSS. They’re counted differently across every building in the league. The NHL separates them by home and road, so what I did is look at a bunch of different numbers including four of the RTSS and compared a team’s ability at home to that same team’s ability on the road. Basically, if a team gives away the puck a lot at home, they should give it away a lot on the road, right?
An r-squared of 0 implies a correlation of zero. An r-squared of 1 implies perfect correlation. The numbers in this post use data from the last three full seasons:
‘Fenwick Close‘ is the most repeatable of all these numbers. A team’s Fenwick rate is the number of unblocked shots taken by a team at the opponent’s net divided by the total number of unblocked shots by both teams. “Close” refers to the state of the game at even strength where the score is tied, or there’s one goal separation between the two teams in the first and second periods.
Goals, goals against and goal differential are all
statistically significant substantial to some degree. Giveaways and takeaways are defined so differently from building-to-building that it’s hardly even worth comparing the two. Hits is also quite low. The Maple Leafs lead the NHL last season with 1279 hits at home and Calgary was last with 672. The Leafs had 939 on the road, and the Flames had 931. It’s almost identical when you make the setting a little more neutral.
That’s the first problem with RTSS: looking at “total” numbers isn’t a reflection of a team’s ability in a certain category. You could learn some things if you only looked at road numbers, though. Which brings us to our second problem with RTSS:
Of the four RTSS categories used by broadcasters, just one of them is in any way predictive of a team’s record. While commentators will bring up the Rangers’ ability to block shots (4th in the NHL with 1338 last season) they’ll invariably never mention good teams who do poorly in that category who are also good. The Los Angeles Kings and New Jersey Devils, for instance, were 29th and 30th in the NHL in blocked shots last season. Three teams that missed the playoffs (and two lottery teams) in the NY Islanders, Minnesota and Montreal were 1-2-3 in the league.
I wanted to see which of the above statistics was closer linked to wins. I looked at only the “road” number to see if it could predict the number of points the same team earned at home. Again, an r-squared of 0 implies a correlation of zero. An r-squared of 1 implies perfect correlation:
Fenwick Close on the road, an obscure statistic found off on some random corner of the Internet, is actually the most predictive of a team’s home record—more predictive than even road wins, road goal differential, or points earned on the road.
And that’s something Hockey Night in Canada will never mention, at least not with PJ Stock or Glenn Healy on the camera who constantly slam their fists talking about the importance of toughness. Teams that have the puck the most tend to the win the most games. All of the useful categories involve some way or another having control of the puck.
Look at the bottom of the chart. The correlation between road blocked shots or road goals against and home points is pretty high, only the correlation is negative. What that means is that the teams that give up the most goals and block the most shots are the ones who are losing games. I find this fascinating, that blocked shots can have such a negative affect on a team’s record.
Less fascinating because the correlations are very small, but also hilarious, is that teams that give the puck away more are slightly better off than teams that lead in “takeaways”. Again, look at what Keefe said above—if hits are indicative of a team not having the puck, so are takeaways. Giveaways are an indication that a team has had the puck at some point.
So what’s the point of all this? Don’t trust what the raw numbers say. I’ve gotten used to calling advanced stats “performance metrics” to make them sound less elitist. That’s in contrast to “production metrics” like goals and assists and wins and losses. Fenwick Close % is a repeatable talent for teams both home and away that correlates more with a team’s record than any available number on NHL.com including wins. It’s predictive of success, and that is why I use it so often.
The mainstream success of Moneyball, if anything, has convinced a large audience of sports fans that there’s value to be found in using statistics with arguments to make points. The problem is that leagues have so much information available that it takes people a long time to figure out what’s worthwhile and what isn’t. To the casual fan, who gets into an online skirmish or two every month and watches Hockey Night each week, blocked shots are fine. The problem is that the numbers used often by broadcasters like the CBC who preach toughness at every available opportunity is that they weave a frail fiction of the game, and all too often, the statements of the PJ Stocks of the world go un-checked, leading to half the population of the Greater Toronto Area walking around like zombies and talking about how important Frazer McLaren has been to the Leafs’ this season.
By the way, as of Sunday night, Toronto is first in blocked shots on the road and 30th in road Fenwick Close %. The Anaheim Ducks are 9th in blocked shots on the road and 27th in road Fenwick Close %. I’ll let you guys guess what happens next.