Some simple math even a scout will understand *pause for laughter*: In a Western Hockey League season, 22 teams each play 72 games. In the Ontario Hockey League, 20 teams play 68 games and in the Quebec Major Junior Hockey League, 18 teams play a 68-game schedule.
All-in-all, not counting overtime or playoffs, that’s 2084 hours of hockey over the course of a season. If I’m lucky, through various streams, television broadcasts, or even live visits to the rink, that means I’ll watch about two games a week, which is reasonable enough. Suppose that’s the case, and I’ll watch 54 Major Junior games over the course of the season, that’s 2.5% of the season to cover.
A scout could maybe watch 5% of the action. The availability of information is not at all like the National League, where every game is televised—in HD—and several people are tracking statistics ranging from blocked shots to which zone a player absorbed a hit. The avid junior hockey watcher has to put up with stats that don’t make sense or they don’t trust, and viewings of prospects are sometimes restricted to a small computer screen wherein you won’t be able to appreciate the intricacies of a player’s game.
You won’t be able to meet a player down below to note if he’s responding well to a hot streak or a cold streak, to just sit there for the pre-game warmup and watch a player prepare, or see him in the hallway prior to the beginning of a game focused on the task at hand, or visit during a practice to see whether or not a player is working hard to add muscle or cardio.
And as weird as it sounds for a numbers guy to endorse, there’s way more value to that side of the game than to worry about the information that comes out of either of the three major junior leagues. I’ve been seeing a lot of complaints, or praise, for hockey players based on their point production after their draft year.
The funny thing about this is that hockey isn’t at the level where it can be landed to statistical analysis as much as baseball, but even in baseball, it was found that the PECOTA system developed by fivethirtyeight‘s Nate Silver, the statistical leader du jour, lost out to traditional scouts in prospect tracking.
From Silver’s book, The Signal and the Noise:
Although it would have been cool if the PECOTA list had gotten the better of the scouts, I didn’t expect it to happen. As I wrote shortly after the lists were published:
“as much fun as it is to play up the scouts-versus-stats angle, I don’t expect the PECOTA rankings to be as accreted as… the rankings you might get from Baseball America.
The fuel of any ranking system is information—and being able to look at both scouting and statistical information means that you have more fuel. The only way that a purely stat-based prospect list should be able to beat a hybrid list is if the biases introduced by the process are so strong that they overwhelm the benefit.”
Looking at point production in junior leagues, if we apply this to hockey, is that there’s a lot more scoring in the Quebec league than the Western league. Rob Pettapiece, who compiles some statistics for the junior hockey blog Buzzing the Net including team RPI rankings and also prospect rankings, said to me in a chat about his 2011 rankings: ”What kind of bugs me is everyone talks about Nail Yakupov as a No. 1 pick, but are they doing so based on more information than I’m using?”
I’m sure they were, but generally, it’s easy to tell the good goal scorers from the not-so-good goal scorers based on the number of goals they score. Measuring defensive ability or playmaking ability is impossible to do with numbers at the junior level. There’s a lot of excitement over point per game rates for defencemen, particularly Morgan Rielly and Dougie Hamilton, but I’m not willing to open up an NHL spot for either as soon as the NHL comes back, purely based on points production.
However, if anything ever came out of the popularity of Moneyball, it’s that even traditionalists are using numbers to back up claims. I watched a game where a junior hockey broadcaster talked about how effective a defenceman was based on “points” and “fights”, which tell you something, but not how good a guy is in his own end. Statistics, I fear, are being overused at each extreme end of the “scouts” vs. “stats” spectrum, and the NHL publishes a lot of information conducive to this.
Some traditional analysts talk about blocked shots and hits but never challenge their models or reasoning. My own belief is that a player that blocks a lot of shots or hits a lot of opponents is making his team worse off in the end because he never has the puck. That said, Silver does make this point:
[Scouts] may have been more concerned about the aesthetics of a player—did he fill out his uniform the right way?—then about his talent. If recent Baseball America lists have been very good, the ones from the early 1990s were full of notorious busts—highly touted prospects like Todd Van Poppel, Ruben Rivera and Brien Taylor who never amounted to much.
It’s that necessary balance, and I think good scouts find their own ways of being able to objectively differentiate between players. Statistics are just note taking, in the end. You can judge a player’s skating based on his form all you want, but what really matters is whether it helps the player get to the other end of the ice quickly and efficiently.
Unlike baseball, hockey doesn’t have “five-tool players”, but like baseball, hockey has players you need to watch over and over and over and over if you expect to come to the right conclusions about a player without recording any data. We can appreciate the difference between a 20-goal scorer and a 30-goal scorer, but the marginal difference between the two is a goal every 19 days, and by then, our eyes are playing tricks on us.
(I watched Nail Yakupov two nights in a row and he failed to record a single point in both games. What am I to believe?)
Billy Beane is quoted in this book. (I talk about baseball a lot on this blog now. Hmm…) He says “what makes a good scout? Finding out information that other people can’t. Getting to know the kid. Getting to know the family.”
The peak for NHL scorers is lower than people would like to think it is. While “prime age” is expected to be around 27 or 28, the reality is that that’s usually the tail end of a player’s premier offensive days. The cheapest and easiest way to get that scorer is in the draft, and as contract restrictions tighten up in the new collective agreement and the salary cap goes down, the teams that draft the best are going to have a significant advantage, just by finding that one extra guy every year or two.
The information Beane is talking about can come in the form of stats. Load up a box score and nobody will tell you how good Dougie Hamilton is at clearing the zone. If you sit next to a guy who’s watched a few Niagara Ice Dogs games this season, he’d be able to give you some indication. Sit next to a guy who has watched every Ice Dogs game this season and tracked the success rate of defensive zone exits by Ice Dog players, you have a much firmer grasp on whether this is something Hamilton is particularly good at.
Ellen calls it microstats. I call it poor man’s note-taking. Either way, somebody is going to have to take those notes, and it’s best left in the hands of somebody who has been around the game for a while so they know exactly what they’re taking, or where they can provide enough data into a player’s overall work habits to let us know whether he is doing enough to make it to the next level.
Once these players have made it to the next level, they’re doing the right things. Until then, it deserves some sort of observation. I don’t know how much just yet, but it requires some.