And sometimes you don't need an algorithm to tell you your knee is jacked.


The Ottawa Knee Rules are fantastic. They’re a list of five simple tests to do on someone with a knee injury that identifies the need for an x-ray. You’ve fallen and whacked your knee, and it hurts like stink. If any of the following are true, you probably have a fracture and need an x-ray: You’re older than 55, your kneecap (and nothing else) hurts to touch, the top of your fibula (one of those lower leg bones) hurts, you can’t bend your knee to 90 degrees, or you can’t bear weight on the leg for four steps. The Ottawa Knee Rules have been tested to show 100% sensitivity (OH GROSS STATISTICS MAKE IT STOP), meaning that 100% of the people it identifies as needing an x-ray due to a possible fracture really do actually have a fracture.

Medicine loves a good algorithm. And good medicine can combine algorithms to solve the problem at hand using the same methods and hopefully yielding the same results each time. In a moment of post-West coast playoff game brain fog, I began to wonder why you couldn’t apply an algorithm to supplemental discipline, as opposed to the current method of whatever the hell it is they’re using (Alchemy? Dice? Majority vote in a room full of toddlers?).

Uh, what’s an algorithm?

It’s a fancy way of saying a list of steps used to solve a problem. It’s shaking the remote, whacking it, then changing the batteries when it won’t change the channel. It’s realizing the milk expired a few days ago, sniffing it, tasting it, then grudgingly going to the store for more. The advantage of well thought-out algorithms is that they tend to be very sensitive, and they yield reproducible results.

Whoa, whoa, whoa. Was that statistics again?

Yes. And I hate statistics as much as you do. Probably more. The point is that an algorithm worth a damn will deliver the correct results in certain situations (ie. identify a fractured knee) every time you use it.

I’m bored, you’d better connect this to hockey ASAP.

If you could come up with an algorithm for punishing a bad hit in a hockey game, and used it every single time, you’d come up with consistent punishment every time.

I have to stop here and point out I’ve made several (currently laughable) assumptions. I’m assuming the NHL has an interest in punishing all offenders equally, whether their name is Shea Weber or Andrew Shaw. I’m assuming the NHL has an interest in sending a message to offenders that bad hits won’t be tolerated, and that repeat offenders REALLY won’t be tolerated. I’m assuming that the NHLPA, GMs and owners are willing to lose time and money and appearances by their star players. I’m assuming all of that, and wondering just how hard it would be to write an algorithm for each situation requiring supplemental discipline.

Can we try?

Sure. Proceeding with the understanding that there are a variety of situations that require supplemental discipline (and thus likely the need for a variety of different algorithms), let’s just pick one. Let’s pick illegal checks to the head. As a refresher:


48.1 Illegal Check to the Head – A hit resulting in contact with an opponent’s head where the head is targeted and the principal point of contact is not permitted. However, in determining whether such a hit should have been permitted, the circumstances of the hit, including whether the opponent put himself in a vulnerable position immediately prior to or simultaneously with the hit or the head contact on an otherwise legal body check was avoidable, can be considered.

48.5 Match Penalty - The Referee, at his discretion, may assess a match penalty if, in his judgment, the player attempted to or deliberately injured his opponent with an illegal check to the head.

If deemed appropriate, supplementary discipline can be applied by the Commissioner at his discretion.


Step 1: The Hit

Player A hits player B in the head (from behind, after the puck is gone, after the whistle, etc). The referee decides this was a bad play and assesses a match penalty. Player A stomps off to the locker room. Player B is or is not injured.

Step 2: Pick your algorithm

This is where the is/is not injured comes into play. As such, here comes another assumption. I’m going to assume supplemental discipline is influenced by whether or not an injury results from the play. Before the screaming starts in earnest, consider the following: Criminal prosecution is influenced by the outcome of the crime. If I punch you in the face and all it did was piss you off, I wouldn’t be charged with the same thing as if I punched you in the face and put you in a coma. No, I’m not going to punch you in the face. Medical algorithms are also dependent upon the severity of the situation. If your knee hurts and you pass Ottawa Knee Rules, I’m giving you naproxen and sending you home. If your knee hurts and you fail Ottawa Knee Rules, I’m sending you to radiology.

Another assumption regarding injuries is that the doctors diagnosing the injury are operating solely in the best interest of the patient. That is to say that they would be completely impartial and disregard any inappropriate diagnosis/treatment (or non-diagnosis/treatment) suggestions from GM, coach, league, player, etc. All physicians should be impartial. That’s their job. That’s also completely untrue. Physicians don’t always do what’s in the patient’s best interest, choosing (or being forced to choose) what’s in the best interest of their employer, themself, or some other entity. The assumption of impartial diagnosis and treatment isn’t ridiculous if an injury is obvious – a broken bone is a broken bone. It’s a difficult assumption when it comes to concussions. You can’t diagnose a concussion with a blood test, they don’t show up on x-rays or CT scans, and a large part of their diagnosis is based on what the patient claims to be experiencing. A truly determined individual could fake their way into or out of a concussion. Neuropsych testing is a great tool, but it’s only one aspect of concussion management.

Step 2 (for real this time): Pick your algorithm

Let’s say Player B has a broken nose as a result of the hit. Supplemental discipline heads down the path of the head hit/injured algorithm.

Head hit: 1 game + monetary fine. (baseline suspension)


  • Minor? Player in question likely to miss one week of play? Add a game + fine (player).
  • Moderate? Player will miss up to a month? Five games + fine (player).
  • Major? Season is threatened? Ten games + fine (player and team).
Repeat offender:
  • First time repeat: Add three games + fine (player and team).
  • Two or more infractions this season: Ten games + fine (player and team).

The games or fines I’ve suggested aren’t important. The levels would have to be set through agreement with GMs, NHLPA, league, etc. The point is to have a defined procedure.

Step 3: Do this every single time

Put it in the rules. Write it down. Lay it out for everyone to see. Nobody is going to whine that big name players get special treatment if everyone is subject to the same algorithm. And players are less likely to reoffend if they know how far their punishment will escalate.

Step 4: Complaints (because I know you have some)

I can already hear the cries of “This is too subjective! No two injuries are the same!” That may be true. But it’s also true that no two cardiac arrests, broken legs, or facepunches are the same, and they’re all dealt with through algorithms. You can’t take subjectivity out of the equation. A ref’s decision to assess a match penalty is subjective. A doctor’s decision to send you home with tylenol instead of scanning your head because you have a headache is subjective. But even in their subjectivity, these two examples involve an algorithmic decision to arrive at an outcome based on experience, the available information, and a set of established guidelines.
The aim isn’t to create a completely perfect system. That’s impossible in any construct involving humans. The aim is to create a system allowing for a standardized method of dealing with the problem at hand, in this case dumb things hockey players do that require supplemental discipline. The aim is to establish a consistent* outcome when one of these problems arises. This is done with the understanding that this is a system run by humans, so there will always be an element of subjectivity. The hope is that through the application of an algorithm that you can get all the humans involved to be more consistent*. If criminal prosecution, medicine, and changing the remote batteries benefit from algorithms, then why couldn’t the NHL?

*Consistent being a relative term because, again, we’re human, so 100% consistency is impossible.

Comments (5)

  1. Really good piece. I found this by accident and it was worth it.

  2. Wait, wait, wait… are you going to be one of those rare ER docs who actually applies the Ottawa Knee and Ankle rules? That may be the most shocking part of this post ;)

    An interesting side note: the Ottawa ankle wikipedia page notes one study that concludes “teaching the rules to patients does not appear to help reduce presentation to hospital.” I imagine, though, that teaching headshot suspension rules to hockey players would reduce their presentation of head shots.

    … which seems to be what Shanny’s trying to do with his videos: a clear play-by-play of each hit, what was wrong with it, key factors in his consideration, and the outcome. He gives us all the steps, but we still don’t quite see his algorithm (if there is one). It’s still a step ahead of Colin Campbell, but it’s not quite there yet. There’s still a lot of room for teams to be, in the words of the Rangers, perplexed.

  3. Great article. Please note, for what it’s worth, that your definition of “sensitivity” is wrong. If the Ottawa Knee Rules are 100% sensitive, that means that they ALWAYS correctly assign a fractured knee into the X-ray group. You phrased it as “100% of the people it identifies as needing an x-ray due to a possible fracture really do actually have a fracture”, but it’s actually best phrased in the opposite way: 100% of people who really do have a fracture are [correctly] identified as possibly having a fracture.

    A test which is 100% sensitive will almost invariably include some false positives in the positive category: so the Ottawa Knee Rules, being used to assign a bunch of injured knees into either the possible-fracture-therefore-Xray group or the no-fracture-send-home group, will assign every truly fractured knee and some non-fractured knees into the Xray group.

    The best way to describe a test which has the features you describe (“100% of the people it identifies as needing an x-ray due to a possible fracture really do actually have a fracture”) uses the concept of predictive value, rather than sensitivity. A test with this characteristic, in which 100% of the patients tested as POSITIVE actually do have the disease, has 100% positive predictive value (PPV).

    As it turns out, the positive predictive value of the Ottawa Knee Rules is about 11% (according to Emparanza JI, Ann Emerg Med, 2001;38:364-368)–which means that although the test does correctly identify everyone who may benefit from a knee Xray, the overwhelming majority of patients it assigns to the Xray group still do not have a knee fracture.

  4. I should add that these concepts (medical test sensitivity, PPV) are difficult, and most medical students (and even practicing physicians?) probably don’t use them properly.

  5. Awesome post. The only subjectivity should be if the hit was illegal and if there was intent to injure. The league can overturn the referee’s call – for better or for worse – in supplemental review. May be an opportunity for the league to let the stars off but there needs to be some form of checks and balances. Totally agree the discipline should be consistent and predictable. Shanahan’s “transparency” in his videos is a noble concept but I’d rather see explanation of why the refs’ got it wrong (or right) than the inconsistent reasoning of his disciplinary decisions.

Leave a Reply

Your email address will not be published. Required fields are marked *