Ladies and Gentlemen!
The assessment and investigation of paediatric head injury is a case study in clinical risk management. Similar to chest pain in grown up medicine, it’s a frequent presentation, with a rare but potentially devastating outcome. It’s these kind of patients where we as ED docs come into our own as diagnosticians (or do I mean probablesticians? 😉 ). Head injury in children is also something of an interest of mine; both as a doc, and as the parent of a rambunctious, seemingly fearless, 6 year old boy.
While we all like to feel that our razor sharp clinical acumen is sufficient to pick out the badness, it is nice to have some evidence based back up for our hunches and ‘gut feelings’. So in order to help guide us through choppy diagnostic waters, a great deal of time and effort has been expended to develop clinical decision rules (CDR’s) (or instruments, if you prefer). For chest pain we have TIMI, PERC, HEART to name just a few. Unsurprisingly, paediatric head injuries also have a selection of decision rules to choose from. Of course, this brings with it yet another question; which rule to use? Which is best? This is the question the authors of this study have tried to answer. They have taken their own cohort, and pitted three decision rules against each other in a Battle Royale!
[DDET Contenders…. ready! ]
And in the um, other corner, last but certainly not least, …. from Pediatric Emergency Research Canada, it’s CATCH!
[DDET We’ve got three fine decision rules here, what kind of form are they in? ]
These rules have been compared to each other before. Pickering et al compared the three rules and concluded that PECARN was the most sensitive with an acceptable specificity. This excellent article by Lyttle et al compared the three decision rules and the methods by which they were derived in great detail, and is well worth a read for a textbook example of how to assess a CDR. It is also worth mentioning that PECARN has been validated in a
separate (but demographically identical cohort) as part of the same study in which it was derived in. CHALICE has not been externally validated, but has been extensively implemented and retrospectively analysed since its inclusion in the NICE guidance.
[DDET So how’s this match shaping up? ]
This is the first paper applying the rules to a new, prospectively collected cohort. Their patients were recruited from the ED at a level 2 paediatric trauma centre in the US. 1009 patients were assessed aged less than 18yrs, who had sustained a head injury within 24 hours of presentation.
Patients with a GCS <13, and those with a coagulopathy, were excluded. There were some eligible patients missed, but they seemed to be demographically similar to those included. While not identical to any of the derivation cohorts from the original studies, this population is probably similar to what we all see as ‘minor’ paediatric head injuries, and therefore the population we would want to apply a CDR to. The outcomes looked at were death from TBI, neurosurgical intervention, ventilation for >24 hours due to TBI, which all seem reasonable and appropriately patient orientated to me – focusing on what happens to the patients rather than findings on CT. By the authors own power calculation, a cohort of 1000 patients would be sufficient, so we’re happy there.
[DDET Leeeeeeeeeeeet’s get ready to rumble! ]
As you can see, PECARN comes out fighting with no misses and 100% sensitivity. CHALICE put up a valiant defence with a 85% specificity but falters with a surprisingly low 84% sensitivity. CATCH puts in a solid bout, but is unable to come out on top. So no knock-outs today, so were going to have to ask the judges to make a decision.
[DDET Judges’ verdict]
One hundred percent sensitivity – now that’s the headline figure right there. When we’re dealing with kids and brain injuries and neurosurgery, we want as close to perfect as we can get, and PECARN seems to deliver. This is consistent with the comparison by Pickering et al that also put PECARN on top. Of course, it’s not all about the offence, a CDR needs a good defensive technique too. It’s all well and good to go all out with the sensitivity, but if you haven’t got the specificity, you’re gonna have a problem. Here, CHALICE performs the best, 85% is a impressive specificity for any decision rule, especially if we are concerned about zapping kids brains with CT scans.
The 84% sensitivity of PECARN is a surprise, this is lower than we would have thought. Even more worrying is the fact that the updated NICE guidance has relaxed some of the criteria, so we are not CT’ing all of the children who are ‘CHALICE positive’. This is a concern, and it’s set me off searching for an explanation for this surprising under-performance. The first thing that catches the eye is those confidence intervals, which for the sensitivity results are pretty big. For CHALICE, the sensitivity is 84%, but the 95% CI is 60-97%. The authors themselves point out that as their cohort is far smaller than that of any of the derivation studies, we should be wary about making absolute conclusions. Could it merely be a lack of power that has resulted in CHALICE’s poor performance? That said, even the top end of the confidence interval, a sensitivity of 97%, may not be sufficient for our needs.
[DDET Let’s see the replay… What did CHALICE do wrong?]
CHALICE ‘missed’ three patients, and the authors give us the information to look at. The first is a 17 year old who was hit by a rock. He was picked up by PECARN and CATCH with a severe headache, which CHALICE doesn’t include. Can’t argue with that one. The second was a 16 year old, who scored PECARN and CATCH positive with a GCS <15. He was not picked up by CHALICE, presumably as his GCS was 14 (CHALICE would CT at a GCS of <14). The third was a 15 yr old who fell 2.5 metres. This counts a significant mechanism in PECARN and CATCH, but not in CHALICE. Why not? Because CHALICE would CT after a fall of 3 metres. For me, those last two are pretty subjective, and easy to imagine that they could just have easily have been assessed as being CHALICE positive.
[DDET Last minute allegations of fight fixing! ]
If you take a look at the PECARN decision rule – it has three final outcomes. A low risk, ‘CT not recommended’ group, a high risk ‘CT recommended group’ and also a moderate ‘observation vs CT’ group. The authors of this comparison treated both the moderate and high risk patients as ‘postive’. This gives PECARN an advantage when calculating its sensitivity, and may not reflect how the rule would be used in the real world, where some of those children categorised as moderate risk may have been missed.
[DDET And the paediatric head injury CDR champion of the world is…..]
I was tempted to cop out and call it a draw, but… Sorry Manchester, but the title has gone overseas. PECARN has taken it. While the specificity is not as good as some, they edge it with an impressive zero miss rate. The sensitivity of PECARN may have gotten a boost from the way the authors applied the rule. In real life, we may not send every PECARN positive kid to CT, but we would be at least observing them. CHALICE may have had some bad luck with some decisions not going their way, but on the day, its the numbers that count.