Ed – Tom Shanahan (here in Virchester) recently published a paper on whether a Dutch prediction model is better at identifying major trauma patients than existing methods. The publication provoked an interesting debate on Twitter and so we have asked him to summarise and explain the study in more detail here. The project used data provided by the MAjor Trauma Triage Study (MATTS) to validate major trauma triage tools (https://www.sheffield.ac.uk/matts).
You can find the full paper here https://authors.elsevier.com/a/1cZIN4b3HU3b4 and the abstract below.
Tom – The aim of the study was to try to externally validate the Dutch model for identifying major trauma patients using UK data. As all emergency clinicians will know, current prehospital prediction tools are not perfect with both undertriage and overtriage of patients. Ideally, those patients who need trauma centre care will be transported directly to a trauma centre (if geography permits), but getting the decision right is difficult.
Why does major trauma triage matter?
Identifying major trauma patients and making sure they get to the right place, first time is important. However, evidence is limited on the best way to do this.
In the South West (and across the UK) use a major trauma triage tool to make decisions about disposition. The tools are not validated. The tool uses a step by step approach. Firstly, vital signs and conscious level is assessed. Secondly, anatomy of injury is determined. Thirdly, mechanism of injury is evaluated. Fourthly, special circumstances are considered i.e. pregnancy, older than 55 years, etc. Fifthly, does the patient had airway compromise or catastrophic haemorrhage. If yes then go to the nearest hospital to stabilise. Lastly, how long will it take to bypass the local hospital to reach a Major Trauma Centre (MTC)? If it is up to 60 minutes then bypass the local hospital and go to the nearest MTC.
In the UK, decisions about whether a patient had major trauma is made retrospectively, based on whether the Injury Severity Score (ISS) was greater than 15. As discussed later, despite ISS being the main reference standard used, it is contested.
Under-triage, whereby we incorrectly send a patient with major trauma (defined as injury severity score > 15) to a Trauma Unit (TU), can result in increased morbidity and mortality. However, over-triage, whereby we incorrectly send a patient without major trauma to an Major Trauma Centre (MTC) results in over-crowding and wasted resources.
Recommendations from the US for developed trauma systems are to allow for an under-triage rate of 5% and an over-triage rate of up to 35%. There is no set rate in the UK. In the Netherlands, an acceptable rate of under-triage was set at 10% and over-triage up to 50%.
In 2019, researchers from the Netherlands developed and validated a prediction model for the triage of trauma patients (https://europepmc.org/article/pmc/pmc6537785). The Dutch model had eight predictors: age; systolic blood pressure; GCS; mechanism criteria (fall greater than 2 metres, motor vehicle collision and any type of entrapment); penetrating injury to head, thorax or abdomen; signs and/or symptoms of head or neck injury; expected injury in the Abbreviated Injury Score (AIS) thorax region; and expected injury in 2 or more AIS regions.
The Dutch prediction model had an under-triage rate of 11.2% and an over-triage rate of 50%, which was impressive, but as we know, prediction scores will always perform well in the cohorts and localities from which they were derived. The real test is whether the model works elsewhere, and in fairness this is what the Dutch team recommended (external validation), and is what we have done.
What did we do?
Retrospective data was provided by the South Western Ambulance Service Foundation Trust (SWASFT) for the period 1 February 2017 to 1 February 2018. Prehospital data provided by SWASFT was matched with data submitted to the Trauma Audit and Research Network (TARN) over the same time period. TARN is the database that records major trauma patients reaching hospital in the UK and which has been running for decades.
The overall cohort had 68799 patients. All patients 16 years and older with a suspected injury and transported by SWASFT were included in the study. Within the 68799 patients, 1624 patients were linked to the TARN database.
Following TRIPOD guidelines for external validation the accuracy of the prediction model in terms of discrimination, calibration, sensitivity (under-triage) and specificity (over-triage) and clinical usefulness was tested in the overall cohort and in the TARN linked patients.
What were the results?
First, the demographics. The median age of patients in the South West were 72 years (i.q.r. 46-84); 55.5% were female; and 524 (0.8%) had major trauma (ISS>15). Patients with major trauma (ISS>15) had a median age 66 years (i.q.r. 43-83); 42% were female and the median ISS was 22 (i.q.r. 17-26). In comparison, the original Dutch cohort was younger (45 years), more were male (58.3%) and more patients had major trauma (8.8%).
Discrimination evaluates whether patients who have the outcome have higher risk predictions calculated by the model than those who do not. C-Statistic was 0.75, 95% CI, 0.63-0.78. Which is classed as good discrimination.
Calibration measures how closely predictions made by the model match observed outcomes. Calibration was not very good in the external validation. This can be seen in the supplementary materials
We looked at under-triage and over-triage rates. In the South West existing triage methods resulted in under-triage rate of 56% and an over-triage rate of 16%. Existing methods are good at finding patients that do not have major trauma and appropriately sending them to the nearest hospital.
The Dutch prediction model in this cohort would result in reducing under-triage by 39%. However, it would increase over-triage by 34% compared to existing methods.
Clinical usefulness analysis suggests the model has potential benefit. The South West regional system would have to accept 49 patients wrongly classified as major trauma for every 1 correctly identified with major trauma.
This is likely far too much in already over-stretched systems. We did a sub-group analysis of 1624 TARN linked patients. So those injured patients at high risk of being classified as major trauma. In this group, which is closer to the original Dutch cohort, the discrimination was good, C-Statistic 0.75, 95% CI, 0.72 – 0.77. Calibration showed very good visual agreement between predicted and observed probability of major trauma. The Dutch model in the sub-group would result in an under-triage rate of 12% and an over-triage rate of 50%. In this case, the regional trauma system would have to accept 6.5 patients wrongly classified as major trauma for every 1 correctly identified with major trauma.
What does this mean for trauma systems in the UK?
The question regional trauma systems have to ask themselves is how much accuracy do they want in triaging patients to MTCs, if they have major trauma? Is it acceptable that 56.1% of major trauma patients are treated in non-MTCs? Other research has shown 59.8% of major trauma patients are treated in non-MTCs in the UK (https://academic.oup.com/ageing/article/49/2/218/5639746). Furthermore, in an inclusive trauma system is it desirable to equate ISS>15 with major trauma and subsequently with the need to be treated in an MTC? I think most colleagues working in trauma, will say it depends.
However, the above question is related to a more fundamental one, about whether ISS>15 is the best reference standard to determine whether a patient had major trauma. Alongside ISS, a range of anatomical, physiological, prognostic and resource based (need for life-saving interventions) measures have been proposed as reference standards for major trauma. It is beyond this study to say what is best. But it is definitely an area for further research.
A note of caution. The effectiveness of pre-hospital triage is more important than the theoretical accuracy of triage tools. This should be evaluated by looking at whether severely injured patients bypass the local hospital and are sent to an MTC. In addition, did the triage tool result in a pre-alert and trauma team activation? This would reflect the extent to which a triage tool is applied in practice. Therefore, these results should be interpreted with caution prior to a large-scale implementation study.
We should also be mindful that these results are in adults and not in children.
Conclusion
This external validation using a retrospective cohort showed theoretically the Dutch prediction model could lower under-triage rates to 17%, however, it would increase over-triage rates to 50%. Further prospective research is needed to determine whether the model can be practically used by paramedics and whether the model’s use is cost-effective.
What is next?
This study is only the beginning. The Dutch team are prospectively testing the model within an app (https://diagnprognres.biomedcentral.com/articles/10.1186/s41512-020-00076-1#:~:text=The%20Trauma%20Triage%20App%20(TTApp,probability%20of%20being%20severely%20injured). This will provide real world data on whether the model decreases under-triage rates of major trauma patients. The MATTS team are working on a specific prehospital trauma triage tool for the UK.
In addition, the study has raised a number of questions, which provide plenty of food for thought for future research. For example, which injured patients benefit most from MTC care in an inclusive trauma system? Is ISS the best reference standard for severe injury, if not what is? What are the most important outcomes in major trauma for patients, clinicians and the system? Those questions are for another time.
Acknowledgments
Co-authors: Dr Gordon Fuller, Professor Trevor Sheldon, Emily Turton, Fionn Quilty and Dr Carl Marincowitz
As part of the MATTS: MAjor Trauma Triage Study data was provided by South Western Ambulance Service NHS Foundation Trust (SWASFT) and Trauma Audit and Research Network (TARN). We would also like to acknowledge Professor van Heijl and colleagues from University Medical Center Utrecht for their assistance with specifying the prediction model.
References
Shanahan T, Fuller G, Sheldon T, Turton E, Quilty F, Marincowitz C. External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom. Injury February 2021. https://authors.elsevier.com/a/1cZIN4b3HU3b4
van der Sluijs R, Fiddelers AAA, Waalwijk JF, Reitsma JB, Dirx MJ, den Hartog D, et al. The impact of the Trauma Triage App on pre-hospital trauma triage: design and protocol of the stepped-wedge, cluster-randomized TESLA trial. Diagnostic Progn Res [Internet]. 2020 Dec 18 [cited 2021 Jan 2];4(1):10. Available from: https://diagnprognres.biomedcentral.com/articles/10.1186/s41512-020-00076-1
Dixon JR, Lecky F, Bouamra O, Dixon P, Wilson F, Edwards A, et al. Age and the distribution of major injury across a national trauma system. Age Ageing [Internet]. 2020 [cited 2020 May 15];49:218–26. Available from: https://academic.oup.com/ageing/article-abstract/49/2/218/5639746
van Rein EAJ, van der Sluijs R, Voskens FJ, Lansink KWW, Houwert RM, Lichtveld RA, et al. Development and Validation of a Prediction Model for Prehospital Triage of Trauma Patients. JAMA Surg [Internet]. 2019 May 1 [cited 2019 Oct 14];154(5):421–9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/30725101
Major Trauma Triage Study (MATTS) https://www.sheffield.ac.uk/matts
Rick Body, “JC: Paediatric Major Trauma Triage,” in St.Emlyn’s, April 11, 2014, https://www.stemlynsblog.org/jc-paediatric-major-trauma-triage/.
