This blog post is based on the paper published in the EMJ this week – Body R et al. Can emergency physicians rule in or rule out acute coronary syndromes using clinical judgement? EMJ 2014 [Online first].
Anyone who’s worked in Emergency Medicine for any length of time will appreciate that an acute coronary syndrome (ACS) is one of the most difficult diagnoses to rule in or rule out in the Emergency Department. We’ve probably all seen the old stat that 2% of all acute myocardial infractions (AMIs) are missed in the ED. You might also have seen the stat that up to 7% of patients discharged from the ED have prognostically important myocardial damage, which by today’s standards would be considered as acute myocardial infarction. Perhaps that’s why missed ACS is one of the leading causes of medical litigation.
[DDET Why is this such a hard diagnosis to make in the ED?]
Maybe we struggle because, as doctors, we like to use our clinical judgement. We like to think that we can use the clinical information we have to make diagnoses without having to rely on tests. Perhaps we also feel like we lose face if we admit patients for investigation when patients don’t actually have the diagnosis we investigated them for. Don’t you hate it when someone from an admitting team catches you in the corridor and says something like, “Hey, you remember that patient you referred to me yesterday with query ACS? We sent him home – it was just gastritis”? It makes you feel inferior – like you should have been able to know that if your gestalt had been a bit better or if you’d have been just a bit bolder.
However, the literature is quite clear about this. If the patient’s symptoms consist of acute discomfort or pain in the chest, epigastrium, jaw, neck, throat or arms and you haven’t otherwise explained it, you’re right to consider ACS – even if the symptoms may not seem so convincing. This is based on good evidence that patients with ACS often have atypical symptoms. Taking individual symptoms, for example, the character of the pain, the radiation of the pain and the number of risk factors a patient has don’t affect the probability of ACS to any significant extent, as you can see from these slides taken from my talk at SMACC Gold (based on our own research – which you can find here).
Even grouping symptoms together as ‘typical’ or ‘atypical’, the patients with atypical symptoms are no less likely to have ACS than those with typical symptoms – as shown this great study from Louise Cullen’s group, which I was honoured to be involved with.
[DDET So what did we do in this research?]
We ran a cohort study. The main aim of the study was to validate the MACS decision rule but we also ran several substudies using the data collected. In this one, we wanted to know about the diagnostic accuracy of emergency physicians’ clinical judgement for acute coronary syndromes – both alone and in combination with the tests available on arrival – troponin and the ECG. To be honest, we didn’t expect to find a useful ‘rule out’ strategy – we simply wanted to quantify the accuracy to better inform our practice and to find out whether we might be able to refine and improve the MACS rule by incorporating gestalt.
In this study we asked emergency physicians to record their ‘gestalt’ or overall clinical judgement for ACS using a 5 point Likert scale (from ‘definitely ACS’ to ‘definitely not’ ACS. We then cross-tabulated this with the patients’ outcomes – and the outcomes we were interested in were a diagnosis of AMI and the occurrence of major adverse cardiac events (MACE) within 30 days.
[DDET What did we find?]
The bottom line is that, for patients in whom we’ve already considered the possibility of ACS as a diagnosis, our ‘gestalt’ regarding the overall probability of that diagnosis can’t be relied upon by itself to rule out that diagnosis. Nor can gestalt be used to ‘rule in’ the diagnosis – only half of those whom clinicians felt ‘definitely’ had ACS actually had AMI or developed MACE within 30 days. Here’s another slide from my SMACC Gold talk to illustrate that.
[DDET Is that all there is to it?]
No, wait, there’s more. And here’s where the story gets really interesting. Nobody in their right mind would consider the possibility of ACS without doing an ECG, right? So it’s probably unfair to assess clinical judgement alone without at least taking account of the ECG. We’re unlikely to want to discharge these patients if they have ischaemic ECGs, even if we do think that ACS is still unlikely. What’s more, we have another amazing test that we’d always run in these patients – troponin. We run that test on arrival too, and again we wouldn’t send the patients home if they had a positive troponin on admission – even if we did think that ACS was unlikely. So we looked at the overall diagnostic value of gestalt combined with the ECG and initial troponin level.
What we found may seem surprising. If we discharged patients who had a normal ECG, a normal troponin and we felt that the diagnosis was ‘probably not’ ACS, then we wouldn’t have missed a single case in this cohort – the sensitivity was 100%. Almost a quarter of patients could have been discharged using that strategy.
If we used a high sensitivity troponin assay rather than a standard contemporary one, we could also have discharged patients in whom we felt the diagnosis ‘could be’ ACS without missing any cases – 100% sensitivity. Using this strategy, over 40% of patients could potentially have been discharged – without missing a single AMI in this cohort.
And one of the really great things about this strategy is that it didn’t seem to matter if it was an experienced consultant or a junior doctor giving their gestalt – we still achieved 100% sensitivity.
[DDET What does that mean for our practice?]
Before anyone gets carried away by these promising findings, it’s important to recognise that there are some limitations to this work. The 95% confidence intervals for sensitivity extend down to 95.6% and the first reports of new diagnostic technology often overestimate performance (commonly due to reporting and publication bias). Therefore we need to validate these findings – first in observational studies and then, if they still show promise, in interventional trials. Of course, doctors might be slightly less bold when it comes to stating their gestalt if they knew that they’d have to send patients home based on their estimates. We can only find that out by evaluating the strategy in practice.
This means that you shouldn’t use gestalt to rule out ACS right now, even in combination with the ECG and troponin. But you can rest assured that the probability of ACS in patients with normal troponin and ECG on arrival is extremely low if your clinical judgement suggests that the diagnosis is unlikely. This can affect your practice – because we often treat patients with possible ACS on the assumption that they have that diagnosis, before further tests can confirm or refute it. Those treatments have risks (especially bleeding). If the diagnosis of ACS is extremely unlikely, the patients really aren’t going to benefit from early treatment overall. So hang fire with your prescribing pens – and rest assured that your judgement is probably right, even though you do still need to rely on those serial troponins (at least for now!)
[DDET EXCLUSIVE ADDITIONAL #FOAMed ANALYSES, ADDED ON 20TH AUGUST 2014]
We recently had an offline request from Anand Swaminathan (@EMSwami) for some extra data from this study. Swami was keen to know how much high sensitivity troponin really adds once the physician’s gestalt and the ECG findings are taken into account. He asked if we can also report the sensitivity and specificity of the gestalt + ECG – i.e. without high sensitivity troponin. So here we go, in what I believe could be a first in the #FOAMed world – new data being presented as #FOAMed outside of a traditional medical journal. Thanks for the request, Swami!
The table below shows the diagnostic accuracy of the combination of gestalt + the ECG (i.e. the presence or absence of acute ischaemia on the initial ECG in the treating physician’s opinion) for the adjudicated diagnosis of AMI (as described in the paper).
|AMI ‘ruled out’ if no ECG ischaemia and ‘probably not’ or ‘definitely not’ ACS||
(87.8 – 98.6)
(25.4 – 34.9)
(18.3 – 27.4)
(91.5 – 99.1)
|AMI ‘ruled out’ if no ECG ischaemia and ‘could be’, ‘probably not’ or ‘definitely not’ ACS||
(72.7 – 90.2)
(56.7 – 66.7)
(25.5 – 38.5)
(90.7 – 96.9)
|AMI ‘ruled out’ if no ECG ischaemia and anything other than ‘definitely ACS’||
(45.3 – 67.8)
(74.6 – 83.0)
(28.4 – 45.9)
(85.7 – 92.6)
The table shows that this approach isn’t sufficient to rule out AMI. The bottom line is that clinical judgement alone isn’t sufficient to rule out AMI (as reported in the main paper). Even if you combine that with ECG findings, you still can’t rule out AMI (or rule it in). It’s only when you start to incorporate troponins that you potentially get the rule out (pending validation, of course).
So, is troponin a friend or a foe in the Emergency Department? We’ll be posting more on that (from SMACC Gold) imminently, but here’s some good evidence to suggest that it may well be a pretty good friend.
Thanks again for a great request, Swami!
25 thoughts on “How accurate is clinical judgement for acute coronary syndromes?”
But is a troponin change 100% specific for an AMI? (see Third Universal Definition of Myocardial Infarction)
Maybe we ought to rephrase the title as “Clinician gestalt is inadequate to anticipate troponin rises”
Thanks a lot for the comment, Derek. You’re absolutely right about the difference between a troponin rise and an AMI and it’s an important point. Actually, in this study the outcome was an adjudicated diagnosis of AMI as per the universal definition. I.e. a troponin rise by itself wasn’t sufficient to make the diagnosis. We also excluded patients with other reasons for hospital admission, which excludes most if not all type 2 AMIs.
Thanks again for reading and commenting,
My concern is that using ‘adjudicated’ clinical diagnosis as the gold standard is somewhat circular. Were the adjudicators blinded to the troponin rise?
The objective definitions of new changes in ECG, echo motion or acute angiographic thrombus would seem to more robust versions of that definition.
Nonetheless, the rapid rule out with hsTrop has great promise. But will it become the next d-dimer leading to increasing over-investigation of improbable ACS?
Thanks again, Derek. The universal definition requires a rise and/or fall of troponin in order to satisfy the diagnosis. Therefore, adjudicators weren’t blinded to troponin levels. It’s a good question about hs-troponin. We’re planning to record some podcasts on it very soon.
Retrospective diagnosis (and clinical decisions) can be coloured by the results of subsequent investigations. But that doesn’t mean the conclusion (AMI) or the proposed intervention (revascularisation of the ‘culprit’ vessel) was ultimately correct.
A really useful paper to help understand how good our gestalt actually is. Is this a relatively high incidence of disease compared to most EDs?
Thanks a lot, Simon! I think the prevalence is fairly typical for the undifferentiated population. The cohorts that you see a lower prevalence in tend to have excluded patients with ECG changes – but we kept them in. Some of the US cohorts also seem to have a lower prevalence. (E.g. Erik Hess’ North American decision rule derivation, where the prevalence was more like 5%). In the UK and Australasia, most papers with undifferentiated patients seem to report a prevalence of 15-20%. Our cohorts from Manchester have consistently found a prevalence of 17-18%, despite a changing definition of AMI over the years, which is interesting!
The AMI rate seems high, no?
It depends what you mean by high, I guess – it’s all relative!
Cohort studies that have recruited similar undifferentiated patients have found similar rates. For example, in Central Manchester, Simon Carley ran a cohort study with similar inclusion criteria in 2004 where the AMI rate was 18.0% (http://www.ncbi.nlm.nih.gov/pubmed/15733759). Similarly, I ran a cohort study in Central Manchester and found a similar rate of AMI – 18.6% (http://www.resuscitationjournal.com/article/S0300-9572(09)00595-4/abstract). Till Keller found a rate of 22.7% in a multicentre cohort study – one of the first landmark papers evaluating sensitive troponin assays (http://www.nejm.org/doi/full/10.1056/NEJMoa0903515#t=articleResults). In Martin Than’s ADAPT trial, the prevalence of AMI was 15.1% (http://content.onlinejacc.org/article.aspx?articleid=1216447). Finally, in the APACE study (one of the large cohorts evaluating high sensitivity troponin – which has yielded lots of publications) the rate was 17% (http://www.nejm.org/doi/full/10.1056/NEJMoa0900428). Those are all similar to the prevalence we found in this study.
I think it probably all depends on the inclusion criteria and the index of suspicion the emergency physicians have for ACS. If you exclude those with normal troponin and ECG on arrival, you’ll get a much lower prevalence. Similarly, in the US where the culture is notoriously litigious it seems that the observed prevalence of AMI is much lower. Erik Hess excluded STEMI from his cohort but those patients account for a minority of AMIs, so it shouldn’t affect the observed prevalence very much. However, Erik found a prevalence of only 6% (http://www.annemergmed.com/article/S0196-0644(11)01343-6/abstract). I wonder if that’s because doctors are much more cautious about suspecting the diagnosis.
Thanks again. I hope that helps!
How do you explain what appears to be a 9.1% MACE rate in the “def not” gestalt group? What’s the NPV of “def not + neg trop + neg EKG”?
Thanks for your comment and I agree that this was really interesting. However, we’re only talking about 1 patient with MACE out of a total of 11 for whom the clinicians checked ‘definitely not’ ACS. With such small numbers, the confidence intervals around that proportion will be quite wide. I’d like to know what would happen to that proportion in a larger sample.
I was actually surprised to find any patients with ‘Definitely not ACS’ given that the inclusion criteria were ‘suspected cardiac chest pain’! However, although the clinicians checked ‘definitely not ACS’ they still proceeded to test for cardiac biomarkers – so I wonder how certain they really were.
The NPV of ‘def not’ + negative troponin + negative ECG is 100%. Essentially we dichotomised gestalt for the analysis – so in the main analysis those who either had ‘probably not’ or ‘definitely not’ ACS recorded were considered to have a ‘negative test’. Of course we then combined that with the troponin and ECG.
Brilliant work (again)
This approach probably is ‘enough’ if we were able to combine it with an ‘acceptable miss rate ‘
Ie that % of ACS identified only by (over) investigating a huge population and for whom we cause more harm than benefit at a population level….
Some relevance of the ‘over diagnosis’ movement…?
Once again – well done
Thanks a lot for the comment – I’m very glad you like it! I totally agree. Martin Than looked at what emergency physicians perceive to be an acceptable risk of major adverse cardiac events in an international survey. It seemed that emergency physicians would find a rate of somewhere between 0.1% and 1% acceptable. (http://www.internationaljournalofcardiology.com/article/S0167-5273(12)01292-2/abstract)
That’s quite arbitrary though. It would be interesting to take a ‘test threshold’ approach as in PE. I bet Jeff Kline would have some interesting thoughts about that.
Rick – fantastic work. Were you able to break down sens/spec in each category of gestalt when ECG also unremarkable? Curious to see how much troponin added after gestalt + ECG.
Thanks a lot for this request. I’ve (finally!) made time to run these additional analyses and I’ve added them to the bottom of the blog post in an additional drop down. I think it makes interesting reading. See what you think!
Thank you so much for this Dr Brody.
I am presenting a talk at my institution and would love to use some of your slides from this recent study and your 2010 study from Resuscitation. I would of course credit you and give a shout out to St Emlyns if you are okay with me to use them.
Dr Cynthia Papendick FACEM
Royal Adelaide Emergency Department
Royal Adelaide Hospital
South Australia, Australia
Sure – no problem, Cynthia. Please feel free!
Rick this is a great read, and thank you for putting this together. I have also put a post together on the value of taking a good history on patients presenting to the ED for chest pain: http://rebelem.com/chest-pain-value-good-history/
I am pretty sure your 2009 publication was used as part of this post.
My Clinical Bottom Line: NO historical elements alone or in combination can reliably help us rule in or rule out ACS or AMI, but instead give us some level of risk stratification in conjunction with ECG and Cardiac Biomarkers.
Fantastic work. We have noticed in Fiji that those of Indian decent so often have atypical symptoms and +ve troponins! Gestalt alone definately doesn’t work well for that group. This is a great article.
Dr. Body, thank you for your excellent data! I have a question for you regarding the following passage from your article:
“To achieve a sensitivity of 100% with a standard troponin
assay in combination with the ECG and clinician gestalt, only
those in whom the clinician felt the diagnosis was either ‘definitely
not’ or ‘probably not’ ACS could be discharged. This strategy
would have enabled 106 (23.1%) patients to be immediately
discharged with no missed AMIs. One (0.9%) of these patients
developed MACE within 30 days.”
My question is: can you tell me what that one MACE in the 30-day follow up was? I would specifically like to know the age ( 40 years?) and outcome (death, AMI, revascularization?) if possible.
Great post and article
I agree that the prevalence of AMI (as opposed to ACS) does seem quite high at 17% or so, but as you say depends on the population being studied – i.e. these patients didn’t necessarily have a normal ECG.
My question is for the patients sent home by the clinicians (or those who would have been sent home) with a negative troponin and ECG along with a low clinical likelihood of ACS, what was the 100% sensitivity quoted referring to – ACS (which includes the nebulous unstable angina diagnosis) or AMI? It looks like it was AMI, with MACE at 30 days being used as the other outcome measure to reflect possible missed ACS on the index visit (0.9% or 1 out of 106)
I couldn’t tell whether these patients were all getting provocative testing after their serial troponins (i.e. treadmill test or stress echo) as is usual practice here in Australia to try and identify the patients who have ACS (but not AMI), and am interested in your thoughts as to whether you think this is necessary in a very low risk cohort (normal ECG, normal serial troponins)
Pingback: The LITFL Review 144 - LITFL
Pingback: Research and Reviews in the Fastlane 042 - LITFL
Pingback: LITFL Review 144 • LITFL Medical Blog • FOAMed Review
Pingback: R&R In The FASTLANE 042 • LITFL • Research and Reviews