Many thanks to the twitter FOAMites for alerting us to an online first paper in the NEJM on the use of therapeutic hypothermia post cardiac arrest. If you’ve followed St.Emlyn’s for any time then you will know that @drgdh has a bit of an interest in the subject and having looked at the previous literature we felt that patients should be cooled.
That’s where the evidence stood until yesterday, but one of the great things about emergency medicine/critical care is the realization that half of what you know is probably wrong, you just don’t know which half. This paper in the NEJM challenges the previous doctrine of cooling our post cardiac arrest patients to a target temperature of 32-34C with recent suggestions to try and achieve this as fast as possible – even during cardiac arrest itself.
The Intensive Care Network have declared it as a superb study and have suggested that we need to reassess our cooling strategies. Click on this link and have a read of what they have to say. I agree with most of it…but I think they and others may have taken the evidence one step too far.
The twittersphere has gone a bit nuts about this too….
Simon @EMManchester, what's your take on TTM? Will cooling post OOHCA continue in the UK do you reckon? http://t.co/JcwifOhm67— IntensiveCareNetwork (@I_C_N) November 18, 2013
so are we all going to stop cooling all the non VF/VT cardiac arrests then? #FOAMed http://t.co/7KgqQExWH5— Andy Neill (@AndyNeill) November 18, 2013
@EMManchester @I_C_N is cooling associated with any harm during ICU stay - yes hence does need serious consideration with new evidence— Henry Morriss (@hrmorriss) November 18, 2013
Therapeutic hypothermia,maybe fundamentally wrong. FEVER control alone might be enough. Thx Amal Mattu for heads up. http://t.co/4EA7jTdvrh— EM:RAP's Mel Herbert (@MelHerbert) November 18, 2013
@EMManchester *stamps foot in frustrated toddler fashion* Is nothing sacred?! Will read and digest…..— Gareth Hardy (@DrGDH) November 18, 2013
@EMManchester @I_C_N working in a centre which contributed significantly to TTN. I know locally our temp target will change. UK-wide (1/2)— Segun Olusanya (He/Him) (@iceman_ex) November 18, 2013
Wow; all change... "@traumagasdoc: No point cooling for 24hrs after cardiac arrest. http://t.co/wNywSH9GuP (via @WessexICS)"— Chris Pollitt (@drspikes) November 18, 2013
I could go on…., but it seems that on the basis of this trial the ice in the ICU freezer is being taken out and defrosted.
So what’s changed then? Well there was still some doubts about how hypothermia works. In some of the past RCTs the control group contained patients with fevers…., so was hypothermia acting by cooling the patient, or by preventing hyperthermia?
As uncertainty existed Nielsen et al designed an RCT of two targeted temperature regimes comparing a target of 33C vs 36C after out of hospital cardiac arrest. The article abstract is below, but as ever make sure that you read the full paper if you have access to the NEJM.
[DDET Who was studied?]
The trial included patients over 18 with a low GCS (<8) after out of hospital cardiac arrest. They included patients irrespective of initial rhythm (previous trials have suggested a greater benefit in those with VF arrests). So, it’s a pragmatic and appropriate group to randomize.
Patients were recruited through the admissions process to ICUs with 36 units in Europe and Australia recruiting.
Over a 2 year, 3 month period they recruited 950 patients (impressive).
There appear to be no major differences between patients in each group.[/DDET]
[DDET Tell me more about the intervention]
Patients were treated the same apart from in respect of the target temperature. All were cooled to target for 28 hours before warming began in 0.5C/hour increments. Sedation was turned off at 36 hours with normal prevention of temperatures >37.5C until 72 hours post arrest.
Apart from the target temperature the groups were treated identically. It’s a nice and simple design and a good example of how an RCT can target an intervention analysis.
[DDET What were the outcome measures and what did they find?]
The principle outcome was death – a pretty good end point that few can argue with. At the end of the intervention 50% died in the 33C group and 48% in the 36C group. An absolute risk reduction of 2% (actually it’s 1.4% but there was some rounding up going on) but the hazard ratio of 0.89 has confidence intervals that extend from 0.89 to 1.28 (so this is a statistically non-significant result).
At 180 days there remained a 2% difference but again there was no statistically significant difference between the groups, nor was there any detectable difference between patients in terms of disability (using modified Rankin scale).
So really little to choose between the interventions, or at least not that we can detect in this study. [/DDET]
[DDET So this is definitive then? Is there really no difference?]
Ah, steady on there chaps(esses). There is a world of difference between not being able to find a difference between two thingies and two thingies being the same. Perhaps we need to look at the sample size again.
My first impression when I looked at some of the tweets coming through from twitter were that this was a fairly small study to demonstrate equivalence, and that’s perhaps because it was not set up to detect equivalence at all.
In this study the apriori sample states that they were looking to find a 20% difference in the hazard ratio between the two therapies. This is a bit tricky to get your head round, but thankfully they have given us an example in the power calculation section in that they would have required 850 patients to be 80% certain of detecting a difference of 11% between patients…..just think that through again…
That’s a number needed to treat of 9.
So they were aiming to find an NNT of 9 by comparing 33C with 36C
That’s a bit ambitious….
…so perhaps not surprising that they did not find it then…
Let’s think of it another way. Say we were going forward from this study and we wanted to look for a 2% difference in survival (i.e. what they found) between these two cooling therapies. That’s a clinically important difference folks a real time Number Needed to Treat (NNT) of 50. That 2% ARR/NNT of 50 (it’s the same thing) is as good as aspirin in MI or Tranexamic acid in major trauma (really effective and fab treatments you will agree). Maybe we would think about our sample size calculations as follows..
The difference we hope to find is 2% (that’s what they found)
The expect our event rates (death) to be 48% and 50% respectively (that’s what they found)
So that’s an NNT of 50 (similar to Aspirin in MI or TXA in major trauma)
Let’s go for a power of 80%
Let’s go for an alpha of 5%
Hang on a sec….just whizzing into the computer….and…the answer is…
9806 patients (for each sample – 19,612 in total)
(Ed- gosh that’s a lot of patients!!!)
It is but not out of keeping with the number needed in CRASH 2 to show a similar difference.
Hypothetical Analysis 1: We can – because we are statistical geeks also ‘back calculate’ the power of this trial to detect a difference of 2% (48%-50%) based on a sample size of 950 patients. I’ll spare you the details, but it’s about 14%.
Hypothetical Analysis 2: We might say that 2% is too small. What is the power of this trial to detect a difference between 47% and 52% (a NNT of 20) – well again it’s sadly many more than 950. In fact it’s about a 60% chance of being able to demonstrate a 5% difference.
Hypothetical Analysis 3: Let’s work out the confidence intervals around that difference they found. It’s quite tough to do with hazard ratios, but we can consider the absolute risk reduction and work out the confidence intervals for that. OK. The actual ARR is 1.4% (they round there figures up in the paper) and the 95% confidence intervals extend from a 5.4% ARR in favour of 33C all the way up to an 8% ARR in favour of 36C. Is your head hurting – if so lets convert that into NNTs. Basically the confidence intervals for this result extend from an NNT of 19 in favour of 33C all the way up to an NNT of 12.5 in favour of 36C.
Are you feeling convinced that there is a difference here? Are you convinced that the results really demonstrate that they are the same? I’m not.
The bottom line here is that we cannot equate a negative finding in this study to an assumption that these two therapies are equivalent. They have failed to find a difference, but they have not demonstrated that there is no difference. The real difference between these therapies could really be quite large AND clinically significant but this study is just too small to detect it.
If you want to have a go at playing around with stats calculations and sample size I recommend statspages, as there’s loads of good stuff on there.
The sample size calculators are here.
[DDET My head hurts with all this stats mumbo-jumbo]
Yeah, sorry about that. The thing is that we need to be really careful when designing studies about what we define as clinically important differences. In this study the power calculation is based on a 10-11% difference in mortality which to be honest is huge. In the Cochrane review of cooling the difference found was roughly 9% (Cochrane study pooled data Analysis 2.1), so this study was attempting to find a similar or greater difference than that comparing placebo against cooling.
We can of course debate what a clinically important difference is, but in terms of death a 2% absolute difference in survival really means a lot to someone like me. I think 10-11% is just a bit too ambitious, but that’s only my opinion and I’m sure others will think differently.
If there are any true statisticians out there then they will be foaming at the mouth at my mixing of non-inferiority principles of trial design with trials designed to detect difference. It is quite true that they require different approaches and in general non-inferiority trials require more patients than more traditional models. I apologise to the statisticians, but I think there is enough here to tell us that this trial is too small to detect what I believe is a potentially clinically important difference.
Even the NEJM Editorial gets in on the act and declares that there is no difference….but have they demonstrated that in this trial? Answer – no.
The conclusion? Well this is a really interesting study but it can only tell us so much. The authors have failed to detect a really big difference in clinical outcomes between these two therapies, that is simply not the same as presuming that they are the same.[/DDET]
[DDET So 33C or 36C what are you saying – what should I do?]
It’s not for me to tell you what to do really. I like cooling patients, I think it works and there is evidence to support it. As for a target temperature then this trial really gets us thinking. It’s not definitive but there is enough to get us interested in questioning whether a prevention of fever is as important as cooling to hypothermic levels. There may well be no difference, there may be harm, there may even be benefit to a lower cooling temperature I just don’t know and I’m not sure this trial helps us that much. As a clinician, and if I was in the ICU I would be faced with the dilemma of uncertain evidence vs. the known harms/difficulties of cooling. Overall there seems to be little with which to base a decision.
Tough call chaps(esses). Let me know what you think.
Sorry folks, but this is a negative study of difference NOT a positive study of similarity.
Basically just chill out…………
This paper has been a fantastic demonstration of the power of #FOAMed to deliver real critical appraisal, understanding and depth to the results of a really interesting and challenging paper. If you’ve made it this far then please read the comments below and also head over to LITFL for further insights and links to other amazing resources. Viva la FOAMed.
What I’ve really enjoyed is the cross speciality interest and understanding on this topic and I’m really pleased to say that the insights and contributions of colleagues have pushed me off my metaphorical fence onto the side of……, well read the comments below and find out 🙂
40 thoughts on “JC: What’s the target temperature for OOHCA cooling. St.Emlyn’s”
This may turn out to be similar to tPA. Works for some, harmful for others, no change for yet others. Here’s to the future of finding out! I personally believe in the benefits, in the right patient. Thanks for the post.-Matt @mastinmd
Mmmmmmm. Stats. Thanks Simon, very useful distillation of the data.
Whilst prognosticating, in the interim I have found an alternative use for all the ice I had previously insisted on being available in resus….it is a 35 degree day outside….more gin in my tonic? With ice? Why thank you….
Maybe it is all down to protection against hyperthermia then. Lots of evidence to support the fact that high temperature bad for you
Pingback: Five Minutes with Jon Rittenberger on the TTM Trial
Superiority vs non-inferiority vs equivalence will always try to catch you out. Great post. This is very much a superiority study. They have failed to demonstrate that cooling at 33 is superior to cooling at 36. More than a few possible reasons for this result. Which may or May not be true.
What are your thoughts about timing of initiation for TH. You seem to imply the sooner the better. Is that what the evidence says?
Keep on blogging on
Not following your argument here Dan, and (sorry Simon), not totally sold on your statistics. I read this as a non-inferiority trial of new treatment (36C) vs “standard” treatment (33C). Now we can argue about whether 10% is “non-inferior”, but given that Cochrane found a pooled benefit of 9% for cooling, I think it’s not bad.
All their hazard ratio confidence intervals for primary and secondary outcomes are solidly crossing 1, and the Kaplan-Meier curve is as good as you could expect.
So, I’m convinced that we shouldn’t let post-CA patients get hot, but as Henry said, cooling is associated with some harms and I’m not sure we have any justification for “the colder the better”.
Thank you Simon for your astute detailed assessment of the recent NEJM article. If I understand this paper and your points correctly – there is not a big difference between cooling unresponsive victims of cardiac arrest to therapeutic hypothermia levels (33 degrees) vs treating only to prevention of fever levels (36 degrees). As you correctly conclude – this was a negative study of difference between the 2 study protocols. That said (If I understand results correctly) – this was also not a positive study on the benefits of therapeutic hypothermia. Sounds like a tincture of time is needed to tell for sure what “the answer” will be.
You are correct and time will indeed help us…. that or a properly powered RCT 😉
Pingback: Therapeutic Hypothermia after Cardiac Arrest: Just Not Cool Anymore? | AmboFOAM
Great post Simon, and really interesting discussion that’s opened up.
I take your points about the power of the study.
However, given that this is better than any other evidence out there, surely the burden of proof now rests with anyone who wants to cool, and maintaining 36 carefully should now be the standard, and if people want to cool, it should be done in the context of a clinical trial?
This has been a great conversation across twitter, blogs and podcasts. Really fantastic stuff and thanks so much for getting us all started.
I’m firmly in clinical equipoise here. I think you can justify a number of strategies because the evidence in NEJM does not help that much.
ICN started with the title Cooling post OOHCA: The world has just changed . I’m not so sure that we can be that certain of the findings here but the title suggests everything we were doing was wrong. I also think that the way the trial is reported in NEJM is quite difficult to get into. Hazard ratios are a pain in the backside to get your head round and so long as journals don’t report in natural frequencies we will be left to work out data that means something to us at the bedside ourselves.
So what can we agree on?
I think we can get consensus that fever is bad. There’s lots of work to support that as a theory and in practice. So no fever.
As for 33 vs 36 then I just don’t know. The reason I put the blog post up in the way that I did was to just caution people about assuming that this is evidence that the outcomes are the same. It does not tell us that, it just tells us that they are not really, really, really, massively different….., but I kind of expected that already. Watching twitter and blogs over the last 24 hours has been fascinating. It started with the world has changed let’s stop cooling patients, has moved onto questions about the trial, and now is moving into a phase of colleagues talking about subgroups for 33C or 36C based on coma scores??? That’s a further extrapolation of the evidence and we just need to be cautious about that.
It’s worth listening to Rittenburg on Scott’s podcast. Is it me or does he sound a little more cautious about the findings on the podcast than in his editorial?
Finally, whilst I am firmly stuck to the top of the fence wondering which way to jump I will take the advice of my ICU chums on this, after all it’s them who are going to manage the complications. I am merely the humble emergency physician with a calculator 😉
Pingback: Should I Cool the Cardiac Arrest Patient? - KI Doc
Pingback: SGEM#54: Baby It’s Cold Outside (Pre-Hospital Therapeutic Hypothermia in Out of Hospital Cardiac Arrest) | The Skeptics Guide to Emergency Medicine
Pingback: All in a lather over TTM - LITFL
Pingback: The LITFL Review 116 - LITFL
OK Prof, after a couple of reads and a bit of experimenting with the sample size calculator on medcalc, I think I’m with you on the maths….
On first read of your analysis, I was pleased. At last, some clever facts to justify my affect heuristic! (I like TH, I have a bit of an interest in it….. I want it to work!). There might be a benefit…. but if there is, it’s going to be small.
Lets not forget that the original RCT’s from 2002 came out with NNT’s of 5 and 6, and these are the numbers that have been enthusiastically promoted to justify the widespread adoption of this therapy. TH also has great potential for anecdote (“…. and he walked out of the ICU 4 days later!”).
This reality seems to be that the benefits of taking patients down to 33C just aren’t that significant. The authors have not proved there is no difference, but I suspect they have proved the difference is enough justify the risk and resource utilisation required to cool all of these patients.
Interesting that the very first comment on here mentioned stroke thrombolysis, there are some good parallels there. We now too have a big data set that suggests one of our favourite therapies (which had also undergone significant indication creep; is cooling inhospital or non VT/VF arrests any more rational than thrombolysing a stroke in a 90yr old?) is not as effective as we have thought. Will we fall into the same trap as the tPA fans, and keep pursuing a treatment the evidence suggests doesn’t work? Or will practice change? Will be fascinating to watch.
P.S – there is another remarkable facet to this story, which is the speed this study has been disseminated, dissected and reviewed on the #FOAMed blogo-twitto-sphere. It’s been remarkable to watch, and this will now be my poster child for convincing colleagues that there is something to this #FOAMed stuff after all. The question now has to be: If Nielson and colleagues had self published this data on a blog, would have we lost out by skipping the ‘formal’ peer review process?
Yep, I think you are correct. As I said in the post we are still uncertain as to whether cooling works or not, but it is fair to say that the very impressive NNTs of the original trials are unlikely to be reproduced (where absolute risk reductions were >10% if I remember rightly).
The issue with the trial is that it remains non-definitive. It has insufficient power to tell us whether there is harm or benefit outside of fairly significant clinically important ranges. That is a shame but it does allow us to raise the question and to reassess where we are in terms of making a decision on the evidence that is available – even if it is imperfect.
Having said that, if you take a wider view and combine this data with what we know from the 2002 trials then I strongly suspect (I’ve not crunched the numbers) that the benefit seen in the Cochrane meta-analysis will disappear. I really hope to see that data soon – will someone redo the Meta-analysis please 🙂
If that is the case then we are left in clinical equipoise. We are left in the position of not knowing whether cooling works or not.
Let’s presume that the meta-analysis comes out as showing equipoise (it will honestly) what can we recommend then? Well to some extent it’s a bit of a grey zone and if you are looking for analogies then it’s not dissimilar to the position pre CRASH 1 (steroids in neuro trauma) where there were a variety of underpowered studies that needed to be clarified by one big one. That may not happen, and if it does it probably won’t be anytime soon. However, there have been harm questions raised by some, and mostly the ICU docs which is fair enough as they are the ones managing them…
In the absence of great evidence either way and with the potential for minor, moderate and significant harm then I think we would have to agree with John Myburgh and say that cooling should be used in the context of well constructed clinical trials.
In other words, as we stand today – the evidence for cooling is not convincing and would be difficult to justify. However, the door of uncertainty remains ever so slightly open and future trials may clarify the position with more certainty than we have today.
I’d be happy with that.
This study was powered for an 11% mortality difference – the study didn’t reach stat significance for the findng — thus we can only say that there is not an 11% mortality difference between 33C and 36C, corrent?
Moreover, the study did not power for neurologic outcomes which IMHO is what matters. Why do people keep powering and having primary outcomes for mortality when they should be powering these studies for improved neurologic outcomes? I think we are missing the boat…
Yes you are correct in terms of their power calculation. You can also work out the confidence intervals of their findings as we did in the blog post. Whichever way it is still unable to tell us whether there is a clinically important difference.
Your point about neuro outcomes has been made by others e.g. Seth Trueger on twitter. It’s a fair point.
Dear Simon (and others).
I get a bit puzzled when I look at negative studies such as this, and drawing conclusions from them.
I take your point about the pre performed power calculation. Before the trial had been conducted, in order to determine sample size, it was decided that there was a 90% power to determine a 11% mortality benefit at the 0.05 significance level. i.e if there was truly a >11% mortality benefit, the trial would be negative at the 0.05 level only 10% of the time.
The difficulty I have is how does one interpret the findings after the trial has been done.
The results of this trial, despite having 950 patients in it, show that there was very little difference in mortality between the intervention (33deg) and control (36deg) arms. In fact there was a trend towards slightly better mortality in the control group. Its hard to imagine, that even with a significantly larger number of patients, the results would have suddenly started to turn the other direction to show benefit in the intervention group. I agree this is possible, but seems very unlikely, and I’m not sure how to quantify this.
I am trying to compare this with something like the Interact II trial, where in ~2800 patients there was a 3.6% mortality benefit in the intervention group, with a p value of 0.06. Here I can see the argument that it is underpowered, and that with a few hundred more patients, the benefit may become statistically significant (at the arbitrary 0.05 level)
The difficulty for me is – both the above trials will be called “negative” trials, which don’t prove that the intervention works nor do they definitively prove that they don’t work. However to me intuitively it seems that whilst the intervention in the Interact II trial possibly has a small benefit, the intervention in the TTM trial seems much less likely to be beneficial, and I am unable to quantify that uncertainty.
any comments would be much appreciated!
Thanks Simon for the insightful analysis – I take with all of your points however agree with Oli that the burden of proof has shifted to those who want to cool. While the trial might be statistically underpowered for equivalence it is still a large trial of relatively uncommon resource intensive patients with no signal of benefit in any of the results. The trials you mentioned for comparison (CRASH2 and aspirin in MI used much cheaper and easier interventions in more readily available patients).
Here are my thoughts as a paediatric and adult intensivist:
– We still need a temperature management program, equipment and approach to avoid fever.
– Targeting mild hypothermia over low normal temperature is very safe (the only significant risk is a slight increase in hypokalemia which is easy to detect and treat) but probably pointless. The main downside is the increased use of bed days (~2 per cardiac arrest patient) as we are mostly unable to prognosticate until 72 hours after the patient is warmed, as opposed to 72 hours post arrest if we don’t cool.
– There may still be a patient important difference in outcome (survival, quality of survival) from some kind of temperature target or for some kind of patients but this is pure speculation. We may see this in TTM long term follow up but this will be post hoc analysis and therefore only hypothesis generating. Reliable results in this area are likely years away.
– The situation in paediatrics is now really confusing. TTM suggests strongly that in adults merely avoiding fever is enough. A moderate sized trial in neonatal HIE (which is primarily hypoxic, Shankaran, NEJM) showed a similar ARR to the HACA /Bernard trials which was maintained at 6 year follow up but again their controls ran in the mid to high 37s for the intervention period. Infants and children have multiple arrest locations and modalities depending on the age of the patient and centre.
– So overall: I think it’s hard to justify the resources to cool adults (though we must actively maintain low normothermia). As cooling is safe and may still have benefit in the neonatal / paediatric group we should continue to do it, though I expect that later research will show no benefit.
Hi Greg, I think I share that view and that of John Myburgh. Although we have not proven that cooling does not work (which was the initial hyperbole) there is insufficient evidence to prove that it does and therefore it should be restricted to clinical trials.
Pingback: Eminence and Evidence Based Medicine, learning, practice evolution and Targeted Temperature Management - EM Tutorials : EM Tutorials
Pingback: TTM: The story continues to unfold… | intensivecarenetwork.com
Pingback: SGEM#82: Melt with You (Targeted Temperature Management) | The Skeptics Guide to Emergency Medicine
HI Prof carley,
Just a very humble thank you and appreciation for / of your comments on and summary of the TTM discussion. It is very hard to see the wood for the trees I find in these things and being quite new to the whole SMACC/ podcast/ blogging part of things I found your comments refereshingly easy to digest.
I am an ST5 doing a journal club talk on TTM and have found this extremely helpful- as I did your help as a medical student at hope hospital with my SSM- a few years back now!! I also met you at QMC Nottuigham again – when you came in a GMC capacity- things are still pretty manic but I think improving….
Dr Lizzie Robinson
Thanks so much for the comment, we really hope you find it helpful and that the blog can help in knowledge translation and dissemination of new practice.
I remember the visit to Nottingham well. A really busy UK department and level 1 trauma centre with many challenges and a great group of trainees and consultants.
Hope all well with you and please say hello if you are at the college conference next week.
Hi again Simon,
Thanks for your reply- and promptness of so doing!
I am enjoying the St Emlyn’s site,blog and FOAM the more I get into it; it is great to be inspired about subjects and issues that you already find interesting- but sometimes struggle with the hard core literature on!!
I will look again at getting to the conference but am currently not planning to do so, so I hope all the hard work that has been put in from your end and by your colleagues, gets it’s due recognition!
Pingback: JC: PARAMEDIC trial m-CPR at St.Emlyn's - St Emlyns
Pingback: We need to talk about TTM…. Again - LITFL
Pingback: JC: Getting Chilly Quickly 3. Hypothermia at St.Emlyn's - St.Emlyn's
Pingback: JC: Is Early Goal Directed Therapy dead? St.Emlyn's - St.Emlyn's
Pingback: JC: What has an awesome NNT of 5.8 for severe sepsis? St.Emlyn's - St.Emlyn's
Pingback: JC: Does EPO improve OOHCA outcomes? St.Emlyn's - St.Emlyn's
Pingback: Does Targeted Temperature Management Actually Work? - R.E.B.E.L. EM - Emergency Medicine Blog
Pingback: Hypothermia in status epilepticus. St.Emlyn's - St.Emlyn's
Pingback: JC: Do you see the light? Serum neurofilament light chain for prognostication following OOHCA • St Emlyn's
Pingback: Does Targeted Temperature Management Actually Work? - REBEL EM - Emergency Medicine Blog
Pingback: We need to talk about TTM…. Again • LITFL
Pingback: All in a lather over TTM • LITFL