Sooooo….this paper turned up at JC last week (thanks to Nicola P) and whilst I’m not sure that it meets all three of our criteria for a top JC paper it is relevant as a week barely seems to go by without someone questioning the dose/route/brand/colour/size/ethnicity of medicine for croup.
Rule of thumb chaps and chapesses ‘The greater the dogma, the greater the ignorance’. Someone cleverer than I said that, but I’m happy to plagiarise ‘cos it’s true.
Anyway, Croup arrives as a question once again in the journal Emergency Medicine Australia, but this time the question relates to speed of onset in mild to moderate croup.
STOP! If you are in exam mode at this point you should read the paper. See what you think about it and give it a mark out of 10.
We’ve talked about this paper and it’s a tricky one. The first question is why has this paper been done (which we cannot answer, but can surmise privately). The use of steroids in the management of croup is very well established and is something we led on here in Virchester many years ago. It was even one of the very first BETs back in 2004 (amazing to think that we are still talking about this 8 years later).
I’ve also seen the Cochrane review and even examined some CTRs for FCEM on the subject. So, it pretty much seems to me that the question of whether we give steroids for croup is well made. The research that remains is, I suppose, about refining and polishing what is surely a well established fact.
STEROIDS WORK IN CROUP Click the link and read the Cochrane review.
So, what about the paper this week? Is there anything we can draw from it and learn? Well, the authors have done an RCT (good) on mild/moderate croup patients. Interesting this as for the mild ones would you give steroids or just let nature take its course? (Ed – depends on how mild as croup score 1-3 is mild) I’m not sure so there maybe an element of over-treatment in comparison to other practices. Whatever, the authors tell us that there is an effect of giving steroids that they can define and detect at 30 minutes following administration of steroid and that this counteracts the information given through Cochrane about a delayed effect taking up to 6 hours.
I have major concerns with this paper and I just don’t see how this is going to make a significant difference to our practice in PEM. I don’t think a paper like this would appear in an exam, but if it did I would be pulling holes in it along the following lines.
1. What is the clinically important question here? It seems that we are looking to see the speed of onset of steroid meds in mild/moderate croup. The clinical importance of this is perhaps unclear except in logistical (admission) terms. What defines a significant difference in this low acuity group? Mild croup is not admitted anyway so what is the issue we are addressing?
2. Sample size. OK. An interest of mine, and if you share that interest (you sad person) then hop over to the podcast to hear more about how to understand and interpret sample size calculations. In this paper they appear to be using tests for continuous data for data which is unlikely to be so. Honestly, it seems as though these are the wrong tests for this data, but there is insufficient information in the paper for us to tell. Where is the clue? Well, the Wesley croup score is a categorical score (at best ordinal). It’s not continuous and is unlikely to be normally distributed, so a t-test is rarely going to be the right test. So hmm, not enough information to know but questions are there to be asked. If you want to know more about stats for Critical Appraisal then click here and here. Apart from anything else, a study of just 70 patients would have to show a massive effect if it is be valid and I don’t see that here. Similarly the graph shows average scores only, and I’m not sure that I’m just interested in the change in average score amongst 35 patients. I want to see the distribution as well. This is a common problem in papers as the mean score reporting removes the depth and character of the data.
3. Right, so we are unsure of the validity of the question and also of the sample size what else? Well, do applaud the authors for defining the numbers of patients that they ‘could’ have recruited and the difference between that number (828) and the number recruited (70) is huge. This suggests a degree of patient selection which may well affect the results. Now, I don’t want to put a massive downer on this as it is an inevitable problem with EM research, but this ratio really asks questions as to whether this is a representative sample, or whether the results will be heavily skewed because it is a sample of convenience.
So, it sounds as though we were pretty down on this paper from a methodological point of view. We gave it a 3/10 to be honest which is clearly not high, but just wait is there ANYTHING we can take away from this piece of work at all. Well, it’s tricky to be honest. It’s likely (but I’m finding it difficult to tell) that oral dex starts working fairly quickly, but that was never a clinical dilemma for me before I read this paper so I’m not going to change practice. However, it’s a useful to use this as a vehicle to discuss Croup (again), to review the relevant BETs and to talk about how to spot flaws in papers.
PS. If you are still in exam mode try answering the following questions…
[EXPAND 1. What is meant by the term ‘double-blinded’ and why is it important in a trial like this?]Double blinding means that neither the participants, nor the staff treating them knew what medication they were receiving. You can also have triple blinding when the statisticians doing the analysis are also unaware of which treatment is which (though you rarely see this). In a trial like this blinding is really important as if staff were aware of which treatment had been given they may have changed their croup assessment scores based on prior beliefs. Similarly they may have been more likely to break the protocol for patients with a poor response in the placebo group.[/EXPAND]
[EXPAND 2. Four patients in the placebo group worsened during the initial phase of the trial and were then given steroids. They were analysed in the placebo group despite getting steroids. What is this type of analysis called and is it the right approach?]This is known as an Intention To Treat (ITT) approach and it absolutely the best way to analyse data. Why you might ask if some patients did not receive the treatment? Well, if you think from a clinicians point of view – and you always should then you would want to make a decision at the point when you need to make the decision!!! Eh, by that I mean that the point of randomisation is the clinically important decision time and that is therefore where you should analyse the results from.[/EXPAND]