So, I am back to rainy Northern England after my exotic travels to Cameroon and Gran Canaria (note: I still managed to join Twitter JC every Friday!) to find a paper on my desk for the weekly JC.
We have a long tradition for the weekly face-to-face JC in Virchester but we also run a Twitter -based JC started recently and which runs alongside it “to bring to the debate to the cyberspace” (as I like to say).
A paper from Belgium this week which is also open access (I just love that country…and its beers!).
So what’s this paper all about?
During my daily practice as an Emergency Physician, I have come to contact with children presenting to our ED with a variety of symptoms. Some of them will look very unwell (easy: treat and save!) but some of them will look well with very vague symptoms. Here is the clinical conundrum: how do you pick up the ones that are seriously ill (probably at the very early stage of a serious illness) and have very deceiving clinical symptoms?
It is clearly impractical (and financially impossible in any part of the world) to admit them all for a period of observation as a safety net. This means that we often have to rely on our intuitive analysis (“gut feeling” if you want to be simplistic) rather than our analytical one. You will no doubt have heard this from a colleague or a worried parent: “I do not know what is wrong exactly but this child is just not right!”. This is what the authors referred to as very nicely “finding a needle in the haystack” (very poetic).
Well, this paper embarked on to elucidate what this “gut feeling” provides in addition to clinical assessment for diagnosing serious infections in children and attempted to identify the associated features of the physician-patient encounter. This observational study set in Flanders recruited 3981 children aged 0-16 years who had presented with an acute illness for a maximum of five days.
You will stop me here and ask me about the arbitrary limitation of five days (I will say: you have to draw a line and it is often arbitrary) or the external validity/generabilisality of this as this kids were seen by primary care colleagues in a developed country (and I will say this could be a first step before validation in other settings).
So what did they look for?
For each child, a list of clinical features were recorded, including the physician’s clinical impression (subjective observation as the illness was serious recorded as present or absent) and their “gut feeling” (intuitive feeling that something was wrong recorded as present, absent or unsure).
You do not have to be a statistician to read a medical paper (and you might not like stats) but you will have picked up on the fact that during the multivariate logistic regression analysis the ‘gut feeling’ was coded as a dichotomy (present/unsure or absent). Again, it does not matter if you understand the intracacies of the method (and I am not sure I do myself!), but in essence the method allows the researcher to analyse a component (in this case gut feeling) as an individual element, adjusting  any raw effect in the data for other factors that might make a difference. The problem as readers is that it is often tricky to get a feel for the data when this sort of analysis has been undertaken and we are quite reliant on the authors.
Clearly it’s important that we are all talking about the same thing so it was resassuring that serious infections (sepsis, pneumonia, meningitis, pyelonephritis, cellulitis, osteomyelitis, bacterial gastroenteritis) were clearly defined by the authors. However, in this study sepsis equated to the finding of  pathogens in blood culture, but we could question whether this is an adequate gold standard. What about the contaminated sample (false positives) or the ones that grow nothing despite an illness (false negative). Furthermore, gastroenteritis equals bacterial pathogens in stool cultures but which pathogens, and might we have included the asymptomatic carriers (incidental finding during a febrile episode)? Again, these were unanswered questions when we read the paper.
The authors then undertook to characterise the diagnostic value of “gut feeling” constructing a 2×2 table. This is a very important point for those of you sitting exams. The CEM Fellowship exam will often ask you to work out sensitivity, specificity, NPV, PPV, etc. from  a similar table.
They then carried out a multivariable logistic regression analysis and used a goodness of fit model to test its strength.
What is this all about? It is much more important to understand these things rather than being able to define them precisely: we are not statisticians, but as medics we need to know enough to test whether a paper is valid. Basically, the first one is a statistical method that estimates the effects of several predictors on an outcome (think Wells score or Ottawa rules) and the second one tests if this works well in practice.
If you are a geek or a nerd, you will argue that the method assumes linearity of data and this does not work well here. I will ask you at this point to go and see someone more intelligent than me!
Moving on swiftly…
The dataset included 380 children, 21 of whom were admitted with a serious infection (12 pneumonia, 6 pyelonephritis, one each for sepsis or meningitis, cellulitis or lymphangitis), the mean age being 5.05 with a range 0.02 – 16.93.
Table 1. shows the overall diagnostic performance of gut feeling. Here is the time to revise your Spin and Snout if you are sitting exams folks!
From this table, the authors concluded that they had the potential to prevent two cases being missed at a cost of 44 false alarms. Interesting numbers, don’t you think so? The balance they give us is 44 admissions for 2 diagnoses to be picked up. We cannot say that it is two lives saved as the potential for intervention is not tested in this paper. Overall then the economic call is whether those admissions are worth it. We thought probably that they were in such a high risk group.
The “gut feeling” was more specific (rule-in value) than the clinical impression and this, irrespective of the seniority of the doctor, the age of the child or his/her diagnosis.
Table 2 shows features associated with gut feeling when the impression overall was of a non-serious illness.
I will let you have a look at the tables on the links and draw your own conclusions but the results were as to be expected really. Basically, you are very sick if you have had a convulsion, lost weight or if mummy was concerned. My grandmother could have told you that (no, she was not medically trained and I do not think she had heard of confidence intervals and the worry when they are as wide as in this table)!
So what is the “take home message”?
Reading this paper brought little to my clinical practice. It however was a great discussion at JC (both face-to-face and on the cyberspace) and I wondered where this would fit within the NICE guidelines we use in the UK (fair enough different population and setting).
It also prompted some brainstorming with colleagues and friends about the utility of intuitive versus analytical thinking (but this is another debate!) in our practice.
As a final note, I like to read papers and I like to critically appraise them. This is not about savage criticism, it is about having a cautiously open mind and trying to extract information form a jargonist manuscript. I said this before and I will repeat it: you do not need to know statistics as a geek to read a medical paper! You do not believe me? Listen to someone who has more experience than I do!
Agree/disagree with anything said? Please post a comment and/or join us at Twitter JC every friday at 13:00GMT (I will be there!) @JC_StE
Janos P Baombe
Before you go please don’t forget to…
- Subscribe to the blog (look top right for the link)
- Subscribe to our PODCAST on iTunes
- Follow us on twitter @stemlyns
- Like us on Facebook
- Find out more about the St.Emlyn’s team
