Podcast – Number Needed to Treat (NNT)

Hello and welcome again to the St. Emlyn’s podcast. I’m Iain Beardsell and I’m Simon Carley. Today we’re going to take some of those discussions further that we’ve been having about diagnosis and think a little bit about treatment.

Today we’re going to take it on from where we were thinking about diagnostics in the previous podcast and think about therapeutics. One of the things I think came out of the diagnostic podcast was that we like talking about natural frequencies. It’s much easier for us to understand a test which works in one in 500 patients or one in 50, rather than talk about rather abstract terms such as sensitivity specificity. So this time we’re going to take that a little bit further and think about therapeutics.

Can we use the same principles and can we use something called the number of needs to treat as a way of really understanding what we’re talking about? So can I give you an example? Of course, of course, okay. So I’m going to take an example. I’m going to do some trials on my cardling function patients. And it’s a great trial and I’m going to do a new drug and it’s going to be really good. So give you some background data and I want to see how you feel about this.

So we’ve got our AMI patients and after a month about 10% of them are died, which is very sad. But I’ve got this new treatment. It’s awesome. And it’s going to improve the mortality by 50%. How good is that?

Wow, that’s amazing. So we’re going to say 50% more lives. Well, do we? Run me through the numbers again. So we’ve got 10% who are dying. And we’re going to improve that number by 50%. Yeah, it’s 50% better. Okay, so do you mean that now only 5% of people are going to die?

Well, it’s difficult, isn’t it? It’s not a very clear thing to say. And it doesn’t really particularly help when people come out with that kind of data because what does 50% mean? Does it mean that you reduce it by 50%. Well, if there’s any 10% for starters, that would end up with a minus 40% mortality, which would be a bit bizarre. Or does it mean it halves it? But when people are talking about these things, they don’t talk very clearly about it.

So let’s just explore that. There’s some terms that we can think about. When we actually do the data in our particular trial is, in a hypothetical trial, the mortality was 10% in our placebo arm, if you like. In an artherapeutic arm, it goes down to 5%. So that is a 50% relative risk reduction. So the risk 10/5 is about 50%. And that’s usually what people are talking about. So that’s pretty good, yeah? So that’s a relative risk reduction of 50%. Yeah. Sounds pretty good. But it sounds awesome. But what’s the real reduction? What’s the actual reduction in mortality here?

I guess it’s a 5% reduction in mortality. Absolutely. So it goes down from 10 to 5. But again, if you go into a speech to a patient or a speech to a colleague and say there’s a 5% absolute risk reduction, does that really instantaneously tell you exactly what’s going on?

Well, 5% risk reduction doesn’t sound like much now. It doesn’t sound like you’re really improving me at all. No, and that’s why drug reps and big farmer often use things like relative risk reduction, because it over emphasizes or exaggerates the effect. So the same data, you could express it as either 50% better or 5% better. Which one do you think the marketing department is going to go for?

Well, I’d very much like to be 50% better, please. Yes, so I’ll give you that option. But it’s the same thing. I don’t think either of those are particularly helpful ways of describing the benefits of treatment, because I’ll give you another example, okay?

I’m going to take those AMI patients again. We’ll take a different cohort. And in this group of patients, we’re going to have a 20% mortality a month. And my new therapy is going to improve it by 10%, it’s going to go from 20% to 10%. What’s the relative risk reduction there?

Well, using the theory we were just talking about, it’s again a 50% reduction, isn’t it? Absolutely, the same. So your relative risk reduction, you could have a completely different therapy and benefit, but the relative risk reduction stays the same. So that can’t be very helpful to us, can it? Well, it doesn’t seem to describe it very well, really. It’s not giving me a true picture of what’s going on. No, and the absolute risk reduction would be 10% in that case. Yeah, so you’ve got, what’s going on here? We’ve got the same relative risk reduction, we’ve got different absolute risk reductions. And that’s not helpful to us at all.

So there’s something that we can do differently. We can work out what’s called a number needed to treat. And that’s the difference, the absolute risk reduction. So either 10 minus 5 in this case or 20 minus 10 and divide that into 100 and that will give you the number needed to treat. So for every x number of patients you treat, you get one benefit. So an absolute risk reduction of 5, 5 into 100 is 20. So for every 20 patients you treat, you get one benefit in this case, one more survivor. But for our second trial where we had a 10% absolute risk reduction, 10 into 100 is 10, that’s a benefit for every 10 patients that you treat. So you can then clearly see if you use something like numbers needed to treat, a natural frequency, that you can demonstrate very clearly that the second trial had twice the benefit than the first.

So we’ve now got this idea that we can explain to patients about the chance of the treatment having a benefit for them. I’m always struck by this that a number needed to treat of 10 is regarded as pretty good, isn’t it? It’s awesome. And actually that means that nine patients we treat see no benefit from the treatment at all. Is that right?

It’s a mixed bag, isn’t it, when you look at the outcome of any trial. So for most trials some people will benefit from treatment, some people will have no effects. And some people actually might do harm, but when you’re looking at the overall figure at the end of the month in this case, it’s an amalgamation of all those potential outcomes. So from this point of view, some people may have actually been harmed by the therapy, so they’ll still be in the mortality group. Some people will receive the benefits of the therapy, they’ll also be in the saved group. So it’s a mixed picture, but that’s okay. I like nice simple summaries at the end, which I can then use to communicate to myself into patients. But a nominated treat of 10 would be seen as a good thing.

It’s far more effective than many of the things that we do in practice. So I give you some examples, and there’s a fantastic site, isn’t there? The nnt.com, which anybody who’s listening to this should really go and have a look at. And they’ve got loads of information on nnts. But give you some simple examples. What about Tranics Amic acid in trauma? Do you know what the number of these treat is for that?

So I can’t remember, crash too, all very good, all very positive. We’re really pushing Tranics Amic acid, might save lives, not much harm. It must be quite high. We’re pushing it really hard. Yeah, I think it was around about 50. I can’t remember, top of head now, but it was around about that level. So for every 50 patients you treat, you gain one survivor. That’s very effective. It’s a really, really good therapy. And that’s similar actually to aspirin in in the treatment of my carlin function. Okay, so also great drug.

And do you think they’re mainly regarded as great drugs because their, their harm is low? Do you think we have to start balancing up? It’s a way we can do a number needed to harm as well. Absolutely. And we saw that in the trials of start solutions for sepsis in the last couple of years. You may have seen that on a lot of the foam sites. So things like the perna trials and they looked at numbers and needed harm. And we came out with figures in the sort of a region of about 10, 12, 14. So for every 10 to 16 patients that you saw, you treated with start solutions, you actually causing one additional death. So number needs to harm is the same, it’s the same calculation, but just done in a different direction.

So when we just, so we had a bit of a revision of that, how do you do the calculation for number needed to harm? Number needs to harm is essentially you take the difference between the two values. So in our case, even in that trial before, we could say the number needed to treat with a therapy was one in, was one in 20. But we could reverse it and say the number to harm with placebo or the alternative medication is also 20. It’s just which way you’re looking at it. But the number needed to harm and the number needed to treat aren’t always the same, are they? You have to look at the outcomes that you’re getting from your trial. You can calculate a number needed to treat or a number used to harm for any outcome in a trial.

So you can look at the overall outcome, live die. So mortality trials tend to work in that way. But you could also look in trials such as low-many accurate heparin for patients undergoing lower limit mobilization. And you can look at the numbers needed to treat to prevent DBT. But you could also look at the numbers needed to harm for major GI hemorrhage. So yes, you can do that. But at the end of the day, I think it’s always important that we have one final overall benefit or harm figure to work with. Because that’s what patients want to know.

Do you not tell them both numbers? You don’t tell them number needed to treat and number needed to harm? In some occasions, yeah. We used to do that a lot with thrombolysis, didn’t we? For the stroke? Yeah, well, we don’t do that in our shop. But in the old days when we were doing thrombolysis for my carlin infarction, we would give them numbers needed to treat and the numbers needed to harm you. I think because I do with, when we had the stroke thrombolysis, we do it in our hospital. I have very little to do with it, which I’m sort of thankful for, I think, because I think it’s a very difficult thing to get informed consent about. We used, they tend to quote the NINDS numbers. Now in my head, the NINDS numbers, which I know this discussion about how accurate they are and what that means. But a number needed to treat that NINDS went for was about one in eight, something about one. And a number needed to harm was about one in 16.

So for every eight patients you treated, one of them had an improved outcome. But for every 16 patients you treated, one of them had a harmful outcome. And usually this was something pretty bad like intracurbo hemorrhage and death. Can I then say that for a group of 16 patients, two of them will have benefits and one of them will see harm and 13 will have no difference if we take the statistical analysis. If I have a patient in front of me, I can say 16 patients present exactly like you, two of them might have an improved outcome, one of them is going to drive intracurbo hemorrhage or am I taking the statistics too far? You look pained as I look at your face here. It depends what you’re talking about, whether those are balanced calculations.

Again, you can calculate a number needed to treat for any outcome, but for a trial of therapeutics you should be looking for what’s a number needed to treat or number needed to harm for their principal primary endpoint of the trial. So for something like stroke thrombolysis, I would be looking at mortality, all cause mortality. I think that’s really good number needed to treat. And that encompasses both harm from therapy and harm from the trial and also the patients who have no benefit or no harm. It’s a global assessment of the benefits of that trial against their primary endpoint. Those are the figures which I think are most powerful in clinical trials.

And is that just described as the number needed to treat? So am I mixing apples and oranges there? If I do, number needed to treat number needs to harm? You’re just saying that you can calculate those figures for any of the potential endpoints in any trial. What I’m saying is you should look for and focus on the primary endpoints.

Great. So we now have a way in which we can describe to patients the benefits of a certain treatment. Yeah, we can take it one step further as well actually, because number needs to treat when you put them into natural frequencies are quite useful because then you can talk about one in a hundred people or one in 50 people. So you can talk about things like one person in your house, one person on your road, one person in your children’s school, one person at Manchester United on a big day, one person at Wembley, one person who attended the World Cup in Brazil. You can use analogies which are very helpful for understanding.

And I think some of the numbers are actually quite surprising when you look at them and the nnt.com is brilliant for that. As you flick through the different things and I trust the guy who run that website to have done that data absolutely perfectly, they’re really surprising that the number of interventions we do that actually have relatively little impact. Absolutely. And that’s not the way that clinicians either want to believe or want to practice. If I say to them, you know, what’s the number needed to treat for thrombolysis, for instance, in inferior myocardial infarction, many clinicians I speak to would say, well, if you don’t have it, you’re going to die. So they’ve got a feeling that the nnt is almost one. In reality, the number needed to treat for those patients is probably over a hundred. And that’s a huge surprise for such a radical therapy for patients which has significant harms associated.

And this then comes back to that idea we’ve been talking about previously of if you miss a diagnosis. Still, I don’t like the word miss, but we’re going to stick with it. So actually, sometimes you may not succeed in saying a patient’s got a diagnosis and you then don’t have the opportunity to give them the treatment. But the actual truth is is that in a large number of those patients, the treatment would have had no benefit anyway. Correct. And for some of those patients, the therapy may actually be harmful, as you say.

So we can really start to think about what it is we do, where we have an impact, where these decisions really matter. I sometimes see people rushing to give certain medicines, clopidogrol in a cute myocardial infarction is an example. And everything is being put on hold to grab the 600-accomplidogrol from the cupboard to get in. But yet the nnt for that is relatively high. We’re forcing ourselves, we’re stopping doing other things for interventions that don’t actually make a huge difference in the majority of patients.

Absolutely. And that’s a good example because the drugs like that, the numinease treat is often quite high. But when you also associate it with a numinease treat with the timeliness of the intervention, so clopidogrol, good idea like it, we don’t use it. But that class of drugs for patients with significant acute coronary syndromes is good. But does it need to be done now versus half an hour? And if I need half an hour to just make sure that it is the right drug for that patient, then that’s a good thing to do. And I’ve had a number of junior doctors come up to me with great anxiety to say that we need to give them the low molecular weight heprin now before they go for their chest x-ray or have another intervention. And say well actually half an hour is probably not going to make a radical amount of difference if you want to do a diagnostic test to make sure that it’s safe to give those drugs. Because with each of these things there will be a harm associated. Absolutely. And we want to be as certain as we can we’re doing the right thing for the right people.

But these are concepts which I find really hard to explain when there’s such a rush to do everything now yesterday before the patient’s even got ill. You know, the rush to do things, we have got time to stop and think. And we don’t give ourselves that chance sometimes I think in this pressure to give them everything straight away.

I would agree. And by pressurising ourselves and making decisions early without adequate levels of information, we potentially put our patients at risk. So let’s just quickly revise because we’ve tackled some pretty complex, I think, statistical type analysis there, but put it into a, into a clinical context.

We’ve talked a bit about relative risk reduction, absolute risk reduction, how they’re very different. They’re the same, they’re the same figure, but just expressed in a different way. So relative reduction, absolute risk reduction and number need to treat are exactly the same data, but just expressed in different ways. But n and t is the easy one to understand. Because other ones can either misrepresent or be harder to understand so the relative risk can be an over exaggeration of benefit. So we’ve got to be slightly mindful of that. The absolute risk reduction doesn’t necessarily explain to patients in a way that’s useful, but we can just flip that and turn it into the n and t, the number needed to treat and that will help us explain the benefits.

And I think signposting people at least three times in this podcast at the n and t.com is probably one of the most useful things we could do because the numbers are simply staggering. So I’m really happy, I think that’s taken us through how to think about those things, anything else you want to add before the end of another surprise question perhaps that you’d just like to spring on me.

Oh, I’m not really into surprise questions. There’s one thing though. Oh, you go on. Yeah, so we in one question for you. Okay. You read a lot of blogs, don’t you? I try to keep up with them as best I can. The number needed to educate is that’s a number needed to treat for education of blogs. How many blogs do you have to read before you make a change in your clinical practice? The n and e, the n and e. I think we just opened a whole kind of research worms here. I think we could start. The number of blogs you need to read to change one person’s, so it’s the number of blogs a person needs to read to change their practice. Yes. So, what’s your n and e?

It’s relatively high, I think, but I guess I’m quite selective about the blogs I read. So I find authors that I enjoy and I think know what they’re talking about. And often they talk about things I don’t know about, and that’s why I deliberately choose them. So I’d hope it’d be relatively high. It’s not going to be one though. I definitely, I’m probably going to read at least ten things before I pick up one thing. So that doesn’t mean it’s not worthwhile because I’ll enjoy reading them and they’ll keep me interested and enthused, but it may not necessarily change what I do. How about you?

I think you’re right, actually. I think that’s a really interesting point about what it makes you think about. Because if I say that the number needs to educate, I reckon it’s about twenty, maybe fifteen. Okay. And the reason why I think it’s so low for me is it’s not necessarily what I do change in patient care, but I reckon for every fifteenth blog I have a conversation with a colleague, or a junior, or another specialty to stop questioning what we do. So my educational intervention, I reckon, is probably somewhere around fifteen. For every fifteen blogs, it drives me to do something else in the workplace to try and make the world a better place.

And I guess as people who write blog posts now and again, and for people listening, that’s something we can think about how we do when we are writing something is, how will this affect people? Will it make a difference? What’s the purpose of why I’m writing it? And hopefully we can influence people with good thoughts and make them enjoy their emergency medicine and be more effective as clinicians.