select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
NHS England have published a press release stating that, since its discovery as an effective treatment for COVID-19, dexamethasone has saved around 22,000 lives in the UK and around one million lives worldwide.
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“I’m pretty dubious about the exact figures for lives saved by the use of dexamethasone. It’s definitely true that large numbers of people are alive today because they were given dexamethasone, but some of the numbers do seem inappropriately precise. 22,000 lives saved in the UK? The press release doesn’t even say “approximately 22,000” until we get to the Background section after the main release. Details on exactly how this number was calculated are vague. It is based on results from a paper published in Nature Communications last month* (NB not Nature itself), but which itself reads a little oddly until you realise that a preprint version of it appeared in July last year**. That’s why it talks about the “potential” impacts of dexamethasone treatment between July and December 2020 – it was originally written right at the start of that period – and why its conclusions depend on a range of assumptions about how many people would be expected to end up in hospital with Covid-19 in the UK, even though by now, and by the time the paper was actually published in the journal, we do know how many were hospitalised. That paper concluded that the number of lives saved in the UK between July and December could be 12,000 but the researchers attached a very wide margin of error to that, from 4,250 to 27,000. Some of that uncertainty could now be removed, because we now do have data on the number of positive cases of Covid-19 over the period. Also the results have been updated to cover the period up to this month (March 2021). However, the original Nature Communications paper seems not to have taken into account uncertainties about proportionally how much reduction in deaths would occur, in patients given oxygen but not on ventilation, and in patients on ventilation, if they were treated with dexamethasone. Estimates of those levels of reduction were taken from the report of the clinical trial of dexamethasone, which was indeed a remarkable UK initiative – but the relevant RECOVERY trial report*** does express some substantial uncertainty about the effects on mortality. For patients receiving mechanical ventilation, the reduction in deaths was indeed estimated as 36%, but with a margin of estimation error that goes from 19% to 49%. For patients receiving oxygen but not on ventilation, the reduction was estimated as 18% but with a margin of error from 6% to 28%. That’s a considerable level of uncertainty. And it could be possible that, in current conditions, the proportion of lives saved by dexamethasone could be rather different from in the original trial anyway, because there have been other improvements to treatment as well. And the balance between numbers of infections, hospitalisations, and the need for oxygen treatment is already changing anyway, because of vaccination.
“We have an assurance from the research team that produced the original Nature Communications paper that the calculations in the press release are consistent with their methods and calculations – but we still do not know how uncertain the detailed figures actually are. Maybe that isn’t crucial. I don’t doubt that many lives were saved – but to attach a precise figure when there is probably still substantial uncertainty does seem to me to be using the number only for rhetorical purposes rather than being appropriately informative.
“So far I’ve described this only in relation to the UK estimate. The press release does at least mention that there is uncertainty about the global estimate for lives saved by the use of dexamethasone – though the main text says the number is a million, the Background section says that the estimate is approximately 958,000 but could be as large as 1,400,000. I’m not going to complain, given this level of uncertainty, that the main text says one million and the actual estimate is slightly less. But I will point out that, if the actual number could be 1.4 million instead of about a million (because of unknown factors), it could presumably be a good bit less than a million, plausibly, as well. Expressing uncertainty in only one direction is bad practice. The original Nature Communications paper gave a range from 240,000 to 1,400,000 for the global figure – probably the extra three months that are now included, and the fact that some things are now known better than they were last summer when that paper was originally published, mean that the reasonable lower limit would now be bigger than 240,000. But it could still be a lot less than a million, and I can’t say how much less because no details are given.
“I should make it very clear that I’m not running down the wonderful work of the UK team that developed and trialled the dexamethasone treatment. It’s extremely important, the work was carried out in conditions of extreme urgency, and huge numbers of lives have been saved, whatever the exact figure might be. I’m also definitely not complaining about the other NHS triumphs described in the press release. NHS workers of all kinds have done very remarkable work during the pandemic, and we’re extraordinarily luck to have them. I am, however, complaining about big numbers being used simply as rhetorical devices.
“And I also want to draw attention to one other aspect. Dexamethasone does save lives by reducing the death rate of people on mechanical ventilation by somewhere around one third, for example. But that means that about two thirds of the people on mechanical ventilation, who would have died if they weren’t given dexamethasone, still die anyway even if they are given it. Similarly, about four fifths of the people receiving oxygen but not ventilation, who would have died if they weren’t given dexamethasone, still die anyway even if they are given it. And there’s no evidence that I know of that dexamethasone does anything to reduce mortality in people in hospital who do not receive oxygen. So it’s wonderful that dexamethasone has saved lives, and continues to save lives, but more lives could have been saved if more people had not been infected at all and therefore had not needed hospital treatment. Dexamethasone, and other treatment improvements, can only do their life-saving work after the whole process has already been started, by people being infected in the first place. I’m not going to comment on what might have been done differently to reduce infections, but looking just at the great work done by the NHS and by those working on treatments is a bit like praising the fire service for doing a good job putting out fires. Of course, they deserve the praise, but that’s only part of the picture – we’ve also got to take into account the work (also done partly by the fire service) in preventing fires from happening in the first place, including work by building control authorities in not allowing the construction of buildings that can catch and spread fire too easily.”
* https://www.nature.com/articles/s41467-021-21134-2
** https://www.medrxiv.org/content/10.1101/2020.07.29.20164269v1
*** https://www.nejm.org/doi/10.1056/NEJMoa2021436, published February 2021. However, this is the final version – a preliminary version was published online by the NEJM in July last year, and a preprint of that at https://www.medrxiv.org/content/10.1101/2020.06.22.20137273v1. The estimates in those versions are close to those in the final paper.
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. I am also a member of the Public Data Advisory Group, which provides expert advice to the Cabinet Office on aspects of public understanding of data during the pandemic. My quote above is in my capacity as an independent professional statistician.”