select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A preprint, posted on ResearchGate, reports on the potential impact of the COVID-19 emergency on mortality rates in people with cancer.
Mr Hugo Pedder, Senior Research Associate in Statistical Modelling at the Bristol Medical School, University of Bristol and Statistical Ambassador to the Royal Statistical Society, said:
“This preprint highlights that the impacts of the response to COVID-19 on other illnesses may eventually cause a comparable number of deaths to those caused directly by the virus if we do not change the way healthcare services are delivered.
“This research has not yet been peer-reviewed. It is a preprint, which means that it has been made available prior to peer-review. Whilst this helps make potentially important findings publicly available as quickly as possible, this means that it hasn’t yet been subjected to the same level scrutiny as peer-reviewed research.
“As with many studies that are being published at the moment, this research involves modelling different scenarios to make predictions about what might happen in the future. Therefore, changes that the authors couldn’t predict may happen that would substantially impact their results and conclusions.
“The press release accurately reflects the findings in the article and clearly highlight the key issue it raises – that COVID-19 and the impacts of the response to it are likely to have significant effects on other illnesses, which will lead to additional deaths.
“The numbers of excess deaths change a lot depending on the choice of input values used in the model (different values for the % population affected by the emergency (PAE) and the relative impact of the emergency (RIE)). However, all the numbers of excess deaths are still large and should be a cause for concern. Focusing on the specific numbers predicted ignores the uncertainty/sensitivity of the model to the input values, but the message from the paper is clear –the COVID-19 emergency will lead to additional deaths from cancer unless there are changes to the delivery of cancer care.
“The figures for the declines in chemotherapy appointment attendance and referrals are clearly a cause for concern regardless of whether the correct number of deaths predicted by their model are accurate or not. It should be clear from these easily measurable numbers alone that this is a problem, regardless of the subsequent modelling results.”
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“It’s well known that there have been considerable changes in the provision of health care for many conditions, including cancers, as a result of all the measures to deal with the COVID-19 crisis. It seems inevitable that there will be increased deaths in cancer patients, if they are infected with the virus, or because of changes in the health services available to them, and quite possibly also from socio-economic effects of the responses to the crisis. This study is the first that I have seen that produces a reasonably-argued numerical estimate of the number of excess deaths of people with cancer arising from these factors, in the UK and the USA.
“The statistical model used to produce these estimates looks generally sensible to me, but that does not mean that the estimates of excess deaths should be treated uncritically. As inputs to the modelling, the researchers have put together some very useful data on the declines in attendances for chemotherapy and in urgent referrals for diagnosis at some major UK cancer centres since the COVID-19 crisis started to bite, and these declines are large. They have also put together detailed and useful data on the effects on cancer death rates if patients have other conditions as well as their cancer. That all makes good sense, but in order to turn these inputs into estimates of excess deaths, the researchers have to make assumptions about certain other quantities, on which reliable data is simply not yet available. This applies particularly to the quantities they call PAE (Proportion of the population Affected by the Emergency) and RIE (Relative Impact of the Emergency). The researchers are quite candid about having no data to establish PAE, and they did their calculations for a range of PAE values going all the way from 10% (relatively low impact) to 80% (relatively very high impact). However, in the main paper (and the press release) they very much concentrate on presenting the numbers for just one value of PAE, 40%. They say that is ‘plausible’, and I don’t disagree, but a wide range of other possible values are also plausible, I’d say. For RIE, the only evidence they present for the particular, quite narrow, range of values that they mainly report on is the Office for National Statistics estimates of excess deaths during the COVID-19 emergency in England, and for this paper they could use only the data for deaths registered up to 10 April. By that date there had been only two weeks with large numbers of excess deaths (compared to the long-term-average), and that’s not much to go on to produce a useful value of this input.
“The researchers quote prominently that there could be 6,270 additional deaths in a year in newly diagnosed cancer patients – but using different numbers for PAE and RIE, in the ranges they considered, this number of excess deaths could be as low as a few hundred, or well into tens of thousands. We really don’t know, because nobody has the right data yet. So, just because of the lack of data on these two quantities, the number of excess deaths in cancer patients could be quite a lot less than the main estimates given in the paper and press release, or it could be a lot more. We simply don’t know, and we won’t know for some considerable time. The researchers themselves also point out several other reasons why the true numbers of excess deaths could be importantly different from the numbers in their paper.
“Really, I think the true message of this research is not the estimates of excess deaths themselves, however prominently they appear in the research results (including in the press release). Any such deaths matter, of course, and the research does plausibly make the case that the number could be pretty large, even if we really can’t be at all sure exactly how large. However, this loss of life is not all inevitable. Where I very definitely agree with the researchers, is that they ask for better and more timely data that will allow better estimates of many quantities that are important for planning cancer and other health services during the continuing crisis. That will allow more effective provision of health care for these vulnerable patients, and that’s what can save lives.
“I should point out that this research paper has not yet been peer reviewed by other scientists, who might find important issues with it that I would not have been able to find in the short time I’ve had to consider it.”
Prof Dame Jessica Corner, Pro-Vice-Chancellor (Research & Knowledge Exchange), The University of Nottingham, said:
“I think this analysis does need thorough review – noting it is pre-peer review, the analysis and interpretation of the data would be further refined with such input.
“It is very likely that we will see significant excess death from cancer (as a result of COVID-19). I am not sure that this dataset yet provides the evidence for this or importantly for insights into causation. March and April will be crucial data points for people accessing services where the peak impact of decisions to not provide cancer treatment or whether social isolation has impacted, will be more evident than in the months preceding this as reported in the data in this paper. The effect is likely to be more profound and to build over time as a result of this. These data, however do represent an important signal that we should be worried about and monitor health outcomes and excess mortality carefully over a longer timeframe. Distinguishing services delivery issues – eg the cancelling of all cancer treatment in March/April vs the impact on health outcomes of late diagnosis, deserve further interrogation.
“Interpretations relating to co-morbidities: the relationship of these to health outcomes following cancer treatment is not a new observation. It is well know that these have a major impact on health outcomes, this was known before COVID-19 and is there highly like to be increased.
“Some important and interesting data in this paper that should raise alarm signals, but really needs a longer timeframe of data points for the full picture to emerge and for confidence in the interpretations as to the causative factors.”
Dr Simon White, University of Cambridge and Statistical Ambassador for the Royal Statistical Society, said:
“Estimating the number of excess deaths is an important question, for researchers, public health policy, and for our society to understand the impact of COVID-19. However, there is substantial uncertainty that his paper does not properly communicate. The authors acknowledge there is limited, or even no, information to inform these calculations. However, they present single numbers to a spurious level of precision, rather than intervals of likely estimates which would better reflect our current level of understanding and would not mislead the public about the accuracy of the numbers presented.
“The article is not yet peer-reviewed, which means that no scientists other than the authors have had an opportunity to read or critique the paper, its methods, or conclusions – often providing suggestions to improve the presentation of a papers results and conclusions, or the challenge some aspect of the research.
“The press release reflects the intention of the article, to present an estimate of the increased mortality faced by patients with cancer and calculate a number of deaths, however it fails to properly qualify the substantial uncertainty in the papers conclusions.
“The authors have used the best available data to inform their conclusions, they acknowledge that better (more complete and detailed) data is not easily available in real-time (p8: “information governance for such data can take months to secure, making data-enabled research and time-sensitive responsive service improvement difficult”). However, these governance systems are in place to balance the security and privacy of patient data and its utility for research. There is no simple answer to this problem (especially at the current time).
“The authors are attempting to estimate the impact of COVID-19 on mortality amongst cancer patients (both in the UK and US contexts). They use a method of estimating the expected number of deaths, using available data on historical cancer incidence (new cases) and prevalence (existing cases), and adjust these estimates based on the “Relative Impact of the Emergency (RIE)” – which they define in their paper. This RIE is, in effect, a combination of all factors that would increase the mortality of cancer patients.
“There is no substantial evidence, in the paper nor practically available, to inform what the values of Proportions of the population Affected by the Emergency (PAE) and RIE should be (p8 “Since empirical estimates of each of these four parameters for PAE are not yet available, we chose a PAE range of 10%-80%”). The authors state, without any supporting justification, “40% as a plausible estimate” (p8). The exact results of the paper very heavily depend on this unjustified choice, however the overall message seems robust as presented.
“Importantly, the authors fail to properly present the uncertainty in their conclusions. They state the point estimate (a single number), but do not express how certain they are. This would typically be expressed as a range of values , which would be a combination of the uncertainty in the data (since it is based on only 8 hospitals in the UK then scaled up to the whole UK population), and their uncertainty in the values of PAE and RIE.
“They do present one example of this reporting within the paper on p3, “PAE 40%, we observed 11,645 excess deaths (range 4,655-23,287)”. All other point estimates within the paper/press release should have an uncertainty interval as well (these are not presented, which should add caution to any interpretation of the findings).
“The number reported is “excess” deaths, this could be zero or even negative, if the number of deaths would not be different without COVID-19. Further highlighting the omission of uncertainty intervals.
“The authors use the phrase “excess deaths” to mean the number of additional deaths predicted by their model accounting for an increase in mortality (combining the PAE and RIE values). The phrase “excess deaths” is not universal, and different researchers may use different definitions.
“Given the substantial uncertainty, most numbers in the paper are reported to a spurious level of accuracy. It is not really 11,645 excess deaths, these numbers should be rounded (probably to the nearest hundred) to stress that they are not know precisely (this is even more true for the US figures).
“In summary, the paper is attempting to quantify the impact of COVID-19 on an at risk patient group. The general opinion is that the impact will be negative, with additional deaths not directly due to COVID-19 infection, but the true extent is still very uncertain and this paper fails to proper express that uncertainty.”
Prof Chris Bunce, Professor of Translational Cancer Biology, University of Birmingham, said:
“The team behind this research are a large group of well-established researchers with considerable combined experience in their field. Thus, although published as a pre-print and as yet not reviewed by an independent group of peers in the field, the study represents a hugely valuable contribution. These are challenging times across the world and if this information is to influence cancer care and guide policy during the COVID19 crisis then it is important that the findings are disseminated and discussed immediately, warranting their release ahead of peer review.
“Although complex at first sight some easy to grasp information can be derived. The team’s near real-time analysis demonstrates dramatic COVID19-related falls in admissions for chemotherapy and still further dramatic drops in urgent referrals for cancer diagnosis. This clearly indicates that increases in delayed diagnosis and treatment is already occurring in a significant number of new cancer cases. The urgency of diagnosis and treatment varies with different cancer types and different individuals. However, it is universally accepted that early diagnosis and treatment and adherence to treatment regimes saves lives and therefore that these COVID19 related impacts will cost lives.
“In addition the study highlights that many cancer patients have other health conditions besides their cancers that increase their risk of death. These patients who are perhaps most vulnerable to the current situation are still more affected since many of their additional conditions render them high risk if they themselves contract COVID19.”
‘Estimating excess mortality in people with cancer and multimorbidity in the COVID-19 emergency’ by Lai et al will be posted on Wednesday 29 April on ResearchGate at 00.01 UK time and is under strict embargo until this time.
This is not peer-reviewed work.
All our previous output on this subject can be seen at this weblink: www.sciencemediacentre.org/tag/covid-19
Declared interests:
Mr Hugo Peddler: None
Prof Kevin McConway: Prof McConway is a member of the SMC Advisory Committee, but his quote above is in his capacity as a professional statistician.
Dr Simon White: None to declare. (The Medical Research Council Biostatistics Unit, University of Cambridge, is involved in COVID-19 research; Dr White is not part of those research teams and The Royal Statistical Society has a COVID-19 task force; Dr White is not part of that task force)
Prof Chris Bunce: I have no conflicts of interest.
None others received.