October 14, 2020

expert reaction to study looking at excess deaths during the COVID-19 first wave in 21 countries

A study, published in Nature Medicine, looked at excess deaths during the COVID-19 first wave in 21 countries.

This Roundup accompanied an SMC Briefing.

Prof Sylvia Richardson, MRC Biostatistics Unit, University of Cambridge, said:

“This is an important and well conducted study of excess mortality across 21 industrialised countries where mortality data was accessible as far back as 2015. Excess mortality is a key metric for international comparison of the effect of the pandemic as it is not influenced by how each country has coded covid-19 related deaths, and it captures both direct and indirect effects of the pandemic.

“The study uses sound time series methodology to predict the counterfactual number of deaths in each country for the period January to May 2020, i.e. to predict the expected number of deaths in the absence of pandemic. They have validated their flexible and robust modelling approach on time series data before the pandemic and hence the estimates they report are backed by solid statistical methodology.

“The conclusion of their analysis makes stark reading for the overall effect of the pandemic in these countries, with over 200,000 estimated excess deaths. Interestingly, the excess deaths did not differ markedly between women and men, in contrast to what is reported for the direct effect of SARS-CoV-2 infections. They find substantial heterogeneity of excess deaths per 100,000 among the 21 countries with England &Wales and Spain excess death rates estimated to be between 90 to 102 for both sex per 100,000 population, the highest rates amongst the group of considered countries. Such well conducted international comparisons are useful for learning best international practise for pandemic management.”

Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:

“Epidemiologists and statisticians have been saying, ever since the start of the current pandemic, that the best way to assess the effects of the new virus on mortality is to look at excess deaths. Just looking at the numbers of deaths attributed to Covid-19 is problematic. What counts as a Covid-19 death is defined differently in different countries. In many countries, including the UK, whether a death is attributed to Covid-19 depends on which data source you look at. The numbers reported daily by the Government are for people who at some stage were ‘confirmed cases’, that is, they had a positive test at some point. In the early stages of the pandemic, it was difficult to get a test, unless one had been hospitalised, so many people who would really have been infected were never ‘confirmed cases’. However, the numbers reported by the Office for National Statistics (ONS) weekly are based on death registrations, and they count deaths where Covid-19 was mentioned on the death certificate by the doctor involved, whether or not the person had been tested. Other countries have different definitions, and definitions have changed over time.

“A clearer picture can come from just counting the number of people who have died, from any cause, in a period of time, and seeing how much bigger (or smaller) that number is than the number that might be expected, on the experience of previous years before this pandemic.  That entirely avoids the issue of what counts as a Covid death, and it also picks up extra deaths that occur because of changes to health services that restrict access to cancer treatments, for example, as well as possible reductions in deaths, maybe because there’s less traffic and fewer road accidents during lockdowns. Looking at excess deaths isn’t new – it’s been used for many years to look at effects of influenza outbreaks, or extreme weather events, for example.

“This new piece of research is far from the first to look at excess deaths during this pandemic. ONS reports weekly on excess deaths in England and Wales, there have been several previous academic studies, and media outlets (including the Economist, the Financial Times and the New York Times) and online information websites such as Our World in Data publish information on excess deaths*, with useful infographics allowing comparisons between different countries. The new report covers 21 countries, though not some of the richer countries that have attracted particular interest (such as the USA or Germany). It looks at excess deaths only up to the end of May, which covers the initial major wave in those 21 countries but not developments since then. And, in broad terms, the researchers report that the numbers they found were not hugely different from those in other sources. There might be some minor differences in which countries look worst and best, in terms of excess deaths, but it’s the usual suspects at the top of the scale of high numbers (England and Wales, Spain, Belgium, Italy, Scotland, Belgium) and further down (with 10 of the 21 countries showing no clear evidence of any rise in deaths above the expected level). So what’s new and different in the new research?

“Though it sounds relatively easy to compare numbers of deaths during the pandemic with the number you’d expect if there were no pandemic (at least in countries with good death records), it’s not as straightforward as you might think. An important choice to be made is how the expected number of deaths, without a pandemic, is calculated. Different analyses of excess deaths have used different methods – average numbers of deaths from the five years before the pandemic, for instance, but that doesn’t take account of changes in the size of the population, or the proportion of older people in the population, that might affect numbers of deaths. There are more elaborate methods that involve building statistical models for past numbers of deaths. While some previous sources of data have also used such methods, this new research takes a comprehensive approach to the modelling, that can take into account some uncertainty about what is the most appropriate model to use. (Should the modelling take into account differences in weather patterns between years, for instance?) I think that the modelling approach in the new research is statistically sound, and a real advance on most previous methods.

“This approach means that the new research can report more detail on two main aspects. First, they can give data for different age groups and can compare the results for men and women. Some of this has been considered before, though not in so much detail. This allows the researchers to establish that numbers of excess deaths for men were not in fact (on average across these countries) all that much greater than the number for women, and to point out that the pattern of gender differences was not the same in all countries, or at all ages, and changed over time even during the relatively short period from mid-February to the end of May. In older ages (65 and older), differences in excess mortality between men and women were less marked in most countries than at younger ages.

“Second, the new research can give information on the uncertainty in its estimates of excess deaths. For instance, it doesn’t just give a single point estimate for the total number of excess deaths across these 21 countries between mid-February and the end of May – it does give a point estimate (206,000) but it also gives a 95% credible interval, from 178,100 to 231,000, that shows the range of estimates that would also be reasonably supported by the data, taking into account the uncertainties in the process for estimating how many deaths might have occurred without the pandemic. For England and Wales, the point estimate is 57,300 but the interval goes from 48,900 to 65,000. That’s quite a wide interval, and it does show that a single number is an oversimplified picture of what happened. (The point estimate for Scotland is 4,600, with an interval from 3,700 to 5,500.)

“A further aspect of the new research, that hasn’t always played much of a role in other data on excess deaths, is a discussion of potential reasons for the difference between countries in the excess deaths that they experienced. Here, the conclusions aren’t by any means so firmly based on data. The available data really didn’t give too much information on the details of what went on in countries – as just one example, the researchers did not have comprehensive data on the recorded causes of death. There are very many ways that these 21 countries differ from one another, and sorting out those differences statistically is very difficult indeed. But the researchers’ three categories of potential determinants of differences do at least look very plausible – characteristics of the populations (including ages, social inequalities, environment differences, and more); the policies of the governments (for instance on the timing of lockdowns and the testing and contact tracing systems); and the state of public health systems and wider health and social care systems in the different countries.

“Because this new analysis only goes up to the end of May, and because it can’t provide a lot of detail in comparing policy choices so far, I don’t think it can provide much clear guidance on policies that should be used in the situation in which we currently find ourselves (in the UK or in any of these countries). But since their modelling system has now been developed, I imagine that it will be relatively straightforward for the researchers to extend the timescale covered. That might throw further useful light on what works under different circumstances.”

* Economist: https://www.economist.com/graphic-detail/2020/07/15/tracking-covid-19-excess-deaths-across-countries

FT: https://www.ft.com/content/a2901ce8-5eb7-4633-b89c-cbdf5b386938

NY Times: https://www.nytimes.com/interactive/2020/04/21/world/coronavirus-missing-deaths.html

Our World in Data: https://ourworldindata.org/excess-mortality-covid

Dr Amitava Banerjee, Associate Professor in Clinical Data Science and Honorary Consultant Cardiologist, Institute of Health Informatics, University College London, said:

“Excess deaths whether compared with previous years, or compared with modelled scenarios in the absence of the pandemic, continue to be an important metric to assess how badly countries are affected by COVID-19. This well-designed cross-country study of excess deaths during the pandemic uses standardised methods and nationally reported data across 21 high-income countries with three important contributions.

“First, the authors divided the countries into four groups (high, medium, low and no effect on excess deaths), a classification which could be used across other countries as well. Second, England and Wales and Spain were the worst affected with 37% and 38% relative increases in deaths respectively, and Bulgaria, New Zealand, Slovakia, Australia, Czechia, Hungary, Poland, Norway, Denmark and Finland had no appreciable effect on excess deaths. Third, they confirm what we have seen since the early pandemic in Wuhan, in that individuals over the age of 65 accounted for 94% of excess deaths across countries.

“A limitation is that the authors did not have access to data about cause of death or specific comorbidities and risk factors for the people who had died during this period. However, based on existing detailed data about demography of these countries, they make insightful conclusions regarding the reasons for wide variation between countries. First, they conclude that although baseline risk is “necessary” to explain excess deaths in older individuals, it is “not sufficient” given that there are similarly age structures and high rates of comorbidities such as obesity in the populations of both the UK and New Zealand. Second, they agree with prior analyses that countries which instituted lockdown early and at the point of lower cases and deaths (e.g. New Zealand), fared better (and actually avoided any effect on excess deaths) than those which were late (e.g. the UK). They add that Sweden, which did not implement a lockdown has faced a prolonged burden of excess deaths, compared with other countries. Third, those countries with public health infrastructure in place have had lower excess mortality.

“Therefore, yet again, the message is very clear and incontrovertible. Even if vaccines and better treatments for severe SARS-CoV-2 infection are developed, the way to minimise excess deaths is to reduce the infection rate through population level measures (including lockdown), protecting and treating those at higher risk, and implementing and maintaining adequate “test, trace and isolate” systems as a matter of urgency. Without these three strategies, more cases and excess deaths will occur. Also, these analyses highlight the importance of having continuous monitoring of available mortality data across countries during the pandemic.”

‘Magnitude, demographics and dynamics of the effect of the first wave of the COVID-19 pandemic on all-cause mortality in 21 industrialized countries’ by Kontis et al was published in Nature Medicine at 10:00am UK time on Wednesday 14^th October.

DOI: 10.1038/s41591-020-1112-0

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

Prof Kevin McConway: “I am a Trustee of the SMC and a member of the Advisory Committee, but my quote above is in my capacity as a professional statistician.”

Dr Amitava Banerjee: “No conflicts of interest.”

None others received.