January 20, 2023

expert reaction to the ONS monthly mortality analysis for England and Wales: December 2022

The Office for National Statistics (ONS) have released their monthly mortality analysis for England and Wales in December 2022.

Prof Sir David Spiegelhalter, Emeritus Professor of Statistics, University of Cambridge, said:

“After adjusting for changes in the size and age of the population, this provisional ONS data show no excess death rate over 2022, but this is in strong contrast to other sources.  This is explained by the sharp deficit in deaths identified earlier in the year, which is partly because they include 2021 as a baseline year, and early 2021 had high Covid mortality.

“Heat is very dangerous.  ONS has also published deaths by day of occurrence, which shows a big spike during the record hottest days of July 19th and 20th – there were over 1,100 excess deaths in England and Wales on those two days alone.

“Over the year there was a substantial excess in the broad category ‘symptoms signs and ill-defined conditions’, often used for the frail elderly, with around 4,700 excess deaths, an increase of over a third.  But this was almost exactly compensated for by a large overall deficit of around 4,700 in ‘influenza and pneumonia’, which caused few deaths last winter.  It is possible that some of the older people would otherwise have been taken earlier by flu.

“ONS report 163,000 deaths caused by Covid-19 since the start of pandemic in England and Wales, while registering 52 deaths as having been caused by the Covid vaccine.”

Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:

“This isn’t a comment on the overall patterns of mortality, which are pretty clearly summarised in the bulletin itself.  There are interesting points, although some are arguably not all that interesting except to nerds like me.  One such point is something that comes up every month in these bulletins but has been in the news and in various public statements recently – it is to do with how to measure excess deaths.  In response to various media stories about excess deaths, based on the ONS weekly provisional analyses of deaths in England and Wales, people have complained that the baseline used to measure the excess, the five-year average number of deaths (with the current 5 years being 2016-19 and 2021, leaving out 2020 because it was so untypical), is not appropriate because it takes no account of changes in population size and age distribution over time.  There’s certainly something in that – but what’s interesting (to nerds like me) is that the monthly bulletin does it that way but also another way, and compares the two results.

“The alternative that they use is to calculate the age-standardized mortality rate (ASMR), which does take into account the changes in population age structure and (in a different way) the population size, and then to compare that with the 5-year average of the ASMRs (for the month in question, and they also do it for the whole of the year to date, so in this case for the whole of 2022).  This doesn’t directly give a number of excess deaths, but it can still give a figure for the percentage by which 2022 ASMRs (and the same for individual months) are above or below the 5-year average.  They calculate the same sort of percentages for the excess numbers of deaths too, and then they compare them in section 3 of the bulletin.  So for December 2022, for all-cause deaths, and for England (they do England and Wales separately) the number of excess deaths was about 5,900 which is 13.5% above the 5-year average, but the ASMR for that month was 5.8% above the 5-year average.  Consistently, and certainly throughout 2022, the percentage excess based on ASMRs is less than the % based on death counts.  That indicates that the criticism of the 5-year average baseline for excess death counts has some validity – part of the ‘excess’ does have to do with population change, and allowing for that by using ASMRs makes the excess smaller.

“For the whole of 2022, again for England, the bulletin says there were about 32,000 excess deaths, which is 6.3% above the 5-year average count of deaths – but the ASMR for the whole year was actually slightly below the 5-year average (0.7% below).  For Wales for all of 2022, there were about 1,700 excess deaths, 5.0% above the 5-year average, but the ASMR was again below the 5-year average, 1.1% below.

“In a sense what this indicates is that measuring excess deaths isn’t that straightforward, the problem (in a country with good death registration where we know pretty accurately how many people actually died) being in the baseline figure you chose to compare with the actual number of deaths (or the actual death rates).  There are yet more ways to do it, for instance those used by the Office for Health Improvement and Disparities (OHID) at https://www.gov.uk/government/statistics/excess-mortality-in-england-and-english-regions (for England), and by the Institute and Faculty of Actuaries at https://www.actuaries.org.uk/learn-and-develop/continuous-mortality-investigation/other-cmi-outputs/mortality-monitor.

“Returning to the ONS monthly bulletin’s methods, they are careful to point out another snag with the ASMR method.  It is that, to calculate the ASMRs, you need to know the population size in each age group.  ONS could not (yet) use estimates based on the 2021 Census results because they are not yet available for the relevant dates, and in fact they used projections based on population estimates from 2018.  These do not currently take into account major effects, such as the Covid pandemic, that would have affected the population size and pattern of ages.  So you certainly can’t say that the ASMR-based excess measures are perfect – but then I don’t think there is such a thing as a perfect excess mortality measure.”

https://www.ons.gov.uk/releases/monthlymortalityanalysisenglandandwalesdecember2022

Declared interests

Prof Sir David Spiegelhalter: “I am a Non-Executive Director of the UK Statistics Authority, which oversees the work of the Office for National Statistics.”

Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee.  My quote above is in my capacity as an independent professional statistician.”