select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A preprint, an unpublished non-peer reviewed study, reports on the latest data from the REACT-1 study on COVID-19 spread across England.
This Roundup accompanied an SMC Briefing.
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“In terms of the broadest patterns of change in the data, the results from Round 13 of the REACT-1 do fit in with what we’ve seen from other data sources. There are, however, several more detailed points that do add to what we’ve seen before – for example on vaccine effectiveness, though those results are not straightforward to interpret, as I’ll explain later. The samples for Round 13 of the REACT-1 survey were taken between 24 June and 12 July. So they can’t possibly show any effects of the removal of most legal restrictions in England on 19 July. Also they can’t possibly tell us about the recent decreases in the daily numbers of new confirmed cases of Covid in England, as shown on the dashboard at coronavirus.data.gov.uk and elsewhere, because those reductions didn’t begin until about 19 July, after this round of the REACT-1 study had ended. Therefore they can’t throw any light at all on the disputed question of the extent to which those recent reductions in confirmed cases counts are really reflecting what’s going on with the pandemic. That’s a pity, because the questions that have been raised about the dashboard figures are concerned with how far the reductions in confirmed cases could be due to differences in the numbers of people, and type of people, who are being routinely tested. Those kinds of possible bias do not appear in a survey like REACT-1, which bases its findings on swab tests carried out on a representative sample of the English community population, who are tested only to measure the state of the pandemic (and not because of their symptom status, or the job they do, or whether they were pinged by the app, or anything like that).
“What they can and do show, though, is a very considerable increase in the estimated number in the population who would test positive for the virus that can cause Covid-19, between Round 12 of the survey (20 May to 7 June) and Round 13. For Round 12, the REACT-1 researchers estimated that about 1 in every 670 people in the English community population (aged 5 and over) would test positive. As this estimate comes from a survey, there is a statistical margin of error, which ran from about 1 in 560 to 1 in 830. Infection rates were very low around the end of May, so that there was quite a lot of statistical uncertainty. For Round 13, the central REACT-1 estimate is that about 1 in every 160 people would test positive, with margin of error from about 1 in 145 to 1 in 175. But that big increase in infections between late May and early July does broadly match the rise in new confirmed cases that we saw over the same period. It also roughly matches the findings of the ONS Covid-19 Infection Survey (CIS) over the same period. For the period of REACT-1’s Round 12, the average CIS estimate of the proportion of people testing positive in England was fairly close to the REACT-1 estimate, at about 1 in 620. For the period of REACT-1’s Round 13, the CIS estimate of about 1 in 120 testing positive was considerably higher that REACT-1’s 1 in 160. It’s not surprising that the two survey estimates don’t line up exactly. Despite the large number of people swabbed for the surveys, it’s not easy to get very precise estimates (because most people do not test positive). And there are differences between the surveys anyway. The ONS CIS estimates are for the population aged 2 and over, and the REACT-1 estimates for age 5 and over (not that that will make all that much difference). The CIS and REACT-1 use different lists to sample from, and CIS samples households while REACT-1 samples individuals, and the two surveys use different statistical methods to deal with any lack of representativeness in their samples. So the fact that they don’t always coincide exactly is mainly just a reminder that we shouldn’t assume the estimates are exact. They do both agree that infections went up really substantially between late May and early July, and that was probably for a number of reasons, including the increasing dominance of the Delta variant. As well as the big increase in infections between Round 12 and Round 13, REACT-1 found clear evidence that the rate of testing positive was increasing during their round 13, and that again broadly fits in with the CIS findings and the confirmed case numbers. Both surveys found that these increases were to a considerable extent due to increased numbers of young people (aged under about 25) testing positive.
“In what I’ve described so far, there’s nothing much in REACT-1 that we didn’t already know from other data sources, though it’s always useful to be able to check that the sources are broadly agreeing. But the REACT-1 analysis looks at several other details that aren’t covered in the other sources, at least not in so much detail. That applies, for instance, to the analysis of vaccine effectiveness. The REACT-1 researchers made estimates of several different aspects of vaccine effectiveness. Those effectiveness results are quite encouraging, even though they are generally lower than many estimates from other sources. But they need a lot of care in interpretation, I think, for several reasons.
“The researchers point out that there are rather large differences in the estimates of vaccine effectiveness between participants who agreed to have their survey results linked to their NHS records, and those who did not agree, so that they had to rely for their main vaccine effectiveness estimates on what participants said their vaccine status was, which may not always be accurate. More widely, the vaccine effectiveness estimates are based on comparisons of vaccinated and unvaccinated people in terms of whether they tested positive during single rounds of the survey, and a round is a fairly short period. And differences in infection rates between vaccinated and unvaccinated people aren’t necessarily all caused by their vaccination status. There are differences between the vaccinated and unvaccinated groups apart from whether they have been vaccinated, and those other differences might well be part of the explanation for differences in infection rates between vaccinated and unvaccinated people. The REACT-1 researchers did make statistical adjustments to allow for some of these other differences – differences in age, sex, the level of deprivation of where the participants lived, the region of England where they lived, and their ethnicity, but there could be other important factors that they could not adjust for because they did not have the data.
“Another issue with the vaccine effectiveness estimates is that they are subject to very considerable amounts of statistical uncertainty. That’s inevitable, given the numbers of people who test positive. For example, the central estimate of vaccine effectiveness (for two doses) for all positive test results in round 13 of the survey, for people aged 18 to 64 years, after the adjustments for age, sex, deprivation and so on that I mentioned, is 49%. That means that the number of positive tests in vaccinated people would be about 49% smaller than the number in a matching group of unvaccinated people – so about half as many positive tests. This number is from Table 5 in the REACT-1 report – but that table also shows that the statistical margin of error around that number runs from 22% to 67%, which is very wide. If the true effectiveness is 22%, that means that the number of infections in a vaccinated group would be 22% smaller than in a matching unvaccinated group. Putting it another way, the number of infections in the vaccinated group would be a little over three quarters of the number in the unvaccinated group. If the true effectiveness is 67%, the number of infections in a vaccinated group would be about a third of the number in a matching unvaccinated group. So this data analysis has really not tied down the effectiveness very precisely at all. The number of infections in the vaccinated group compared to a matching unvaccinated group could be three quarters of the number in the unvaccinated group, of more likely somewhere near half the number, or perhaps a third of the number. So the vaccine effectiveness estimates do indeed indicate that vaccines reduce infections, and also that they probably reduce the chance of passing on the virus to others, but they really do not tell us anything very precise about how big the reductions are. I mention this partly because the press release from DHSC says “double vaccinated people in the most recent round were estimated to have around 50 to 60% reduced risk of infection, including asymptomatic infection, compared to unvaccinated people”, and one might think that the 50 to 60% range is a measure of the statistical margin of error. It isn’t – I believe it is just giving the range of central estimates for various different groups of participants and various ways of calculating the effectiveness, and the range of statistical uncertainty associated with the vaccine effectiveness estimates is considerably wider than “50 to 60%” would indicate.”
Dr Tom Wingfield, Senior Clinical Lecturer and Honorary Consultant Physician, Liverpool School of Tropical Medicine, said:
“The latest findings from the REACT-1 study indicate that the increasing prevalence of SARS-CoV-2 infection that we have seen in this country since May 2021 has been driven by the Delta variant and infections in young, unvaccinated people. The data also shows that vaccines (in this case self-reported rather than confirmed vaccination status) are contributing to a decrease in SARS-CoV-2 transmission and symptomatic illness.
“The REACT-1 findings, when coupled with other studies demonstrating the impact of coronavirus vaccines on reducing hospitalisation and death from Covid-19, are encouraging. While hospitals, including here in Liverpool, remain extremely busy with people requiring care for Covid-19 and non-Covid-19 illnesses, it appears that rates of hospitalisation nationally may have reached a plateau. This is in keeping with the apparent sustained reduction in community SARS-CoV-infection rates we have seen in government figures over the past few weeks.
“However, the latest REACT-1 study data also serve as a reminder that, even with extremely high vaccine coverage, we are highly likely to have a further wave of SARS-CoV-infections in the autumn. By September, we will likely have moved to “Step 4” of the government’s roadmap to ease restrictions, schools and Universities will be returning, and there will be an inevitable increase in social mixing and SARS-CoV-2 transmission. This is concerning as we look towards a long and difficult winter for the NHS, in which we will have to deal with Covid-19 and/or other seasonal respiratory infections plus an ongoing backlog of people with other illnesses who may have been unable to access care and health services due to the pandemic.
Prof Jonathan Ball, Professor of Molecular Virology, University of Nottingham, said:
“It confirms that SARS2 prevalence increased markedly from spring into early summer, and this increase was driven by circulation of the Delta variant, mainly in younger unvaccinated people.
“The data-capture did not extend into the past couple of weeks, where we are seeing a similarly rapid decline in reported cases. This reduction was to be expected given the increasing levels of population immunity, driven by both vaccination and natural infection.
“Whilst natural infection does provide immunity, is possible that this is relatively short-lived. Therefore, I would urge all eligible people to get fully vaccinated as soon as possible, even if you have recently been infected, so that we can continue to keep this virus under control.”
‘REACT-1 round 13 final report: exponential growth, high prevalence of SARS-CoV-2 and vaccine effectiveness associated with Delta variant in England during May to July 2021’ by Steven Riley et al is a preprint embargoed until 00:01 Wednesday 4th August.
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. I am also a member of the Public Data Advisory Group, which provides expert advice to the Cabinet Office on aspects of public understanding of data during the pandemic. My quote above is in my capacity as an independent professional statistician.”
None others received.