December 23, 2020

expert reaction to Interim Evaluation Report from the Liverpool Covid-19 Community Testing Pilot

A report from the University of Liverpool, has done an interim analysis of the Liverpool Covid-19 Community Testing Pilot.

This Roundup accompanied an SMC Briefing.

Dr Angela Raffle, Consultant in Public Health and Honorary Senior Lecturer University of Bristol Medical School, said:

“Liverpool University staff have worked hard to produce a comprehensive report on a pilot project that was implemented at very short notice in November.  In brief, the community testing in Liverpool was a costly undertaking and of uncertain value.  The underlying purpose of the exercise shifted with time.  Valuable information was gained about the performance of the Innova Lateral Flow device, which missed 60 percent of SARS-CoV-2 cases (if PCR testing is taken as the ‘standard’).  The take home message is that open access Lateral Flow testing for symptomless people is very unlikely to provide a cost-effective additional element in managing the covid-19 pandemic.

“Here are some brief summary points from me:

“498,000 residents in the City of Liverpool were offered tested using the Innova Lateral Flow Device, and 25 % of them underwent a Lateral Flow Test from 6 November and up to 9 December.

“897 individuals were identified from a Lateral Flow test as having SARS-CoV-2 infection, most of these people said that they immediately self-isolated and informed their family, friends or employer.  A proportion of these people will have had falsely positive results, and of asymptomatic people who truly had viral material present it is not possible to say how many were infectious.

“The authors report that the Lateral Flow testing had a sensitivity of 40%, or in other words it missed 60% of PCR positive cases.  This reveals that the continuing statements by DHSC that the Innova Lateral Flow test achieves ‘high accuracy’ and ‘78% sensitivity’ are not valid.

“There were concerns about the variation in quality of the testing kits and of the lack of quality assurance certification by batch of test.

“People from least deprived sections of the population and with lower risk of infection were twice as likely to attend than those from deprived groups, with higher infection rates.  People who had little ability to isolate if found positive were reluctant to attend.

“The costs of the exercise were considerable, with 2,000 military personnel assisting numerous Liverpool staff to establish and run multiple testing centres for several weeks.  The administrative costs will have been substantial.  The authors note that economic evaluation should be examined but that this is beyond the scope of their report.  Even with a low cost test, what matters is the full cost of delivering a high quality and effective testing programme.

“There is no evidence that that the community testing led to a change in Covid-19 case incidence or hospital admissions in Liverpool.

“Contrary to publicity during the Liverpool pilot, the report states that testing was initiated by the Department of Health and Social Care, rather than requested by Liverpool City Council.

“The short time allowed for planning and preparation was insufficient to enable adequate public information and good governance arrangements to be fully in place.

“The authors conclude that caution should be exercised regarding use of Lateral Flow tests, particularly when using the test to try and protect vulnerable people.

“Residents reported that they felt an ‘ethical imperative’ to take part, something which is not consistent with the principles of autonomy and fully informed participation.

“Of those who tested negative on Lateral Flow, some reported that they would be more likely to go out for exercise (23%), to the shops (17% ), visit family and friends ( 9%) or go to work (7%).

“Sending out of home testing kits achieved 8.3% compliance so was discontinued.

“Uptake in schoolchildren was 52.6%.  The authors attribute this to ‘negative media from outside Liverpool’, but it would be fair to acknowledge that the reason for this so-called ‘negative media’ included uncertainty about the value of testing all schoolchildren, concerns about rushed implementation, lack of standardised balanced information for parents and pupils, and lack of fully informed parental consent.”

Prof Paul Hunter, Professor in Medicine, The Norwich School of Medicine, University of East Anglia, said:

“This is a valuable summary of the results of the implementation of Liverpool Community testing pilot.  Clearly any intervention like this is primarily of value if it actually reduces the spread of the virus, or in other words reduces the R value in the area where it is being conducted.

“The pilot was able to increase the detection of asymptomatic cases and contacts compared to Manchester, but this is hardly surprising as outside of this pilot very few people would have access to screening for asymptomatic infection.  However the most important conclusion is “At present, there is no clear evidence that that the introduction of MAST led to a change in Covid-19 case incidence or hospital admissions in Liverpool”.  So no clear evidence that this pilot is reducing the incidence of infection.

“If the pilot has not been able to reduce the population incidence of COVID-19 why may this be?

1. The opportunity to have asymptomatic testing was taken up by a little more than 1/3rd of the population.

2. Testing uptake was lower in more deprived areas, where test positivity was higher.

3. The sensitivity of the lateral flow device was lower than expected at 40% and even for cases with higher viral loads (assumed to be more infectious) was only about 2/3rd.

4. As yet compliance with isolation requirements if found to be positive is still be analysed.

“My reading of this report is that the Liverpool pilot was successfully implemented but it did not have much if any impact on the trajectory of the epidemic in Liverpool.  Indeed with the low overall take up rate, the even lower up take in populations with the higher infection rates and the poorer than expected sensitivity of the lateral flow test in use, I doubt that such as scheme as that piloted in Liverpool will have more than a marginal impact on the spread of the COVID-19 in the UK.  Whether such population-wide testing of asymptomatic individuals could be improved to the point that it would have a valuable impact on the epidemic especially with the more infectious new-variant must be uncertain.”

Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:

“MHRA has today granted exceptional use authorisation for NHS Test and Trace COVID-19 Self-Test device.  Which device (presumably INNOVA?); evidence-base for different contexts-of-use; and MHRA’s detailed appraisal.  Meanwhile, salutary and important work from Liverpool.

“Key release today is Liverpool’s quality assurance of its in-context use of the INNOVA lateral flow test (LFT) for screening of asymptomatic citizens.

“Around 6,000 of Liverpool’s asymptomatic citizens provided dual swabs, the first for INNOVA-testing, the second for RT-PCR: 70 asymptomatic citizens were positive for SARS-CoV-2 according to RT-PCR, of whom only 28 were INNOVA-positives so that sensitivity for INNOVA LFT was only 40% (95% CI: 28.5% to 52.4%).  Specificity was high at 99.9% with 99.8% as the lower 95% confidence limit.

“Interestingly, the percentage of those 70 asymptomatic RT-PCR positive swabs which needed fewer than 25 amplification cycles (Ct-value less than 25) for SARS-CoV-2 antigen-recognition was quite high at 39/70 (56%, 95% CI: 44% to 67%).  The INNOVA test picked up 26 of these 39: two-thirds but with wide uncertainty (95% CI: 50% to 81%).

“Another take-home message from Liverpool’s Mass Asymptomatic Serial Testing (MAST) is that, overall, only 25% of Liverpool’s citizens attended for asymptomatic screening.  But uptake and asymptomatic prevalence were negatively correlated.  The report does not mention how many 1st-attenders were re-screened within 1 week of their 1st attendance for screening – perhaps because the answer is glum.

“Liverpool’s mass screening did include a specific focus on secondary schools (uptake ~54%) but, unfortunately, the quality assurance scheme did not include these secondary schools.  However, many universities were persuaded to adopt the INNOVA lateral flow test for screening their students prior to the Christmas vacation.  And were surprised by the low numbers of students who tested positive.  Birmingham University, top of the class, did its own quality-assurance (as explained in this podcast: https://www.bbc.co.uk/sounds/play/p0927j62): it found 2 INNOVA-positives in around 7,000 asymptomatic students who came forward for screening but Birmingham offered additional PCR-testing to 1 in 10 of the INNOVA-negatives.  Six PCR-positives were found in this 10% sample which implies of the order of 22 to 130 (95% confidence interval) LFT-missed positives.  Birmingham University’s student-data are a clear warning of the need for a well-designed quality assurance evaluation in 25 to 50 large secondary schools – to be run and reported publicly – before the INNOVA lateral flow test is steam-rollered to all secondary schools in England.

“Meanwhile, although it will take two or three days to clear some 3,000 lorries from Dover (sufficient time for well-organized, prioritized PCR-testing for stranded lorry drivers), UK has opted to deploy a lateral flow test, probably INNOVA.  Liverpool’s evaluation warns that INNOVA-testing would miss 3 out of 5 SARS-CoV-2 antigen positives.  Detected LFT-positives will almost surely require a PCR test to know whether they are likely to have been infected by the new Variant of Concern.

“Drivers returning to the UK are likely to want Rt-PCR testing for themselves: to safeguard their family and home-region.”

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

Dr Angela Raffle: “I deliver education and training for the UK National Screening Programmes.”

None others received.