select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A new analysis from the Department of Health and Social Care suggests that lateral flow tests (LFDs) have a specificity of at least 99.9% when used to test in the community, and could be as high as 99.97%.
Prof Jon Deeks, Professor of Biostatistics and head of the Biostatistics, Evidence Synthesis and Test Evaluation Research Group, University of Birmingham, said:
“Unfortunately the main conclusion of this DHSC report is not supported by the data they present, and it cannot provide reassurance that there is a low risk of positive LFT results in children at school being false positives. When disease prevalence drops, even the most specific tests face problems as the number of false positives can outnumber the number of true positives – a scenario where a confirmatory test is absolutely essential to prevent large numbers being made to unnecessarily isolate.
“The scenarios considered in this report are based on data where the prevalence of infection was high (8.7%), but are extrapolated down to a lower prevalence (0.5%). In this situation the report predicts that around half of test positives will be false positives – which would indicate that half of the children, teachers, families and their bubbles being asked to isolate this week are doing so unnecessarily. If this were true, the report’s conclusion that confirmation with PCR is not needed is bizarre.
“However, the reality is likely to be even worse. In recent weeks, test positivity rates in schools (as reported by test and trace) have been as low as 0.07% (only 1 case in 1500 tested) and have never been higher than 0.32%. This is a real concern, as these rates are about one quarter of the rates from using the test in adults, raising concerns that the performance of the tests may be compromised in teenagers. But they also mean that the prevalence is likely to be much lower than 0.5%, and that we would expect far more false positives than true positives amongst those testing positive in schools.
“I have reported on the likely performance of these tests elsewhere today (https://unherd.com/thepost/why-the-school-testing-regime-needs-to-change/) – there are many uncertainties but given the DHSC data it seems likely that over 70% of positive test results are false positives, potentially many more. Addition of a confirmatory PCR would add little cost (it will be required in less than 1 in 500 cases), and would most likely reduce false positives to 1 in 1,000,000. There is a small decrement in the numbers of cases detected as the PCR does not have 100% sensitivity.
“A confirmatory PCR would also provide much needed evidence on the performance of LFTs in children (for which there currently are no data at all), investigate the incredibly low test positivity rate we are observing, and allow tracking of the new variants. It is also a key recommendation in the WHO implementation guidelines.
“Given this analysis and the DHSC’s own data, the refusal to confirm LFT results with PCR is at best perplexing, will make testing less attractive, and create harm by wrongly isolating individuals, families and other close contacts.”
Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:
“NHS Test & Trace and Public Health England have today [10 March 2021] attempted to justify not offering PCR-adjudication during school-monitored use of LFDs for asymptomatic screening of secondary pupils on their return to school in England. Parents and pupils have been asked to give consent for three LFD tests in two weeks. The Royal Statistical Society called for transparency about the data held by NHS Test and Trace on PCR-adjudications of secondary pupils’ LFD-positive results; and for their distribution of ct-values as proxy for viral load, see https://rss.org.uk/RSS/media/File-library/News/2021/RSS-statement-on-surveillance-in-schools-5-March-2021.pdf.
“Neither is presented. On page 9, the authors reveal that confirmatory PCRs for LFD tests were quietly suspended with effect from 27 January 2021. For which, justification is offered on 10 March 2021.
“Context of LFD use matters. The context that matters here is both asymptomatic screening and secondary-age pupils.
“The authors’ Table 1 pools LFD/PCR at three sites where symptomatic persons attended for PCR-diagnosis (identifiable because over 20% of PCR-tests were positive) versus a single site, Liverpool, where citizens attended for asymptomatic screening where only 1.6% of those who kindly agreed to be dually-swabbed were PCR-positive. Table 1 fails to identify how many (if any) of the Liverpool contingent were secondary-school age. Only the Liverpool results contribute to the question of whether asymptomatic LFD-positives require PCR-adjudication.
“Table 2 addresses itself to specificity but, for Liverpool’s community testing, fails to remind readers that out of 31 LFD positives, three were PCR-negative (10%, wide 95% confidence interval). That’s the primary reason for PCR-adjudication of LFD-use for asymptomatic screening.
“The authors’ appropriation of Table 2 is to ask what percentage of those who were PCR-negative tested LFD positive. That question is irrelevant for the roll-out of asymptomatic screening in secondary schools as pupils are not being PCR-tested.
The final section is robustly empirical – or could be. From 20 November 2020 to 27 January 2021, 38,270 positive LFDs (2.2% of LFDs performed) were registered by NHS Test & Trace. Two-thirds were matched to an adjudication PCR-test within the subsequent 5 days. We are not told how many were matched to an adjudication PCR-test taken on the same-day or next-day after LFD, that is: if the calliper was tighter. {Authors did sensitivity analyses with shorter callipers: results not shown}.
“Data are not presented for asymptomatic secondary-age pupils, for whom there seemed to be 626 LFD-positives before 27 January 2021 according to NHS Test and Trace versus 1,386 for secondary school staff.
“Cryptically, the final paragraph of this report refers to School/College LFD-testing but authors fail to admit that staff LFD-positives in 2021 outnumbered those for pupils by more than 2:1 (say 70%). Five-day calliper matching to PCR results was achieved for 1,948/2,332 (84%) of School/College LFT-positives: 20% (372/1,948) were not PCR-positive. If we assume that staff accounted for 70% of 1,948 matched School/College LFTs and that the authors’ “mainly-adult” PCR-negative rate of 6.3% applied , then staff would account for 6.3% * 1364 = 89 of these 372 PCR-negatives, leaving 286 to pertain to 584 matched secondary pupils. In other words, 49% of secondary pupils’ LFD-positives (289/584) may have been not PCR-positive.
“Why – despite the call for transparency by the Royal Statistical Society’s COVID-19 Taskforce – did the authors fail to present the only data that NHS Test & Trace has on secondary pupils’ PCR-confirmations of asymptomatic LFT-positives?
More than 600 INNOVA-positives can be expected this week when over 1.5 million asymptomatic secondary pupils have been screened. Half may be PCR-negative. By withholding data and denying PCR-confirmations, the half who are PCR-negative will never know – unless families and schools demand fairness for our children.”
Dr Alexander Edwards, Associate Professor in Biomedical Technology, Reading School of Pharmacy, University of Reading, said:
“This type of analysis is vital but still doesn’t address the unknown question of how effective lateral flow rapid testing will be in reducing spread. The numbers can quickly get quite hard to make sense of, and this report is full of calculations and important data that take time to check and analyse. It would be ideal to see a significant number of peer-reviewed, independent, field studies but that may take significant time. In the interim we have to do the best we can as decisions need to be made now and testing programs are still essential.
“The headline suggesting specificity must be at least 99.9% is reassuring as it suggests false positives are rarer than initial smaller studies could show. This corresponds to less than 1 in 1000 false positives. As far as I can understand from quickly checking recent NHS Test & Trace weekly reports, in three consecutive weeks (4-24 Feb) fewer than 1 in 1000 positive LFD tests were reported overall in secondary schools and colleges (e.g. week ending 24 Feb 189 positives out of 288,958 tests). It is impossible to get fewer overall positive results than false positives, so even if every single one of these positive LFD tests was a false positive, the specificity of these tests must be 99.9% or more.
“How we manage the impact of any false positives, as well as false negatives (where cases are missed) remains a big challenge. Although there has been a lot of discussion and argument about accuracy of testing, it remains one of the most vital tools to tackle spread of infection.”
‘Lateral flow device specificity in phase 4 (post marketing) surveillance’ by Achim Wolf, Jack Hulmes and Susan Hopkins was published by the Department of Health and Social Care
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Jon Deeks: “Jon Deeks is Professor of Biostatistics at the University of Birmingham and fully funded by the University of Birmingham. He leads the international Cochrane COVID-19 Diagnostic Test Accuracy Reviews team summarising the evidence of the accuracy of tests for Covid-19; he is a member of the Royal Statistical Society (RSS) Covid-19 taskforce steering group, and co-chair of the RSS Diagnostic Test Advisory Group; he is a consultant adviser to the WHO Essential Diagnostic List; and he receives payment from the BMJ as their Chief Statistical advisor.”
Prof Sheila Bird: “SMB chaired the Royal Statistical Society’s Working Party on Performance Monitoring in the Public Services, serves on the RSS’s COVID-19 Taskforce and chairs the RSS/DHSC Panel on Test and Trace. Since mid-January, SMB also serves on the Testing Initiatives Evaluation Board.”
None others received.