select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A paper prepared by scientists at the University of Liverpool and published by SAGE presents preliminary data on the accuracy of the Innova Lateral Flow SARS-CoV-2 Antigen test used in the Liverpool pilot.
Prof Richard Tedder, Senior Research Investigator in Medical Virology, Imperial College London, said:
“The data presented on the performance of the Innova lateral flow assay for detecting SARS-CoV-2 clearly indicate in this study that when compared against a benchmark molecular test such as a quantitative PCR the lateral flow test is very very much less sensitive, detecting only about half of the known positive samples subjected to testing. Indeed it even failed to detect two out of five samples with the highest level of viral gene copies, these displayed a PCR CT of < 20 (i.e. were samples which could be diluted one million fold and still give a positive PCR signal). This graphically shows that this assay is simply not sensitive enough to use to test persons with a view to confirming an absence of infection and thereby an absence of infectivity. A negative result with this lateral flow test simply does not infer an absence of infection.”
Dr Jason Oke, Senior Statistician at the Nuffield Department of Primary Care Health Sciences, University of Oxford, said:
“This reports provides preliminary results from a field evaluation of mass testing using the Innova lateral flow antigen test. From these data we can conclude that the lateral flow tests provide reproducible results but tend to miss a significant proportion of PCR positive individuals. When LFT was compared with PCR, a false positive result occurred in just 2 out of 2981 PCR negative individuals (< 0.1%), but LFT missed 23 of the 35 PCR positive individuals (51%). Because there were only 45 PCR positives in total, there is some uncertainty but the sensitivity of the lateral flow test for SARS Cov-2 judged by PCR is likely to be between 34% and 64%.
“Breaking down test results by PCR cycle threshold indicates that the LFTs are more likely to miss individuals with higher cycle threshold values. As infectiousness tends to decrease as the cycle threshold increases, these results suggests that the tests tend to miss individuals with a lower viral load and who are probably less likely to spread the infection. This could potentially represent an advantage over PCR which has been criticised as being overly sensitive.
“The low sensitivity of the lateral flow test suggests that in a low prevalence setting, relatively few patients will be given false reassurance by a negative test result but the implications of this unknown and could be important. There may still be utility in mass testing for ruling in COVID-19 in asymptomatic people but crucially we are no clearer on whether this approach translates to overall lower infection rates in the community and fewer hospitalisations or deaths.”
Dr Alexander Edwards, Associate Professor in Biomedical Technology, Reading School of Pharmacy, University of Reading, said:
“It’s fantastic to see this Quality Assurance data published as it gives us a better understanding of the performance of these Innova rapid swab tests when used in the ‘real world’. It is not fantastic to see that the sensitivity vs RT-PCR drops below 50%. This low accuracy was mentioned in passing from a guidance document1 encouraging the use of this product for mass testing programs. This is also rather different to the product instructions stating sensitivity of 96%.
“This kind of study checking accuracy during use is absolutely vital when using new diagnostic testing products, but also very hard work, and this data set represents over 3000 people being tested twice using rapid test and lab RT-PCR. Not everyone running testing services can do this kind of checking. Sometimes lateral flow tests can be interpreted as positive by one person, but the same test appears negative to another tester. Many universities have reported very low levels of positive results from students using this product – let’s hope this reflects low numbers of actual cases, not just low test sensitivity.
“It must be remembered that not every infected case will be detected by RT-PCR itself, even though this is the “gold standard”. Therefore, if not every test is found using RT-PCR, and only 50% of RT-PCR positive cases can be detected by Innova tests, then the real sensitivity may be even lower than 50%.
“There is an indication that patients with more virus are detected more frequently by the rapid tests – this makes sense because the limitation of “lateral flow” testing technology is that it’s not as good as lab tests at picking out lower amounts of virus target. RT-PCR can pick up really tiny amounts of virus. Some people have argued that people with small amounts of virus are not as likely to infect others than people with high amount of virus. Whilst this is true, it’s also true that swab technique can affect the amount of virus that can be detected, and it’s also known that RT-PCR products themselves have varying sensitivity. Furthermore, someone asymptomatic with low virus load, could well rapidly turn into someone with very high levels of virus in just a few days. Or they may have recovered already. So it’s not yet possible to assume that detecting only those with the highest levels of virus is a safe and effective screening tool.
“Great care must be taken to make safe, efficient, and cost-effective use of all testing tools available. The speed, relative simplicity and relatively low cost of this particular rapid test, may come at the cost of performance. But the more cases that are detected and helped to isolate before they can spread, the better.”
1 “Community Testing a guide for local delivery”: https://www.gov.uk/government/publications/community-testing-explainer/community-testing-a-guide-for-local-delivery#what-the-community-testing-programme-is
Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:
“Liverpool’s in-context quality assurance evaluated the SARS-CoV-2 screening of some 3,000 asymptomatic citizens using nasopharyngeal-swab with INNOVA rapid-test against contemporaneous second swab analysed by the reference RT-PCR antigen (‘have I got it?’) test. Of the initial 3,000 citizen volunteers, about 1.5% were RT-PCR positive.
“First, the SAGE results are interim. Liverpool’s completed quality assurance is based on around 5,000 citizens.
“Second, the void-rate for RT-PCR test was higher – almost surely because the second swab was always assigned for RT-PCR testing.
“Third, at interim analysis, 46 asymptomatic citizens were positive by RT-PCR (only 22 by INNOVA rapid test; one void INNOVA-test).
“For these 46 RT-PCR positive cases, quantitative Ct-value (loosely interpreted as “infectiousness”) was less than 25 (presumably a pre-determined cut-off) for 30/46, one of whom had the void INNOVA test: 20 of these 30 asymptomatic citizens (67%) had a positive INNOVA-test. Only two (13%) were INNOVA-test positive out of 16 asymptomatic RT-PCR positives with Ct-value greater than or equal to 25. Hence, the interim analysis gives persuasive evidence that the INNOVA rapid-test performs better – but not well – when Ct-value is low (i.e. below 25). Moreover, around two-thirds of Liverpool’s asymptomatic RT-PCR positive citizens in this interim analysis had low Ct-values.
“Of course, we need now – urgently – to see the final results of Liverpool’s in-context evaluation on 5,000 asymptomatic citizens, not just these interim findings.
“The INNOVA rapid-test missed more asymptomatic citizens than it detected. This test’s performance – as with any test – may differ according to the context of its deployment.
Hence, if the INNOVA rapid test has been used for the screening of asymptomatic university students or for school-children, or to guide entry to care-homes, then its performance in these three different contexts should be formally checked against reference RT-PCR testing for 5,000 consecutive volunteers (as, responsibly, Liverpool did in its mass screening).”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Dr Alexander Edwards: “I am co-founder and shareholder of a diagnostic technology company, but with no COVID-19 products and not developing COVID-19 swab tests.”
Prof Sheila Bird: “SMB serves on Royal Statistical Society’s COVID-19 Taskforce and chairs its Panel on Testing which liaises with analysts at DHSC and Test & Trace.”
None others received.