select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
An updated Cochrane Review, published in the Cochrane Database of Systematic Reviews, assesses the use of rapid, point-of-care antigen and molecular-based tests for the diagnosis of SARS-CoV-2 infection.
This Roundup accompanies an SMC Briefing.
Prof Irene Petersen, Professor of Epidemiology and Health Informatics, UCL, said:
“I have concerns about the methodology applied in this review. The key issue is that it is using PCR as ‘reference-standard’ when evaluating the sensitivity of rapid antigen tests. Unfortunately, the PCR tests have low specificity in terms of identifying individuals with live virus among people without symptoms. Therefore, the review cannot provide a reliable answer to the actual sensitivity of rapid antigen tests in terms of identification of individuals with SARS-CoV-2 infection.
“The PCR test is based on identification of viral genetic material (RNA). However, it is well recognised that many people have viral genetic material in their body for several weeks after they were infectious with SARS-CoV-21. Thus, when using PCR tests in an asymptomatic population it is likely that more than half of the PCR positive results are from individuals who are post-infectious (no longer infectious).
“When comparing the performance of rapid antigen test and PCR tests it may appear that the rapid tests have a low relative sensitivity. In reality, it is the other way round i.e. the PCR tests have a low specificity to identify individuals with current infections among people without symptoms. For example, if over half of the individuals no longer have live virus in their body a validation study using PCR test as the reference standard can never reach an apparent relative sensitivity of the rapid tests of more than 50%. On the other hand, in studies which include symptomatic individuals the proportion of individuals with live virus in their body is likely to be much higher and thus we would expect the apparent relative sensitivity to be higher, which is exactly what was observed in the validation studies in Denmark, Spain and US3-5. The proportion of individuals with live virus in a sample also varies over time and across locations depending on whether the epidemic is growing or shrinking. Thus, we may find the apparent relative sensitivity of the same type of rapid tests also varies substantially between studies carried out in different location and at different times. Hence, a large variation of the apparent relative sensitivity has been observed in empirical studies carried out so far3-6.
“With knowledge of the biology of the virus and information about the local developments of the pandemic it is really important that we do not take these ‘validation’ studies at face value. Instead we have to recalibrate these to provide realistic estimates of the rapid antigen tests ability to identify individuals with live virus. For example, a crude recalibration of the apparent relative sensitivity of 40% found in the Liverpool pilot study suggests that the actual sensitivity is likely to be above 80% and probably higher.”
References
Dr Thomas House, Reader in Mathematical Statistics, University of Manchester; Dr Elizabeth Fearon, Assistant Professor in Epidemiology, London School of Hygiene & Tropical Medicine; and Martyn Fyles (University of Manchester, Alan Turing Institute); said:
“This appears to be a high-quality review of the evidence on the performance of rapid point of care tests for SARS-CoV2 infection. We note that the WHO guidance the report references is nuanced as to the criteria required of tests (WHO 2020c in the Cochrane report): “It is anticipated that emerging diagnostic products will not necessarily meet all criteria outlined in the TPP […] meeting several desirable criteria may outweigh failure to meet multiple acceptable characteristics”.
“Antigen tests do have desirable properties for deployment in pandemic response: they are cheap; they can be self-administered and run without specialist equipment; the results are delivered quickly; and their sensitivity is highest at high viral loads [1]. While there are difficulties that the report notes in converting CT values to precise viral loads and in defining a threshold viral load or CT value for transmissibility, lower CT values at time of testing have been associated with increases in the attack rate amongst a case’s contacts [2, 3]. As the report notes, antigen tests were 94.5% sensitive in those with Ct<25, indicating higher viral loads. Further, the quoted WHO guidance states that “It is considered acceptable to target patients with high viral loads often present in the first week following infection7,8,9,10 because they are most likely to transmit the infection to others”. This makes antigen tests potentially suitable for deployment in a SMART manner to break chains of transmission [4], with consideration as to the purpose of the testing, the population and the timing of their possible exposure, the details of its regime (frequency and duration of testing where repeated) and the broader intervention in which testing is embedded.
“The experience of Liverpool did provide evidence for the utility of these tests in the real world [4, 5]. Nevertheless, we do agree that given our level of uncertainty, studies of various designs, potentially including formal control trials, of different testing policies would be helpful to our understanding as to the effectiveness of different testing and isolation/quarantine interventions in different settings.”
References:
[1] Comparative performance of SARS CoV-2 lateral flow antigen tests demonstrates their utility for high sensitivity detection of infectious virus in clinical specimens. Suzanne Pickering, Rahul Batra, Luke B. Snell, Blair Merrick, Gaia Nebbia, Sam Douthwaite, Amita Patel, Mark Tan Kia Ik, Bindi Patel, Themoula Charalampous, Adela Alcolea-Medina, Maria Jose Lista, Penelope R. Cliff, Emma Cunningham, Jane Mullen, Katie J. Doores, Jonathan D. Edgeworth, Michael H. Malim, Stuart J.D. Neil, Rui Pedro Galão
medRxiv 2021.02.27.21252427; doi: https://doi.org/10.1101/2021.02.27.21252427
[2] An observational study of SARS-CoV-2infectivity by viral load and demographic factors and the utility lateral flow devices to prevent transmission. Lee, LYW et al. 2021. http://modmedmicro.nsms.ox.ac.uk/wp-content/uploads/2021/01/infectivity_manuscript_20210119_merged.pdf
[3] Marks M et al. Transmission of COVID-19 in 282 clusters in Catalonia, Spain: a cohort study. 2021. Lancet Infectious Diseases, published online ahead of print, 2 February, 2021. https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(20)30985-3/fulltext
[4] Liverpool Covid-SMART Pilot: Systematic Meaningful Asymptomatic Repeated Testing Addressing SARS-CoV-2 transmission and harms from Covid-19 restrictions, as one system 10th December 2020 for SAGE Covid 72 Buchan, Semple et al. https://www.gov.uk/government/publications/liverpool-covid-19-community-testing-pilot-interim-evaluation-report-summary/liverpool-covid-19-community-testing-pilot-interim-evaluation-report-summary
[5] Crozier A, Rajan S, Buchan I, McKee M. Put to the test: use of rapid testing technologies for covid-19 BMJ 2021; 372 :n208 doi:10.1136/bmj.n208
Prof Louise Kenny, Executive Pro-Vice-Chancellor, Faculty of Health & Life Sciences, University of Liverpool, said:
“This is a large Cochrane review of rapid, point of care antigen and molecular based tests for SARS-CoV-2 infection.
“The authors collected and analysed data from studies published up to 16th November 2020. Inevitably, in this fast moving field, the review is already out of date. The very small number of studies (12) and samples (1581) in asymptomatic cohorts and the exclusion of many of the largest recently reported and ongoing studies such as Liverpool, limit the applicability of the findings.
“Nevertheless, the review offers supportive evidence for the clinical utility of rapid point of care testing for SARS-CoV-2. Of note, the authors flag that antigen tests vary in sensitivity and some perform better than others. The best performing tests are as effective as PCR testing and have the added critical advantage of speed. Ongoing studies will add further evidence of the utility of repeat rapid antigen testing and testing in asymptomatic cohorts.”
Dr Jeremy Rossman, Senior Lecturer in Virology, University of Kent, said:
“Testing is one of the most important tools we have to control SARS-CoV-2 transmission. The recent Cochrane systematic review examines the literature on rapid antigen tests in order to better understand test accuracy. This is a very important analysis; however, the conclusions are limited by the focus on comparing antigen test accuracy to that of PCR tests. The high sensitivity of PCR tests makes them ideal for clinical diagnosis, but neither PCR positivity nor the presence of symptoms perfectly correlate with infectivity. Controlling the spread of SARS-CoV-2 requires rapidly identifying and isolating infectious individuals. In this context, the speed and frequency of testing has a greater impact on transmission than does test accuracy. Because of this, we need to evaluate surveillance and clinical diagnostic tests using different metrics instead of comparing all testing modalities to the ‘gold standard’ PCR test.”
Prof Richard Tedder, Senior Research Investigator in Medical Virology, Imperial College London, said:
“This is an extremely valuable analysis of global data published up to 30th September 2020 relating to the use of lateral flow and other point-of-care tests for the detection of infection in humans by SARS COV 2. It includes the multiple evaluations of 58 antigen tests and a smaller number of molecular tests.
“The estimates of sensitivity varied greatly between point of care antigen tests and, as one might anticipate, sensitivity was highest for antigen detection in the first week after symptom onset with an overall mean detection rate of 72% (ranging from 34% to 88%). Detection correlated with the diagnostic CT values (i.e. the amount of virus), but with the detection rate falling away after the first week. Overall specificity was higher ranging between 99% and 99.8% in both uninfected asymptomatic patients and those uninfected patients with symptoms from other causes.
“Given the WHO desired “acceptable” sensitivity of better than 80% and specificity of 97%, the authors suggest it is possible to consider the best of these assays as an alternative to the conventional molecular PCR-based when immediate decisions about patient care have to be made and also where RT-PCR cannot deliver the need for viral diagnosis in the time course required. The authors make the point that due to the variable sensitivity of antigen tests people whose test is negative may still be infected.”
Dr Alexander Edwards, Associate Professor in Biomedical Technology, Reading School of Pharmacy, University of Reading, said:
“A major foundation for modern medicine is systematic reviews that collect and interpret multiple studies of changes to clinical practice – hence the vital importance of this Cochrane review. This huge and excellent body of work surveys a collection of studies of many products including many lateral flow antigen tests that detect virus components in a swab or similar sample, to diagnose COVID-19. Overall this 400-page study collects together a clear picture of the performance of current products, including many thousands of participants. What is at first sight surprising is that in spite of the scale of trials reported (and there are even more trials out there now), we are still missing some important information about COVID-19 rapid tests.
“Testing remains vital, yet we don’t yet know the best ways to use tests to slow the spread of COVID-19. Diagnostic testing at first sight might seem simple – surely all we need to know is “is the test accurate” and get testing? But for effective real-world use of tests, especially when screening large populations, a lot of additional factors must be considered. Many trials and studies use test products in a different way to the manufacturer’s instructions. There is no fundamental problem with this – of course you must adapt products to your needs, but re-validation becomes essential after modification.
“Why do we need community screening with rapid tests? By the time people have symptoms, they may already have spread the virus. There are massive potential benefits of regular community screening. Why is it difficult? Most tests were designed and tested for trained experts to test ill people. Better instructions and easier-to-use products are needed, and we need full evaluation in clinical trials of the improved products in asymptomatic people – which is slow, expensive and hard. A great deal of effort and investment has already been focussed on overcoming these challenges, but it takes time to publish and so the most recent studies won’t be included in the Cochrane review. This is not a criticism – it is essential that time is taken to review data thoroughly.
“The current urgent need to re-open means we have not waited for better data to be fully published and systematically reviewed, and so rapid testing is now widespread across the UK. We eagerly await publication and independent discussion of the full evaluation data. More studies will emerge that will help us to understand how effective community rapid testing really is at slowing spread of COVID-19. The testing program may need adjusting, depending on what emerges. It follows that we must get better at evaluating diagnostic testing, especially for community testing programs. Although the UK does have significant expertise in evaluating diagnostics, more investment is essential so we can make good use of exciting new diagnostic technology.”
Dr Angela Raffle, Honorary Senior Lecturer, University of Bristol Medical School, Population Health Sciences, and Consultant to the UK National Screening Programmes, said:
“This Cochrane systematic review examines the quality of, and findings from, 68 relevant study reports on performance of rapid tests for SARS-CoV-2. Cochrane reviews are the best available in terms of meticulous scientific rigour, comprehensiveness, international cooperation, transparency, clarity of presentation and freedom from political or commercial influence. This review meets these high standards, and the press statement accurately matches the findings of the report. This review is a welcome contrast to the many poor quality ‘feel good’ articles in the press about rapid testing. The Cochrane review brings a much needed real world overview, and the reviewers will continue to update their analysis to include new studies recently reported. Their first systematic review on this topic was in August 2020.
“The review reveals some tests perform far better than others, and demonstrates that sound evaluation, in the settings in which the tests will be used, is an essential prerequisite of major decisions on testing policy and procurement.
“I hope this review will prompt a better scientific approach to policy on testing in the pandemic. We absolutely need rapid tests, and their development is welcome. But the tendency to view testing as a black box has not been helpful. We would not advocate “treat, treat, treat” or “vaccinate, vaccinate, vaccinate” without instigating rapid studies to work out what treatment or vaccine and in which patients. The Cochrane review reveals that only 295 of the cases (and 1,581 of the samples) in the 58 reported evaluations examined were actually people without symptoms, and none of the studies examined self testing or repeated lateral flow testing. Yet the UK government decided last August to devote massive resources to routine indiscriminate lateral flow testing for low risk symptomless people, in the absence of any empirical evidence that this is a worthwhile venture and without any quality assurance of how the tests are taken and read.
“Testing programmes are massive undertakings, and require the same scientific rigour that we are so proud of when it comes to treatments and vaccines. The Cochrane review is a step along this path. We now need well designed field trials to see how best to implement effective, locally based, test and trace systems that use the advantages of the best performing rapid tests backed up where necessary by laboratory PCR. The focus should be rapid identification of cases no matter how minor the symptoms; testing for infection in contacts – both backwards i.e. who did the case catch it from, and forwards i.e. who have they passed it on to; exemplary outbreak management; and testing of those who need to be in contact with vulnerable people. If we get this right then testing of the whole of society becomes a wasteful irrelevance. Far better early warning systems can be achieved through measures such as waste water testing.”
Prof Paul Hunter, Professor in Medicine, The Norwich School of Medicine, University of East Anglia, said:
“This Cochrane Review is a very detailed and well conducted systematic review of rapid methods for detection the diagnosis of COVID. This review includes analysis of the lateral flow Antigen tests like the ones currently being rolled out in the UK. There has been quite a lot of scientific debate about the value of these tests in either symptomatic or asymptomatic screening. The authors of this review undertook a number of analyses in order to identify the sensitivity and specificity these rapid tests and in different contexts and particularly their use for diagnosis of symptomatic and asymptomatic individuals and also the accuracy of the tests in the hands of trained laboratory scientists and when done by people who were not so trained.
“In general their conclusions were that the reported sensitivity (percentage of true positives detected by the test) varied substantially from one study to another and from one product to another. Specificity (percentage of positive results that are truly positive) was quite high in most tests and generally above 98%. The tests as a whole performed better in the diagnosis of people with symptoms suggestive of COVID than in asymptomatic people. In those few studies were comparisons were possible, the tests had rather higher sensitivity when undertaken by trained laboratory scientists than when undertaken by other health care workers which was also higher than tests undertaken by self-trained non-health care workers. There was a considerable variation in the sensitivity of tests from different manufacturers though some of that variation was likely to be due to study methods rather than variation in the accuracy of the tests themselves.
“In conclusion the authors considered these rapid tests have good value for diagnosis of infection in symptomatic individuals within one week of onset of symptoms or when the Ct value is low (usually the first week after onset). But the authors much more sceptical about the value of these methods for screening asymptomatic people especially when the risk of being positive becomes quite low. The point out that using the most sensitive tests when the prevalence of true positives in the asymptomatic population is 0.5% (which is greater than currently the case in England) between 7 and 9 out of every 10 positives will be false positives and between 1 in 2 and 1 in 3 true cases will be missed. Given that testing undertaken by self-trained people is generally less accurate these estimates will probably be poorer with self-administered tests.
“So overall these tests are likely to be valuable for rapid diagnosis of people who have symptoms consistent with a diagnosis of COVID, if undertaken in the first week of illness or in settings where the probability of infection is high. However, their value in screening asymptomatic individuals, especially when the real prevalence of the infection in the community is low and when those tests are performed by self-trained individuals at home, is still uncertain.”
‘Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection (Review)’ by Dinnes J et al. is published in the Cochrane Database of Systematic Reviews on Wednesday 24 March 2021.
DOI: 10.1002/14651858.CD013705.pub2
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013705.pub2/full
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Irene Petersen: “I have no conflicts of interest.”
Dr Thomas House, Dr Elizabeth Fearon, and Martyn Fyles: “We’re funded by UKRI to model Test, Trace and Isolate but I don’t think this counts as a material conflict of interest.”
Prof Louise Kenny: “None.”
Dr Jeremy Rossman: “I do not have any conflicts of interest to declare.”
Prof Richard Tedder: “None that I’m aware of.”
Dr Alexander Edwards: “I am a shareholder and founder of diagnostic technology company, but this does not sell COVID-19 tests.”
Dr Angela Raffle: “I work as a Consultant to the UK National Screening Programmes and I am lead author of an international textbook on screening.”
None others received.