June 25, 2020

expert reaction to Cochrane Review of antibody tests for SARS-CoV-2

Cochrane have conducted a review into antibody tests for SARS-CoV-2.

This Roundup accompanied an SMC Briefing 

Dr Alexander Edwards, Associate Professor in Biomedical Technology, Reading School of Pharmacy, University of Reading, said:

“I found the most helpful element of this well-structured systematic review is in organising and clearly laying out the critical questions that must be answered by antibody test trial data.  The painstaking methodology of translating sets of different published data into evidence based medicine and policy is vital to ensure we make best use of the tests and tools that are available.  Systematic reviews are a major part of this, and it’s therefore very important work.

“It’s important to remember that all the new trial data that has been published more recently (for example data sets for Roche and Abbott products that have been widely discussed) is not included in this review but can now be included and assessed using this systematic approach in future updates, which will give us further confidence in how antibody tests can be used.

“There is no doubt that antibody tests are essential, and are already extremely useful.  But what must be kept in mind is that there are many different ways that we already use, and will continue to use, antibody tests- and each use may require slightly different data or different test performance.  For example, we do not yet have enough data to understand how antibody responses relate to the risk of re-infection or transmission- so we can’t use antibody tests for ‘immunity passports’.  In contrast, we do have plenty of data showing how antibody tests can be used to survey the number of people who have already been infected.  We don’t yet know what antibody levels after an experimental vaccine will show that the vaccine works – but without using antibody tests, vaccine clinical trials will be much harder to run and slower.”

Dr Hayley Jones, Senior Lecturer in Medical Statistics, University of Bristol, said:

“With the current proliferation in new antibody tests for SARS-CoV-2, establishment of this ‘living’ (i.e. to be continuously updated) systematic review of test accuracy is an important initiative.  This first version covers study results available until the end of April 2020, largely from China.  Data on the accuracy of 27 different tests were found and tabulated, but the data on each individual test was scarce: in particular, only three tests had estimates of sensitivity (how good the test is at detecting previous SARS-Cov-2 infection among people who were previously infected) based on more than 100 samples.  We need more data than this to be confident about a test’s accuracy.  Because of the lack of data on particular tests, the authors have at this stage focused on combined results across multiple antibody tests.  Going forwards, when the review is updated, it will be important to estimate how different tests compare with each other, as some may be much less accurate than others.

“How accurate these tests “need” to be depends crucially on how they will be used in practice.  As highlighted by experts in the BMJ yesterday (doi: 10.1136/bmj.m2420), whether the presence of antibodies implies immunity and, if so, to what extent, is still unclear.  Even if immunity among individuals with an IgG antibody response is established, we would need to be very confident that a test rarely gives false positive results for it to be suitable for mass testing, if the purpose was “immunity passports” or similar.  In this review, tests were found overall to incorrectly give a positive result in 1-2% of people who have never been infected.  Although this might sound impressive, this could lead to a large number of false positive results if such a test was applied to entire populations, rather than only people who have reason to believe they’ve been previously infected.”

Prof Eleanor Riley, Professor of Immunology and Infectious Disease, University of Edinburgh, said:

“This is a useful addition to the knowledge base around serological assays for COVID-19.  It pulls all the publicly available data together in one place, will be periodically updated as new data emerge and it gives us a baseline to work from.  The analysis only goes up until mid-April and thus only captures data from the early stage of the epidemic; many of the questions we would like answers to cannot yet be answered but regular updates should alleviate this problem.

“However, it is already very clear from this analysis that antibody testing early in the course of disease is unreliable and thus antibody tests cannot (and should not) replace virus detection for diagnosis of acute cases except where the time-course of disease is already well advanced.  This is a useful and important clarification.

“The data also make it very clear that most, if not all, commercially available tests are not sufficiently accurate to warrant their use outside of the healthcare setting.

“The assessment of the quality of the available data, and in particular the recommendations of how data should be presented in future publications, is also very useful and should lead to better quality of data presentation in the future, enabling some of our outstanding questions to be answered.”

‘Antibody tests for identification of current and past infection with SARS-CoV-2 (Review)’ by Jon Deeks et al. was published in the Cochrane Library at 16:00 UK time on Thursday 25 June 2020.

DOI: 10.1002/14651858.CD013652

Declared interests

Dr Alexander Edwards: “I am director and co-founder of a company that develops antibody testing technology, although we are not offering any COVID-19 antibody test products.”

Dr Hayley Jones: “I am also working on antibody test evaluation research.”

Prof Eleanor Riley: “No COI.”