select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
Here is a comment from Professor Sheila Bird on lateral flow tests and evaluation in case useful.
Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:
Heterogeneity of antigen lateral flow devices (LFDs) made available to UK public for self-testing:
“At least three antigen lateral flow devices (LFDs) have been made available to the UK publics for self-testing. First was INNOVA/DHSC. Over the past three months, at least two other LFDs have been distributed to the public for self-testing.
“How citizens use these LFD-kits (INNOVA/DHSC, Orient-Gene, Flowflex) in practice deviates from the originally recommended twice-weekly self-testing as citizens may choose to self-test less frequently or selectively, such as on pre-risk or post-risk bases. In addition, the request that citizens should register every LFD-negative test-result with NHS Test & Trace is seen as intrusively burdensome since no follow-up actions by NHS Test & Trace are needed. The public has exercised common-sense and focuses its reporting on LFD-positive test results. The innovation of AI-assisted reading of photographed LFDs test-results offers reassurance that may encourage more comprehensive reporting. Time will tell.
“Unlike INNOVA/DHSC, the more recently distributed kit-types require the swabbing of nares only so that regular self-testing may be more attractive. But an attractive test – were it to perform poorly in terms of its sensitivity in asymptomatic testing – is not necessarily a win.
“I have failed to locate any published evaluation, whether by dual testing or otherwise, of the sensitivity of FLOWFlex for asymptomatic testing of UK citizens. The “Information for Users” insert in the FLOWflex pack provides data about FLOWflex’s sensitivity when was used for testing persons with symptoms – a different context from asymptomatic self-testing; and advises that test-results in be discussed with your doctor. See Background for brief description of published pre-Omicron evaluations, based on dual testing against Polymerase Chain Reaction (PCR), for the other two LFD-types1 2.
“NHS Test & Trace does not publish the results of PCR-adjudication for LFD-positive test results by LFD-type. UKHSA’s Technical Briefing 31(10 Dec, p38) reported perfunctorily: “UKHSA have performed an initial laboratory validation of the current lateral flow devices for COVID-19 in current use by NHS Test and Trace. Preliminary data assess these devices as effective at detection Omicron as Delta”. Neither study-design nor results were reported: hence, take nothing on trust.
“On 11 December 2021, as I mused that: “UKHSA must publish study-design and robust data on FLOWflex’s sensitivity for asymptomatic testing of UK citizens; or acknowledge the absence of such data”, an alerting and exciting email arrived from a scientist-friend (AA).
“AA had ordered a 7-pack of LFDs. The pack had just arrived by post: it was FLOWflex. Her pack included an additional PCR-test. She was invited to take part in a dual-testing evaluation of LFDs versus PCR. At her earliest LFD-test opportunity, she was invited to administer the PCR-swab while she awaited the LFD-result. Note: if she did not wish to opt-in to the dual-testing evaluation, AA could make use of the PCR-swab if she needed it for adjudication of an LFD-positive test-result.
“Several interesting questions arise. For example, I am unclear if those who order a postal 7-pack of LFDs are randomized to LFD-type? Perhaps only a random-selection of citizens who order postal-delivery of LFD-pack is issued with a dual-testing-evaluation 7-pack which also includes PCR-swab and their invitation to take part in the ongoing evaluation. The invitation explains that the evaluation aims to assess sensitivity in different context of self-use. Different contexts could be by occupation, age-group, index of multiple deprivation, gender, ethnicity.
“My own FLOWflex 7-pack, acquired on 11 December from a local pharmacy, asks me to “Report all your results to the NHS”; but is not an evaluation-pack. An Orient-Gene 7-pack, acquired yesterday by a family-member from a pharmacy in Scotland, is also not an evaluation-pack.
“Assume several evaluation-strata. If dual testing within each stratum (e.g. occupational use) and per LFD-type is based on a target of 100 or 200 asymptomatic PCR-positives per LFD-type, then each 95% confidence interval – assuming around 50% sensitivity – would have an expected width of 10 or 7 percentage points respectively. The former, per LFD and stratum, is sufficient for 80% power to discern a target-difference of 20 percentage-points in sensitivity whereas the latter affords just over 50% power to discern a target-difference of 10 percentage-points in sensitivity.
“Clearly, we’d prefer a user-friendly LFD with 70% sensitivity for asymptomatic testing than one with 50% or 30% sensitivity. But do we have such a contender? But, we’d also prefer 60% to 50% so that if comparisons were based on 400 PCR-positives per LFD-type [either within or across evaluation-strata], we’d have 80% power to discern a target-difference of 10 percentage points between LFD-types.
“Full details of the ongoing dual-testing evaluation are not to hand but, assuming some element of randomization, and clear opt-in guidance to citizens, this evaluation-initiative in the Omicron-era is a major step forward.”
Background
By late December 2020, an academic team in Liverpool published that they had consented around 6,000 asymptomatic citizens for dual testing [INNOVA/Polymerase Chain Reaction (PCR)], who voluntarily attended Operation Moonshot during November 2020 for asymptomatic screening1. Thereby, the sensitivity of INNOVA/DHSC against PCR for asymptomatic testing (pre-Delta) was evaluated as 40% (95% CI: 28% to 52%) since only 28 out of 70 PCR-positive asymptomatic citizens were INNOVA-positive. Sensitivity was better (around 67%) for those who were not only PCR-positive but had cycle threshold value (ct-value) less than 25.3, which is a proxy for viral load of at least 100,000.
Sensitivity of the Orient-Gene test for asymptomatic testing was evaluated, again by dual testing, in secondary pupils during the transition-to-Delta-era in an important school-based cluster randomized trial during April to June 20212. Based on 32/60 PCR-positives in dual testing that were LFD-positive also, Orient Gene’s sensitivity was estimated at 53% (95% CI: 40% to 66%).
Overlapping 95% confidence intervals (as above) but higher central estimate for Orient-Gene’s sensitivity for asymptomatic self-testing by secondary pupils in the Delta-transition era versus for INNOVA/DHSC in asymptomatic adults in Liverpool’s Moonshot pre-Delta give only modest support for considering that Orient-Gene is an improvement on INNOVA/DHSC.
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Sheila Bird: “SMB is a member of the Royal Statistical Society’s Working Group on Diagnostic Tests, of RSS’s COVID-19 Taskforce and of UKHSA’s Testing Initiatives Evaluation Board. In 2003, SMB chaired RSS’s Working Party on Performance Monitoring in the Public Services. SMB is hugely grateful to her female-scientist-friend for timely email.”