October 5, 2020

expert reaction to file size limit being the cause of the test & trace data problem

It has been reported that a technical issue causing issues in the test and trace system originated from a file size limit being reached. 

Prof Jon Crowcroft FRS FREng, Marconi Professor of Communications Systems in the Computer Laboratory of the University of Cambridge, said:

“There are 66 million people in the UK (about 56 million in England and Wales).  So if we were in very bad shape in terms of positive cases per day, and very good shape in terms of testing working well, we could easily get more than a million positives in a day.  File size is a basic consideration and one would think that a software engineer would have considered a worst case test scenario for any system design (on paper before you even get to writing code) just to make sure it wouldn’t blow up in any way like this.

“If this was Excel as is being reported, the limitations of Excel (which is generally a very decent piece of software) in terms of Big Data are well known – if you look at how many people are expressing astonishment at this online, you can see that. 

“Any decent contract for such a system ought to have acceptance tests – one wonders if there are penalty clauses for not delivering something that met those (if there were any).  There are many big systems in government (e.g. DVLA’s or HMRCs) that work at this scale so there’s no excuse.  Also, a simple sanity check on the data or error checks in the system might have told them when they hit this limit instead of discovering it after the event.  This sort of thing is standard in 6th form or undergraduate computer science training too.”

Dr Peter Bannister, Executive Chair of the Institution of Engineering and Technology Healthcare Sector, said:

“It’s disappointing to read that a lack of awareness around the limitations of a consumer software product may have led to such a negative impact for all those who are relying on the Covid testing programme. It’s widely known within medical device development that the use of commercial off-the-shelf products, such as Excel, requires additional testing to ensure that they are able to meet the stringent requirements of use in a healthcare setting. In particular, regulatory authorities such as the FDA in the United States have called out the limitations of Excel when applied to storing and manipulating medical data and furthermore have issued guidance on validation and risk management for these products if they are to be used in such a safety-critical manner.”

https://www.gov.uk/government/news/phe-statement-on-delayed-reporting-of-covid-19-cases

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

None received.