July 31, 2019

expert reaction to study on a deep learning approach to predicting acute kidney injury

Research, published in Nature, reports that an AI system is able to predict acute kidney injury up to 48 hours before it occurs in patients.

Dr Franz Kiraly, UCL and Turing Fellow in the Department for Statistical Science, Alan Turing Institute, said:

“To discuss this paper, we need to focus on two questions:

• are modern AI algorithms capable of early warning of acute kidney failure in a clinical hospital setting at scale?
• are the DeepMind style deep neural networks the best choice to do so?

“The paper provides some evidence that AI algorithms are capable of early warnings, and not that the DeepMind algorithms are necessarily the best choice to do so. Deep learning, or neural network algorithms, have disadvantages including that they are very resource demanding (they need AI experts, hardware, and time to run the algorithm), it is difficult to understand why a recommendation by an algorithm is made and they can be prone to failure outside of the range of data they were trained upon. The alternatives to deep learning algorithms are simpler approaches, such as systems involving logistic regression models, which are fast, readily available and easily interpretable and well-understood by doctors. From a healthcare perspective, it may be preferable to use a logistic regression model to a deep learning algorithm as long as they were similarly successful in predicting an outcome correctly.

“In the paper, the evaluation and reporting of results is very precise, to the level that it notably exceeds the quality of most AI related research appearing in top tier journals. The results are therefore very credible, although it is not possible to independently reproduce them as there is no link to the code. The work fits well with findings from previous studies of AI applications and electronic health records in that clinically relevant predictions seem possible. The algorithm also highlights patient features as important which were known before to clinicians as important (though clinicians are sometimes quick to come up with interpretations of claimed associations). However, there are some limitations and because of these I would argue that the study only provides evidence that AI is applicable to this scenario in general (which is in itself a major research achievement!), but not that the DeepMind deep learning algorithms are necessarily the only, or best, way to do this. One of the limitations of the study is a lack of descriptive detail about the structure of the neural network, particularly when it is said to be difference from the standard structure, and there is also a lack of description for the simpler competitor models.

“In-principle this application is ready for testing in a clinical setting. From a scientific viewpoint, there should be a clinical trial conducted, which would ideally be randomised and double-blinded. The clinical trial would look at whether patients treated by doctors with access to the system’s recommendations have better clinical outcomes (a chance of survival) than those treated by doctors without access to the algorithm. The current study only shows (subject to the caveats) that the algorithm can predict problematic episodes in patients – this could be subject to subtle errors in the algorithm, or due to other factors, so the findings need to be replicated in a clinical trial before we assume it works. An algorithm making accurate predictions should not be confused with these predictions improving patient outcomes, we cannot tell this without a clinical trial.

“If the algorithm is solid – and that is a plausible possibility – it could proceed to direct use in hospitals, or to clinical studies. Due to lack of regulation in the area, unlike for pharmaceutical drugs, it will probably proceed to direct use in some hospitals, rather than a clinical study. It should be noted that DeepMind is the unit within Google/Alphabet specialising in deep neural networks. Further research may show whether or not other simple models could achieve what this DeepMind algorithm achieves.”

Prof Paul Leeson, Professor of Cardiovascular Medicine, University of Oxford, said:

“This is important work in which the team have overcome several technical challenges to show it is possible to successfully apply AI to large scale electronic health records. The AI was able to identify over half the patients who went on to develop kidney problems during the next 48 hours. Trials are still needed to test whether this early warning is useful to doctors to improve patient care, without causing too many false alarms, or missing patients that the AI also overlooked. However, this is another strong example of how AI appears to have the potential to augment delivery of healthcare.”

Dr Saurabh Johri, Chief Scientific Officer, Babylon, said:

“This is a very interesting paper, one that shows the potential benefit of applying modern deep-learning approaches to real-world clinical settings. One of the strengths of the study is the size and diversity of the dataset used to train their model, illustrating what can be achieved where such data are available. However, as the authors highlight themselves, there is still some way to go before these and similar approaches can be applied in real-world clinical settings. First, despite using data from over 700,000 patients, there are significant limitations in terms of how well predictions from these approaches might generalise to patients that are not well represented in the data, for example the relatively small proportion of women in the dataset. Second, to assess the impact of these approaches in the real-world, a prospective clinical study should be considered. Thirdly, to ensure that these approaches are trusted and optimally used to support clinical decision-making, it will be important to consider how interpretable the predictions from such approaches are. These are all important considerations for many academics and organisations, who are exploring the application of AI for healthcare and this study offers a valuable contribution to the field.”

* ‘A clinically applicable approach to continuous prediction of future acute kidney injury’ by Tomašev et al. was published in Nature at 18:00 UK time on Wednesday 31 July. 

DOI: 10.1038/s41586-019-1390-1

Declared interests

Dr Franz Kiraly: “I’m a member of UCL, which is heavily involved in the project. I’m not involved or related to the project in any way but have had professional contact with one of the authors (Geraint Rees).”

Prof Paul Leeson: “I research grants in the field of medical AI and am academic founder of Ultromics, an AI-imaging company.”

Dr Saurabh Johri: “I work for, and have shares in, Babylon which is an AI healthcare company that uses machine learning techniques. Two of the Nature paper’s authors are investors in our company.”