October 30, 2017

expert reaction to using machine-learning to identify suicidal patients

A new study in Nature Human Behaviour used machine-learning algorithms to identify individuals who engage in suicidal thinking.

Dr Dina Popovic, Head of Department of Psychiatry, Sheba Medical Center, Israel, said:

“Psychiatric patients are 10 times more likely to attempt suicide than the general population, and 50-75% of individuals who commit suicide suffer from depression or bipolar disorder. Nonetheless, suicide is a very rare event and thus difficult to predict. There is a desperate need for tools that would allow people to detect patients who are likely to commit suicide.”

“The study is very interesting and innovative, and represents an original attempt to overcome the current lack of instruments enabling the clinicians to better predict suicidality. The results are very promising, but the sample size is very small, it would be necessary to increase the number of patients and to see if other groups reach same results.

“It is necessary to compare the suicidal patients with depressed patients without suicidal thoughts in order to see if the machine-learning classifier is really able to detect suicidality, and that the abnormalities (‘neural signature’) are not due to depression.

“Finally, the machine-learning classifier requires patients to be cooperative and focused for 30 minutes – most patients commit suicide when they are in an agitated state, and inability to focus is very common in depressed patients. Still, the study is very interesting and may represent a breakthrough point in modern psychiatry.”

Prof. Seena Fazel, Professor of Forensic Psychiatry, University of Oxford, said:

“This paper is valuable insofar as it provides more information on possible cognitive mechanisms for suicidal ideation. However, in terms of predicting suicide risk, it is unlikely to move the field forward.

“First, it is not a scalable assessment as it requires participants to focus their attention for about 30 minutes and be willing to undertake a functional MRI.

“Second, the neurosemantic tests used show some discrimination between people who have suicidal ideas and those who do not, but in clinical practice, this is not the issue, which is around identifying and then managing risk of suicide attempts and completed suicide.

“Third, their findings do not show that the neurosemantic tests add any incremental performance to predictors of suicide risk that rely on taking a relevant history and conducting an clinical assessment, which are likely to be more scalable approaches.

“Fourth, any prediction study needs typically to have around 10 ‘events’ per risk assessment item in a validation sample. In this study, we have 21 individuals with suicidal ideation in a validation sample, and 30 ‘stimulus items’, which would suggest at least 300 people with suicidal ideation are required to conduct a robust validation.

“Finally, the control group used in the validation was drawn from the same control group used in the discovery sample, which would likely over-estimate the performance of their tests.”

Prof. Derek Hill, President, Regulatory Science and External Relations, IXICO plc and Professor of Medical Imaging Science, UCL, said:

“Identifying suicidal young adults is a hugely important and challenging task in medicine. This paper demonstrates the potential of brains scans, processed by a computer, to identify those at risk of suicide, but the result must be considered very preliminary. As the authors acknowledge, they have looked at only a small number of subjects (17 controls and 17 at risk of suicide, nine of whom had actually attempted suicide). It is important that their results are replicated before having confidence in their results.

“They used a method called ‘cross validation’ to both train and test their machine learning algorithm on the same small dataset. While this is a widely used approach, it is not a true replication study, so it isn’t yet clear whether their algorithm would work on another separate group of patients.

“Furthermore, there are many challenges to routine use of their method in a healthcare setting. The sort of functional brain scanning that the researchers employed is only available in advanced research institutions, and requires cooperative patients, so wouldn’t be widely available to mental health patients in the near future.

“Also, their algorithm would have to be turned into a cleared or approved medical device before it could be used to help manage patients, and this would require data from a much larger study and take several years to achieve.

“Machine learning uses computer algorithms to discover features in data (such as brain scans in this case) that can classify people into groups.  This research looks at people at risk of suicide, and shows good ability to predict from people’s brain scans whether they are at risk of attempting suicide.”

* ‘Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth’ by Just et al. published in Nature Human Behaviour on Monday 30^th October.

Declared interests

Dr Popovic: “Dr.Popovic has served as a speaker, medical writer or has participated in advisory boards for Bristol-Myers Squibb, Merck Sharp& Dohme, Janssen-Cilag, Ferrer and Forum Pharmaceuticals.”

Prof. Fazel: “Received funding from the Wellcome Trust to examine approaches to suicide risk assessment in individuals with mental illness. Nil else.”

Prof. Hill: “I have no conflict of interest regarding this story. I am employed by IXICO, which uses MRI biomarkers and digital biomarkers (wearable biosensors) in clinical trials of brain diseases, including psychiatric condition, but have no commercial interest in the research described.”