June 4, 2015

expert reaction to study investigating predictors of risk of death within five years, and accompanying comment piece

Predicting the risk of death is the topic of a study and a comment piece published in the Lancet journal, in which the authors of the study used UK Biobank data and questionnaires to predict the risk of death in middle-aged and elderly people.

These comments accompanied a briefing.

Prof. David Coggon, Professor of Occupational and Environmental Medicine, University of Southampton, said:

“This study shows that, as might have been expected, the best predictors of mortality among middle-aged adults over a five year period were whether they were already ill at the start, and their smoking habits. The authors have developed and tested a score based on a small number of questions (13 in men and 11 in women), which can be used to predict people’s personal risk of dying over the next five years. However, I have doubts about the practical value of such scores. Most of the predictive factors do not directly cause disease, and even where they do, few are under the control of the individual. The authors suggest that knowing one is at higher risk may be an incentive to changes in lifestyle, but experience with smoking and obesity suggests that knowledge of increased risk has only limited impact on most people’s behaviours.”

Prof. Kevin McConway, Professor of Applied Statistics, The Open University, said:

“This is a good study that tells us some things we didn’t already know, and points the way to future use of a large and important data set. But it certainly isn’t some sort of oracle that you can use to predict exactly when you will shuffle off this mortal coil.

“My first comment is that anyone preparing a story on this study should be sure to read the accompanying commentary by Thompson and Willeit, which makes many important points about how the study should be interpreted.

“For me the most important aspect of this study is the way that it demonstrates the potential of data from UK Biobank to throw light on the causes of disease and death, and so potentially to suggest new ways of avoiding disease and deferring death. But because it is the first major study using the data in this way, the light that it throws is inevitably a bit limited. The researchers, Gamma and Ingelsson, have used appropriate and sophisticated statistical methods on a large data set to produce predictors of death rates according to a range of individual characteristics, and they have even made a website available that predicts individual chances of dying in the next five years.

“Although, on average, the predictions look rather accurate, you’ve got to bear in mind that age is included in the prediction method. One’s age does (on average) have quite a bearing on one’s chance of dying, and the relationship between age and the chance of death is well understood (and indeed is a key basis for the life insurance industry). The new prediction scores do add something considerable to age. However, many of the risk factors are also quite well known – most of us are aware, for instance, of the bad effect of smoking and lack of exercise on our life chances. For other risk factors identified in the study, knowing that the risk factor exists may not help much in reducing its potential impact. That’s for two reasons. First, one can’t do anything about some of the risk factors. Having had an illness or injury in the last two years is associated with an increased risk of death in the next five years, for example. But if I was ill last year, I can’t now go back and change that, in order to decrease my chances of death in the future.

“Second, the nature of the study means that we can’t tell which risk factors actually cause an increased chance of death, and which merely happen to be associated with something else that causes an increased chance of death. A good example of this is that ‘usual walking pace’ is a risk factor in men. Men who say they walk slowly have a higher chance of dying in the next five years. But that doesn’t mean that I can decrease my risk simply by making an effort to walk faster. It seems to me more likely that some men walk slowly because of some underlying health problem, and that it is that underlying problem that increases the risk rather than slow walking pace itself. But I don’t actually know whether that’s how the causation works, and a study of this sort can’t tell me.”

Prof. Tim Spector, Professor of Genetic Epidemiology, King’s College London, said:

“This is an impressively large study providing simple predictions from questionnaires of risk of early death up to the age of 70. It is novel because of the age group and the fact that it is one of the first to use the massive UK Biobank resource and has a website to help DIY searches. What is less novel is that many other similar measures of longevity and frailty also work well and the main risk factors will not be a surprise to us – or the insurance companies who have been doing this for years.”

Prof. Stephen Evans, Professor of Pharmacoepidemiology, London School of Hygiene & Tropical Medicine, said:

“If everyone read the commentary carefully and therefore modified the tendency to overstate the results from the paper by Ganna and Ingelsson, then everyone would benefit.

“This large study shows the hazards of ‘big data’ as well as their limited benefits. A large number (but not a random sample of the population) of men and women have been followed up for nearly five years on average, starting between age 40 and 70. 2.3% of the men have died and 1.2% of the women have died. If you are not currently seriously ill, then it is clear that age is the best predictor of your (quite low) chances of dying in the next five years. If you are currently seriously ill, then, not unexpectedly, your chances of dying are greater than those of someone who is not ill.

“None of these findings help in determining the reasons why you are ill or why you will die. It is almost a pity that the first analysis of data from Biobank does not use the data for which Biobank was set up, though it is not yet ready for good analysis.

“The authors’ claims that their scoring system has major advantages is over-stated. They say “Clinicians might use this score to identify patients at high risk of mortality to target with specific interventions”. In fact there is no evidence that their scoring system adds anything to clinicians’ judgement, knowing the age, gender and health status of their patients, since they are already reasonably good at prediction of mortality in a group, though prediction for the individual is much more difficult. This scoring system is also limited in its ability to predict in individual cases.

“They also say “Finally, government and health organisations can use this information to prioritise public policy to decrease the burden of specific risk factors.” This contradicts what they have said earlier, that what they have studied is associations, not causally-based risk factors. Reliance on studies of this type in ‘big data’ can mislead people into placing more reliance on the analysis than is warranted.

“The practical gains of the prediction are not stated clearly. The validation using data from Scotland is useful, but shows the results are not highly generalisable. The emphasis in the presentation of the ‘C-index’ is of technical interest, but does not communicate to a general reader the impact of that index. The vast majority of people in this age range do not die in the next five years, and it would seem, based on a calculation from their data, that even those with high scores on their index, 90% or more will survive the next five years. As the commentators Thompson and Willeit note, there is a real danger of ‘Cyberchondria’ and it seems likely that the publicity around this paper may feed that.”

Study: ‘5 year mortality predictors in 498 103 UK Biobank participants: a prospective population-based study’ by Andrea Ganna and Erik Ingelsson published in the Lancet on Thursday 4 June 2015. 

Comment piece: ‘UK Biobank comes of age’ by Simon Thompson and Peter Willeit published in the Lancet on Thursday 4 June 2015.

Declared interests

Prof. David Coggon, Prof. Kevin McConway, Prof. Tim Spector, Prof. Stephen Evans all have no interests to declare.