November 16, 2017

brain training for dementia

A new study, published in Alzheimer’s & Dementia: Translational Research & Clinical Interventions, reports that computerised brain training may reduce the risk of dementia by 29%.

A roundup accompanied this analysis.

Title, Date of Publication & Journal

‘Speed of processing training results in lower risk of dementia’ by Manson et al.

Published: November 2017

Alzheimer’s & Dementia: Translational Research & Clinical Interventions

Study’s main claims – and are they supported by the data

The main study claim is that following a computerised brain training programme can reduce the risk of dementia.

However, the study does not provide evidence to confirm this claim.

The study compares three cognitive training interventions to a control group. Among these three interventions, the proportion of participants with dementia 10 years later was 24.2%, 24.2% and 22.7%, as opposed to 28.8% in the control group.

The study title and press release focusses on one finding: that dementia was less frequent in the speed-training group, compared to the control group, with a 5% chance that this difference could have happened by chance alone (p-value*=0.049).  0.05 is a common threshold in statistics to consider that results were ‘unlikely’ to be due to chance alone, but this threshold is reduced in trials with multiple groups (for example to 0.01). Statisticians do not consider a p-value sitting at the 0.05 level in a multiple groups trial to be sufficient evidence of an effect.

The other finding highlighted, is that the higher number of sessions attended, the lower the risk of dementia. It is important to note that this effect is not necessarily causal** (participants who attended more sessions could have different characteristics to those who did not, since the number of sessions they attended was not randomised). The study took care not to conclude to a causal association**, but the press release contains a misleading statement that “those who trained more received more protective benefit”. A more accurate statement would be to say “a higher number of training sessions was associated with reduced risk of dementia”.

In the press release, we suspect a typo in the last paragraph. “the risk reduction ranged from 13-33% depending on how dementia was defined”.

Finally, the press release provides little mention of the other two interventions, only stating that there was “no significant difference”, but for both they found a reduction of 21% reduction in the risk of dementia. Although none of these results are statistically significant, it is unclear whether “speed training” is more effective than the other trainings, as these are not directly compared statistically. Essentially, the release focuses on one ‘amazing intervention’, but they all look more or less as effective except that one is below the ‘magic 0.05 threshold’.

While this study provides encouraging results, and the interventions seem unlikely to cause harm, further evidence is needed before making claims that training can effectively reduce the risk of dementia.

Strengths/Limitations

Strengths

Strengths of this study are the randomised comparison of the different interventions (so the participants in the different groups are comparable), and the long-term follow-up.

Limitations

The study does not provide conclusive evidence, as discussed above.

The ‘dose-response’ effect of more training being associated with lower risk of dementia is non-randomised only observational evidence.

‘Dementia’ was not based on a clinical diagnosis. Family members could have, for example, themselves reported their relative had dementia.

This study is a secondary analysis of a trial, which did not initially aim to look at the effect of dementia. A previous secondary analysis of the results at 5 years found no difference in the rate of dementia by training arm at 5 years, and therefore choosing to extend the analysis to 10 years could be seen as continuing to re-analyse the data until significant results are found. As the number of analyses from a trial increase, so does the chance to find a difference (by chance alone). The 10-year analysis was not specified in the initial trial protocol, which raises questions over whether it was added following the initial encouraging (but non-significant) results, which increases the chance of finding a ‘significant’ result even in non-effective treatments.

Only 50% of the study participants completed the 10 years’ follow-up. However this was expected in this population and, as discussed by the authors, dropout of participants was similar for all treatment groups and therefore it may not have affected the findings.

Though this would be very difficult (if not impossible) for these type of treatments, the treatment assignment is not “blinded***” from participants, meaning that those who receive the booster treatments are aware that they are receiving a “better” treatment. Without a true placebo any positive results could be due to participants’ perception that they are on a “better” treatment, rather than due to the effects of the treatment itself.

Most of these limitations were well acknowledged by the authors in the papers, but not mentioned in the abstract or in the press release.

Glossary

*p-value:  the chance to have observed such a difference between groups if the treatment does not have a real effect (difference observed “by chance alone”).

**Causal association: cause and effect. Epidemiologists pay attention to differentiate between crude ‘association’ (e.g. ‘more attendance was associated with lower proportion of dementia’ (those who attended more had less dementia, but they could just be those healthier etc. )),and ‘causal association’ (e.g. ‘more attendance reduces the risk of dementia’, meaning, the attendance itself had a beneficial effect).

***Blinding:  refers to the concealment of group allocation from participants involved in a clinical trial. It is used to reduce bias that could arise due to participants’ perceptions or knowledge about a treatment. This ‘performance bias’ can lead to improved outcomes, and so can be confused with a true treatment effect.

* ‘Speed of processing training results in lower risk of dementia’ by name of first author et al. published in Alzheimer’s & Dementia: Translational Research & Clinical Interventions on Thursday 16^th November.

Before The Headlines is a service provided to the SMC by volunteer statisticians: members of the Royal Statistical Society (RSS), Statisticians in the Pharmaceutical Industry (PSI) and experienced statisticians in academia and research. A list of contributors, including affiliations, is available here.