select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A systematic review published in The Lancet looks at the comparative effects of pharmacological interventions for the acute and long-term management of insomnia disorder in adults.
This Roundup accompanied an SMC Briefing.
Dr Baptiste Leurent, Lecturer in Medical Statistics, University College London, said:
“This was a very well conducted meta-analysis, and probably the best comparison of the different drug treatments available for insomnia to date.
“The studies included were all randomised, so there is no “confounding” and we can attribute the differences seen to a pharmacological effect of the drugs.
“A limitation of network meta-analysis is that it relies on “indirect” comparison, and assumes that two trials comparing A vs. B and B vs. C can give you information about A vs. C. This is not always the case as the two trials involved could be quite different (e.g. different population or drug dosage). This assumption is difficult to verify, but the authors discuss this possibility and do not raise major concern.
“One important point to keep in mind is that this review do not compare drugs to non-pharmacological interventions (sleep hygiene, CBT), and does not allow to draw any conclusions on the efficacy of one versus the other. The information it provides is only about the relative efficacy and side effects of different pharmacological treatments. The bottom line is that drugs can be effective in reducing insomnia, but often come with side effects.”
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“Other things being equal (though in real life they hardly ever are), a good randomised controlled trial (RCT) is generally considered to be a strong way of finding evidence on whether one intervention – a drug, or some other kind of treatment – provides better outcomes than another intervention in treating a health condition. That’s partly because, in an RCT, it is decided by a random process which patient gets which intervention when. That means that the choice of intervention is not, on average, related to any other characteristics of the patient, so that, if one intervention has a better outcome than the other, it’s unlikely that that difference is caused by anything other than the interventions themselves. And there are statistical ways of quantifying the uncertainty about which intervention is better.
“However, RCTs have their limitations. One is that an intervention may work differently in different places, or in different types of patient, and it isn’t usually possible to include every place were the treatments might be used, or every relevant type of patient, in a single RCT. For most commonly-used interventions, particularly pharmacological ones, there will have been more than one RCT. Therefore it has become useful and common to carry to systematic reviews of all the RCTs that have been carried out to compare two or more interventions for a disease or condition. These will typically involve using a statistical approach called meta-analysis to put the results from the RCTs together. The meta-analysis can produce an overall estimate of the average difference in outcomes between the interventions that are being compared, and it can also examine whether the results from different trials in different circumstances are reasonably compatible with one another. If they aren’t, for example if one of the treatments is better with particular types of patients but not with others, the meta-analysis might be able to summarise those patterns of difference.
“What you might call the standard methods of meta-analysis generally work by combining RCTs that all compared the same, fairly small, set of treatments – often only two treatments. This would cause a problem if (in the simplest case I can think of) there were three treatments available for a condition, and the available RCTs had each compared just two of the three treatments. So perhaps there would be one set of RCTs that compared treatment 1 with treatment 2, and another set that compared treatment 1 with treatment 3. A standard meta-analysis could produce an estimate of how well treatment 1 worked in comparison with treatment 2, and treatment 1 compared to treatment 3, but it would not tell you how treatment 2 compared to treatment 3 if none of the individual RCTs had tested treatment 2 against treatment 3. However, a newer approach, called network meta-analysis, could tell you something about how treatments 2 and 3 compare. To over-simplify a bit, it does this by (say) calculating how much better treatment 2 is than treatment 1, and also how much better treatment 3 is than treatment 1, and using these two findings together, with treatment 1 as a sort of baseline, to estimate how much better treatment 2 is than treatment 3 (or the other way round if in fact treatment 3 seems to be better).
“This new study involves a network meta-analysis, but it is much more complicated than a comparison of just three interventions. Instead it combines the results from RCTs of a large number of different pharmacological interventions (drug treatments, speaking crudely) for insomnia. The researchers compared 30 different interventions, and also placebo (an intervention of taking a ‘dummy’ tablet that contains no active drug), using data from 154 trials. They also compared more than one type of outcome – they looked at whether the trial participants’ condition improved after treatment, and at how many of them dropped out of the trial (which could be because of adverse side-effects of the treatment, though there are other possible reasons), and where possible they looked at both of those aspects both in the early stages of treatment an in the long term. They also looked at how many of the participants suffered adverse effects from each treatment.
“So a lot of drugs, and several different comparisons. The results are complicated. I’m not a clinician so I won’t attempt to summarise all the details. It would have been convenient if there were just one treatment that caused more participants to improve than all the others, and had fewer side effects than all the others, but life is hardly ever that simple. The researchers instead found that some interventions were more efficacious than others, that is, more patients improved with them than with others, and that some appeared to produce more adverse effects, but that the patterns of comparison were different with different pairs of treatments. So in several cases there were trade-offs to consider – a treatment might be more efficacious than most but have more adverse effects, so whether to recommend that treatment might depend on how the clinician and the patient weighed up the risks of adverse effects against the chance of a more successful treatment. So there isn’t anything like a clear overall ‘winner’ amongst the interventions being compared.
“All the estimated comparisons between interventions are subject to statistical uncertainty, because every RCT’s results are subject to some statistical uncertainty and that uncertainty can’t entirely vanish when the trial results are put together. The network meta-analysis also provided useful but detailed information on the degree of statistical uncertainty for each comparison. Some of the comparisons did not provide clear results, usually because there was not enough data. Further, the researchers assessed the quality of the information from the trials that they considered, and concluded that many of the comparisons were of low or very low quality because of methodological or reporting issues in the individual RCTs. So, even though this is network meta-analysis on a major scale, and even though it appears to have been conducted very competently, it can’t answer every question that clinicians and patients would want to ask. That’s life, I’m afraid.
“One particular issue to which the researchers draw attention is that insomnia tends to be a long-term problem, but that the meta-analysis was limited in what it could say about the long-term efficacy or safety of many of the treatments, because there were only eight long-term studies out of the 154 that were included. Unfortunately this seems to be a more general problem with RCTs, not just in insomnia. An RCT is an expensive undertaking, and the longer it goes on, the more expense is involved, so long-term data does tend to be in relatively short supply. That’s clearly an important problem with treatments of long-term health issues.
“Another question relates to the fact that this research deals only with pharmacological interventions, and this despite what the researchers say at the very beginning of the summary section of their report: “Behavioural, cognitive, and pharmacological interventions can all be effective for insomnia. However, because of inadequate resources, medications are more frequently used worldwide.” This study did not compare the pharmacological treatments with behavioural or cognitive interventions. It would probably have been difficult to do so, given that there are would have been methodological differences between RCTs comparing drugs and RCTs comparing behavioural or cognitive interventions that could cause difficulties in comparing them adequately. But this does not mean that the drug treatments should always be used as the first line of treatment of insomnia, as the research leader, Professor Cipriani, makes clear in his quote in the press release. As a non-clinician, I can’t comment further on this aspect.”
‘Comparative effects of pharmacological interventions for the acute and long-term management of insomnia disorder in adults: a systematic review and network meta-analysis’ by Franco De Crescenzo et al. was published in The Lancet at 23:30 UK time on Thursday 14th July.
Declared interests
Dr Baptiste Leurent: “No conflicts of interest to declare.”
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. My quote above is in my capacity as an independent professional statistician.”