Title, Date of Publication & Journal

Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Updated Cochrane Review, to be published in the Cochrane Library, April 2014.

Claim supported by evidence?

Neither the Cochrane review nor the press release claims that these drugs (Tamiflu and Relenza) are ineffective. The claims are that they shorten the symptoms of influenza but only by between a day and half a day, that there is some evidence of increases in some adverse effects in people taking the drugs, and that there is no good evidence to support claims that the drugs reduce admissions to hospital or complications of influenza. In my view these claims are all broadly justified by the data and the analyses in the Cochrane review, though there are some issues of detail, dealt with below. (But you must bear in mind that having no good evidence to support a claim does not mean that the claim is false – perhaps not enough evidence has yet been collected.)

Summary

• A major review of data from clinical trials, published and unpublished, which allows evaluation of almost all the available evidence.
• Some important questions about Tamiflu and Relenza cannot be answered from this work.
• The review raises important questions about how decisions on drugs should be made, but in itself it cannot answer those questions.

Study Conclusions

The review is about two different antiviral drugs, Tamiflu (oseltamivir) and Relenza (zanamivir), and considers their use in both adults and children, both to treat flu-like symptoms in people who already have the disease, and to try to prevent influenza in people who do not yet have it. But it is important to note that the title refers to preventing and treating influenza in healthy adults and children. It (mostly) does not review data on the use of these drugs in specific groups of people that are vulnerable because of some pre-existing disease, for instance.

The key difference between this review and other systematic reviews is that it is based on data in the form of ‘clinical study reports’, made available by the drug manufacturers, which were originally produced for internal purposes and for submission to the authorities in the US, Europe and Japan who decide whether the drugs can be licensed. The review is also based on comments made by the regulators. Systematic reviews are more usually based on reports of clinical trials published in medical journals.

Many of the conclusions in the review are strongly justified by the data and analyses.

I will consider separately the two major types of study, for treatment of influenza when symptoms already exist, and for prevention of influenza in healthy people. First, treatment.

1. TREATMENT IN PEOPLE WITH INFLUENZA

1.1. Symptoms

The conclusion that taking one of these drugs reduces the time taken until flu-like symptoms are relieved, in adults, is clearly justified. The conclusions on the size of this reduction are also reasonably secure, but one must take into account that the reductions are averages and are subject to some uncertainty. (For instance, the average reduction with Tamiflu is about 17 hours, somewhat more than half a day, but the review also states that the average may be somewhat more or less than that (plausibly between 8 hours and 25 hours), and the reduction in an individual patient could in any case be a lot more or less than this average.)

The review and press release describe this reduction as ‘just half a day’ and ‘small’. This is a value judgement about the size of the reduction that cannot be supported or refuted by the data.

The press release says that the effect on the length of time that symptoms were apparent was ‘more uncertain’ in children. In fact this is really the only situation where the review separates out patients with a particular risk factor for analysis. They did find a significant reduction in time for alleviation of symptoms, with Tamiflu, in children who did not have chronic asthma, but they did not find evidence of a reduction in children with chronic asthma. (They found no effect on duration of symptoms with Relenza in children.)

1.2 Hospital admissions and complications

The reviewers found very little evidence that taking Tamiflu to treat influenza reduces admissions to hospital. They also found very little evidence that taking either drug reduced complications of influenza. However, in this respect, lack of evidence for a reduction cannot immediately be taken to mean that there is no reduction in the complications.

There are two problems in studying complications. First, according to the review, most of the trials were not very precise or reliable in how they defined and recorded the complications. Second, most of these complications are relatively uncommon. Even with the large number of patients included in all the trials under review, the great majority of patients had no such complication whether they were taking the drug or a placebo, so that there were sometimes too few of these complications to allow any statistically clear conclusion to be drawn.

1.3 Harms

Finally on the treatment trials, the reviewers considered whether the drugs had raised levels of adverse effects, compared to placebo. Here the evidence is mixed. As the press release says, Tamiflu increased somewhat the level of nausea and vomiting, though the press release does not add that it decreased the risk of diarrhoea in adults. Both the press release and the review point out that there is evidence of a small increased risk of psychiatric events. In fact this risk was not statistically significant, and the main basis for the claim appears to be an increase of psychiatric adverse events in two trials, only in patients who were given a larger than standard dose of Tamiflu.

2. PROPHYLAXIS / PREVENTION

2.1 Symptoms

I now turn to trials for prevention. The reviewers and the press release point out that both drugs reduce the risk of people suffering symptomatic influenza. The press release does not say how large these reductions are; the position is complicated, depending on whether influenza is present in the family of the person being studied.

2.1 Harms

The risk of harms from preventive treatment was also considered. With Tamiflu, there was statistically significant evidence of an increased risk of headache and nausea when taking the drug. The review and the press release also report an increased risk of renal (kidney) adverse events, and psychiatric adverse events. However, the evidence for these increases is weak. The difference between those on the drug and those on the placebo was not quite statistically significant for renal events, in the main statistical analysis. The difference for psychiatric events was significant only when the reviewers included events that occurred while the patients were being studied but not actually taking the drug, as well as events when they were taking it. These events were in any case not common, and most (though not all) of them were not severe, but this may well be another situation where there is not enough data from the trials to be conclusive.

2.3 Transmission

Finally, the review and press release state that it is unproven that Tamiflu can stop people carrying the virus and transmitting it to others. The basis for this conclusion is rather different from the others. The review found that one important reason for the claim was not in fact supported by the evidence. Furthermore, the researchers discovered a statement by the FDA (the Food and Drugs Administration, which regulates drugs in the USA) that a claim of interruption of person-to-person transmission was not justified by the data presented to them.

Strengths/Limitations

Strengths

Compared to reviews based on published work, this review is much more comprehensive in covering almost all of the clinical trials that have been done on these drugs, and is also based on much more comprehensive data on each trial than would typically appear in a paper in a medical journal. This is a major strength. In addition to simply making more of the usual kinds of data available, the reviewers had access to information that would never be in published trial reports, such as comments made by the regulatory authorities in response to the information provided by the drug manufacturers. This Cochrane review is a revised version of previous reviews based (mainly) on published data, which found evidence of publication bias – that is, the published studies did not fairly represent the conclusions one would draw from the results of all trials. This new review corrects this important problem.

A further strength is that the information from different trials is aggregated by standard statistical meta-analysis methods for this kind of review, and that the detailed results of all these meta-analyses are presented in the Cochrane review report (one of the reasons why it is 558 pages long).

Limitations

One limitation, made very clear by the reviewers, is that previous systematic reviews have not used the same kind of data source (clinical study reports and regulator comments), and that they had to develop new methods of working with the resulting huge quantity of data. The reviewers are assiduous about describing how they approached this enormous task, and in my opinion they did use appropriate methods, but it is impossible to be certain that this approach did not lead to any biases or miss important information. I have no reason to believe that it did suffer from any such flaws, but really we will be able to tell whether the methodology is entirely appropriate only after it has been more widely applied.

Arguably, a limitation is that the review, huge as it is, is based only on data from randomised clinical trials. As mentioned above, despite the size of the trials, there were simply insufficient data on complications of influenza and on some possible adverse effects of the drugs, because the complications and adverse effects are rare. It is sometimes possible to get further data on these rare events from other types of study (observational studies in very large groups of people who have used the drugs, for instance), though such studies are difficult to interpret and can be subject to serious biases. There have been some such studies, and indeed reviews of them, but the conclusions so far have generally been inconclusive. It would not have been appropriate for this particular review to use data from such studies, but those making public health decisions on the drugs may wish to take them into account, where possible.

One concern is that, in their summarised conclusions, the authors describe the reductions in time to alleviation of symptoms after taking Tamiflu as small in adults and non-existent in asthmatic children, without mentioning that they do also exist in children without asthma. Furthermore, they do not describe the increases in risk of adverse effects as small, even though at least some of them arguably are small, a few of them are arguably not statistically significant, and some show evidence of a reduced risk in those taking the drug.