June 20, 2017

expert reaction to new report by the Academy of Medical Sciences

The Academy of Medical Sciences has published a new report entitled, ‘Enhancing the use of scientific evidence to judge the potential benefits and harms of medicines‘.

A briefing accompanied these roundup comments.

Prof Carl Heneghan, Director of the Centre for Evidence-Based Medicine, University of Oxford, said:

“The Academy of Medical Sciences report sets out the sizeable problems with the production and publication of medical research that is undermining trust and putting patients at risk.  Given the past problems it is not surprising that four-fifths of GPs and two-thirds of patients do not trust the findings from clinical trials done by the pharmaceutical industry.

“The report sets out many points that could improve the current deluge of poor quality evidence: improving trial registration, accessibility and reproducibility are essential to providing a robust evidence-base. Educating the public, dealing with the poor-quality information supplied with medicines, and the essential role of the media in disseminating evidence to a wide audience are vital to create better informed patients.

“However, there are shortcomings in the report due to the lack of specific measurable recommendations that could translate into real improvements. For example, conflicts of interests significantly undermine trust, but whereas specific legislation mandates industry to declare who they pay in the US this is largely left to the discretion of individual researchers based on this report.

“We now need a meaningful culture shift to produce research that is free from bias, is transparent and accessible to all, and ensures that clinicians and patient are truly informed about healthcare choices.”

Prof. Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:

“This is a good and extremely wide-ranging report.  The team behind it have done a thorough job of reviewing and considering evidence about the use of scientific evidence in medicine and health, and have come to sane conclusions about how things might be improved.  I agree with pretty well all their recommendations, and I hope that they will be taken up and acted on.

“That said, it’s difficult to predict what long-term impact the report will have.  There are twelve recommendations, and helpfully each uses bold type to indicate who the recommendation is aimed at.  In all, I counted 28 different organisations, groups, or generic types of people who have one or more recommendations aimed at them.  Some of these, such as NICE or the Science Media Centre, are pretty specific and it will be relatively easy to judge whether they have made progress.  Others, such as ‘the global research community’, or ‘the public’, might be rather more difficult to work with.  I note there isn’t a recommendation directly aimed at the Academy of Medical Sciences itself, but I hope they do continue to be actively involved.

“Speaking primarily as a patient who does assiduously read these things, I’m particularly happy with the recommendation to improve the content of patient information leaflets.  Given my professional background, I’m lucky enough to be able to understand what these documents are actually saying, but it (mostly) isn’t what I want to know, and it usually doesn’t help me much, for reasons that the report describes.

“The main body of the report starts by describing (in ‘Box 1’) some of the causes for concern that led to its production.  Having read the report, though, I was still left wondering what the authors would have preferred to happen in some of the case studies they describe.  Take for instance the HPV vaccine.  There was adverse media coverage about whether the benefits of vaccinating girls against HPV outweighed the risks, and there is evidence that this reduced uptake.  But there was, at the time anyway, uncertainty about the balance between risks and benefits.  There’s uncertainty at some level about almost everything in medicine, as the report itself acknowledges.  Perhaps the report’s authors would have wished the research on possible side effects to be better, or for it to have been better reported, but I hope they aren’t implying that it should not have been reported at all, and if it is reported, that will change some people’s decisions.  However good NHS Choices may become, and it’s already pretty good in my view, it won’t be people’s only port of call for health information and evidence.

“I’m pleased to see that uncertainty is given some prominence in the report, including some detail on good ways of communicating it.  But uncertainty also comes into the trust people put in scientific evidence.  The report found that patients and the public are not familiar with how scientific evidence is produced, and comments that this may lead some people to consider personal experience as more reliable.  Is this partly because scientific conclusions are generally uncertain to some extent, while personal experience can appear certain, even if it’s wrong?  How should we communicate about the inevitability of uncertainty in a world where we all seek certainty?”

Dr Louise Brown, Senior Statistician, MRC Clinical Trials Unit, UCL, said:

“I welcome this report, both the content and the message are clear and The AMS should be congratulated not only for highlighting the problems but also offering some progressive steps for change.  Of course the challenges we face are not new, they have been a recognised problem since the beginnings of medical endeavour but we are at a point when they have been heightened by the digital age in which we live.  The internet and social media are both a blessing and a curse.  We are all bombarded with an unrelenting stream of new information that is overwhelming and very difficult to process.  Unsurprisingly, this leads to feelings of scepticism and mistrust and I feel that the recommendations of this report are much needed.  In particular, an improvement in the ways in which the REF might reward robustness and reliability of research outputs is encouraging.

“The report quite rightly points the finger at all of us who are involved with undertaking, disseminating and publicising clinical research.  However, I feel there is a noticeable absence from the list of recommendations for change.  The responsibilities of government to see beyond the short-termism of vote-grabbing and party political brouhaha might be another welcome area for change.  Government needs to engender an open and balanced system within which all stakeholders in clinical research can flourish with adequate resource and not feel penalised for sometimes speaking openly on subjects that may not court popular public support.  The recommendation for incorporating ‘intelligent openness’ into the next REF is to be welcomed providing this openness becomes an influential part of policy-making, not just within NICE and the MHRA but also in the highest levels of government.”

Prof Sir Nilesh Samani, Medical Director, British Heart Foundation (BHF), said:

“Patients and doctors need to be confident that recommendations about treatments are based on the best available scientific evidence.  It is equally importantly that the benefits and potential harms of treatments are explained to patients in ways that are simple to understand, are not hyped up and are meaningful to their personal situation so they can make an informed choice.

“We strongly welcome this important report from the Academy of Medical Sciences.  As the largest independent funder of cardiovascular research in the UK, we look forward to working with others to implement these excellent recommendations to improve the information and support for people living with heart and circulatory disease in the UK.”

Prof Peter Openshaw, President, British Society for Immunology, and Professor of Experimental Medicine, Imperial College London, said:

“The use of powerful new approaches to discovering novel treatments is leading to a gratifying flood of new drugs to treat a range of conditions.  For example, immunology is seeing a revolution in our understanding of the basic science behind how the immune system functions; month by month, new treatments are licensed that extend the therapeutic options we can offer patients.  As we gather knowledge and experience, vaccines and immune-based treatments are set to bring immense benefits to patients.

“However, this report from the Academy of Medical Sciences is a timely reminder that we must always keep in mind the most important person in this equation – the patient.  Engaging with patients, funders and the public to clearly communicate the science behind these medical innovations and in turn listening their viewpoints and concerns is a key part of our role as doctors and scientists.  Providing accurate and accessible information on medical innovations and educating people on how to judge the benefits and harms of medicines should be seen as an integral part of developing any new treatment.  It is only by working in close partnership with patients, clearly and honestly explaining the scientific evidence, that we can fully realise the huge potential that 21st century medical science offers.

“The British Society for Immunology is pleased to have provided financial support for this important initiative.”

Jonathan Brüün, Chief Executive, British Pharmacological Society, said:

“While the UK is a world leader in medical research, and at a time when the NHS spends almost £17 billion on prescription medicines, it is striking that there is a confidence deficit among prescribers and patients about making sense of medical research findings.  Transparent and accessible information about the scientific evidence for safety and efficacy of medicines should be a priority.  On behalf of the British Pharmacological Society, I welcome this report, and look forward to the pharmacology and therapeutics community playing a key role in addressing the challenges that have been identified.”

Fiona Fox, Chief Executive, Science Media Centre, said:

“Too many reports of this kind ignore the role of science press officers when focusing on the media reporting of medical research.  But it’s impossible to understand what goes right and wrong in the reporting of stories like statins, Tamiflu and HRT without getting to grips with the role that scientists, journals and press officers play in those stories.

“Doubtless the food industry would remind us that a traffic light system can be a blunt tool.  But we need to do something more to ensure that journalists and through them the public can distinguish between studies that are interesting but small, preliminary and need to be replicated, and large randomised controlled trials or Cochrane reviews that confirm previous findings.

“For me this is less about flagging this up to our science journalists, though that is useful, and more about the medical research community taking responsibility for more measured communication of our own research.  For example press releases from abstracts presented at conferences which may never be published may get a red light.  Observational studies that can show an association but not a cause might be amber.  If the findings are interesting or surprising they will still be reported.  But hopefully the process of badging will encourage press officers, authors and journals to highlight caveats and limitations.  Interestingly recent research showing that exaggerations in media stories could often be traced back to press releases also showed that the more measured press releases were covered just as much.”

* ‘Enhancing the use of scientific evidence to judge the potential benefits and harms of medicines’ will be published by the Academy of Medical Sciences at 00:01 UK time on Tuesday 20 June 2017, which is also when the embargo will lift. 

Declared interests

Prof. Kevin McConway: “I have no relevant interests to declare, even after reading the excellent section in the report about declaring interests.”

Dr Louise Brown: “No conflicts or declarations to make.”

Prof Sir Nilesh Samani: “The report was part-funded by the BHF.”

Prof. Peter Openshaw: “The British Society for Immunology did provide a financial donation towards this report and Peter did give evidence in one of the hearings for it.  Prof Peter Openshaw’s research is funded by the Wellcome Trust, the MRC, BBSRC and the European Union. He has received honoraria or consultancy fees from GSK, Janssen, and Mucosis BV.”

Jonathan Brüün: “The British Pharmacological Society co-funded and its members contributed to key elements of this research.”

Fiona Fox: The Science Media Centre contributed to this report.  Fiona Fox gave oral evidence to the oversight group, and the SMC contributed written evidence to the project.  Fiona Fox took part in a session on conflicts of interest.  The SMC organised and ran a workshop on communicating evidence with journalists and press officers.