select search filters
briefings
roundups & rapid reactions
Fiona fox's blog
A study published in the BMJ looks at the association between chocolate intake and type 2 diabetes.
Dr Mohammad Talaei, Postdoctoral Researcher, Queen Mary University of London (QMUL), said:
“The press release is generally okay, but I would prefer a more cautious title, something like: “People who ate more dark but not milk chocolate were at lower risk of type 2 diabetes.” This is what the study is precisely reporting; I hope the distinction is clear.
“The study has controlled for a long list of potential confounders, i.e. common causes of a higher chocolate intake and diabetes. They controlled for some indicators of health-conscious behaviour, like multivitamin intake, physical activity, and overall diet quality. However, in observational research, it is hard to make sure the effect of those factors is fully controlled or that unknown factors are not missed. A hypothetical confounder associated with chocolate intake and diabetes with a strength of at least 86% (1.86) would be able to explain away the observed association and a strength of 29% would explain away the statistical significance. Although these numbers per se are not large enough to make such an unknown confounding effect unlikely, it would be hard to think of a single missing factor with such a strength, particularly given the long list of potential confounders that authors have considered. However, multiple weaker factors can still make up such an association between dark chocolate and diabetes, particularly when the problem is how well the known factors like sugar intake were measured. The authors have adjusted for added sugar and sugar-sweetened beverages, which slightly attenuated the association but did not include total sugar intake. Given the strong link between sugar intake and chocolate intake and diabetes, the suboptimal assessment of sugar intake might substantially contribute to residual confounding with great consequences for the observed associations.
“For now, as the authors have mentioned, one experimental study has already disputed these findings. However, the intervention was cocoa flavanol supplements for only 3.5 years, which is not completely comparable with the much longer follow-up in this study. There is an important sign that dark chocolate might be an indicator of something else and not necessarily the agent of effect. Researchers in this study reported a much stronger association in half of the participants with a relatively healthier diet. This could be interpreted as a sign of an unknown biological interplay between dark chocolate and other aspects of a healthy diet. An alternative interpretation, however, could be that dark chocolate intake is an indicator of a sweet tooth, hence a more accurate predictor of total sugar intake, which produced a more precise signal in the context of a healthier diet. The bottom line is that we still cannot recommend dark chocolate to prevent diabetes until these results are replicated in other observational studies with different population profiles and verified in randomized controlled trials.”
Prof Kevin McConway, Emeritus Professor of Applied Statistics, Open University, said:
“This is a carefully conducted study, with large numbers of participants, but its results are not straightforward to interpret. Inevitably it leaves a lot of questions unanswered. That’s because it’s an observational study, and observational studies cannot provide reliable information on what causes what. Also, the data comes from three large long-term studies in the USA, which recorded a great deal of information for health research – but researchers using those results have to work with what was collected in the original studies, and that doesn’t always match what ideally the new researchers would have wanted to know.
“I’ll just point out one issue with the version of the press release that I have. It says that there were 11,654 participants in the part of the study that looked at different subtypes of chocolate (milk or dark). That’s a typo – according to the research paper, the number was 111,654 (and that fits with other numbers in the press release and the research paper).
“The issue about cause and effect is clearly acknowledged by the researchers in their research paper, and in the press release. The association that they found between eating more dark chocolate, and having a lower risk of contracting type 2 diabetes, is an association, or in another word, a correlation. We can’t be at all sure that eating more dark chocolate causes the reduction in diabetes risk. It may be the cause, or part of the cause, but it may not.
“That’s because people who eat different amounts of dark chocolate differ in many other ways, apart from their dark chocolate consumption. One or more of these other differences might be the actual cause of the differences in type 2 diabetes risk.
“Obviously the researchers are aware of that issue. It’s possible to make statistical adjustments to try to allow for the effects of other factors that could be related to both chocolate consumption and diabetes risk. These researchers made many adjustments of that kind. Their results after making the adjustments were often quite a lot different from the unadjusted figures, showing how important it is to make adjustments.
“Also, the researchers carried out a wide range of so-called sensitivity analyses, where they looked at different statistical analyses that made rather different choices about how to deal with the data. On the whole, these sensitivity analyses didn’t affect the overall findings in any major way.
“However, as the researchers say explicitly, that doesn’t remove the doubts about cause and effect. They have some ideas (from other research) about what might be driving an association between dark chocolate consumption and diabetes risk, to do with substances present in chocolate (flavonoids, particularly flavanol), but this new study didn’t throw much further light on that possibility.
“The researchers had an interesting secondary finding, that there was an association between increased milk chocolate consumption over a four-year period and weight gain over the same period, but that that was not the case for consumption of dark chocolate. But this result has the same kinds of issue. We don’t know whether consuming more milk chocolate caused the weight increase – other things would have changed too. And, if nothing else, this result does indicate that patterns of cause and effect are likely to be complicated, involving possible changes in body weight as well as in diabetes risk
“Therefore what the researchers recommend, rightly in my view, is not that we should all go and eat more dark chocolate now, but that randomised trials are carried out to see whether they replicate the findings of this study, and to explore what might be causing these associations. That’s a good recommendation, though I think it will not be easy to carry out trials like that.
“The nature of data on food consumption collected in the three long-term studies means that the researchers had data on how many times a week or a month the participants consumed chocolate, but, it appears, no good data on how much chocolate they ate when they did eat some. That makes it a little difficult to interpret the different levels of chocolate consumption that are reported in the results. On average someone who eats five or more servings of chocolate a week is going to eat more than someone who eats fewer than one serving a week, but how much more?
“Another inevitable limitation arising from the data that the researchers used is that some important outcomes, were still not very common. Yes, over 111,000 participants were included in the analysis of type 2 diabetes risk by type of chocolate (dark or milk), but under 5,000 of them were diagnosed with a new case of type 2 diabetes during the many years of study. That’s still quite a lot – but there were only 132 type 2 diabetes diagnoses in people who ate at least 5 servings of dark chocolate a week. That’s really not many to compare with the diagnosis rates in people who ate dark chocolate a lot less frequently – so there are rather wide margins of statistical error in the risk estimates.
“For instance, the researchers report that the risk of a new type 2 diabetes diagnosis, in people who ate five or more servings of dark chocolate a week, was 21% less than for people who ate dark chocolate less than once a month (or not at all). That sounds quite impressive, even though we don’t know if the risk difference is caused by the difference in chocolate consumption.
“But we need to consider two further things about this result, and the same goes for most of the other findings.
“First, there’s considerable statistical uncertainty about that 21% figure. The true figure according to calculations reported in the research paper, could plausibly be somewhere else between a 5% smaller risk and a 34% smaller risk.
“Then, 21% (or 5% or 34%) of what? The absolute risks involved are not all that large. Imagine a group of 10,000 people, like those in this study who ate very little or no dark chocolate (less than one serving a month). On the basis of the data in the study, we’d expect 40 of them to have a new type 2 diabetes diagnosis in a period of a year. In another group people with the same characteristics, but who ate at least 5 servings of dark chocolate a week, if the dark chocolate consumption really is causing the risk difference , we’d expect 32 new diagnoses in a year (on the basis of the 21% reduction estimate), or somewhere between 27 and 38 allowing for the statistical margin of error – so a reduction in numbers of diagnoses in a year between 2 and 13 in 10,000 people, if the association is really one of cause and effect.
“A reduction like that could be worthwhile, because type 2 diabetes is a serious cause of ill health – but it’s not huge, and it might not exist at all because of the doubts over what is causing the risk difference. That’s why more research is needed.
“But randomised trials are, I suspect, not going to be easy. In principle, a trial that randomises people to eat different amounts of dark chocolate might be feasible, though milk chocolate would be more problematic given the finding about weight gain. But over what timescale? The effects of diet on weight, diabetes risk and many other health issues take time to develop. A trial that follows people up until they have a new type 2 diabetes diagnosis would be a long-term endeavour, so it would be expensive, difficult to keep in touch with participants, and probably difficult to recruit them and keep control over how much chocolate they eat, so that may well be out of the question.
“A shorter-term trial that works with people at particularly high risk of diabetes, and that records some other outcome than an actual type 2 diabetes diagnosis, could be feasible. Indeed the research paper describes one that was reported in 2008. (Reference 45 in the paper.) The participants already all had high blood pressure and glucose intolerance (which includes type 2 diabetes but also other conditions that might lead to type 2 diabetes). Some of them, chosen at random, ate quite a large amount (100 grams a day) of dark chocolate which was high in polyphenols and the others (the control group) ate 90 grams a day of white chocolate instead. By the end of the trial those eating dark chocolate had lower blood pressure and improvements in insulin sensitivity compared to the control group. Trials like that can tell us something about how dark chocolate might make changes inside the body, that might be favourable in reducing diabetes risk – but they aren’t measuring diabetes diagnoses in people who are typical of the general population.”
Prof George Davey Smith, FRS FMedSci, Professor of Clinical Epidemiology, University of Bristol, said:
“Xmas is supposed to be a time of goodwill, and medical journals like the BMJ print seasonal papers not intended to be taken too seriously. The topic of this paper – chocolate – has been the topic of Xmas BMJ papers in the past, but it appears this is intended to be a serious paper; it is certainly not entertaining. At the risk of being a Grinch at Christmas, it has to be said that this is, frankly, a meaningless paper. Indeed it is surprising to see such a paper published in 2024 in a supposedly reputable journal.
“As with the vast array of dietary factors investigated in the studies used in this paper the authors find something which appears to be protective of future disease, in this case dark chocolate supposedly preventing type 2 diabetes. The authors carry out a range of statistical analyses supposedly adjusting for confounding factors, and report that a residual “statistically significant” finding remains, and then go on to imply this could indicate protection against diabetes by increasing dark chocolate consumption. In the same studies when such findings have been followed by randomized controlled trials (RCTs) – which is relatively rarely – they have consistently found no actual effect, and it is clear that the observations just reflected residual confounding and selection bias.
“For examples of these see the first few minutes of the recent inaugural Bennett Lecture at Ben Goldacre’s Bennett Institute for Applied Data Science1. This shows how in the same studies and using the same methods more than 30 years ago (and with some of the same authors) a large protective effect of vitamin E supplement use was shown for coronary heart disease, later followed by a substantial number of RCTs finding absolutely no such effect. The same sad story has been repeated again and again, as documented in the beginning of the lecture and the first couple of pages of this.
“That consumers of milk chocolate differ, on average, from those of dark chocolate will not surprise anyone. As much research has shown2, dark chocolate consumption is higher among individuals in higher-income groups. Studies attribute this pattern to the greater availability of premium and fine chocolate varieties in affluent markets and the appeal of dark chocolate’s perceived health benefits, which align with the lifestyle preferences of wealthier (and healthier) demographics. Given this it is unsurprising that in the study under discussion participants consuming ≥5 servings/week of milk chocolate were twice as likely to be current smokers than those reporting ≤1 serving/week. “Conversely for dark chocolate it was those reporting ≤1serving/month who were most likely to be current smokers. Whilst current smoking was relatively uncommon in the study population, the point is there will clearly be a large difference in many health-related factors that are not measured. This cannot be taken into account with a handful of confounders, and measured with error, as it is simple to demonstrate3.
“The repeated failure of RCTs that test hypothesis from these studies, mentioned above, confirms that this is the case; indeed a large RCT has failed to demonstrate any protection at all from coco extract supplementation in type 2 diabetes4.
“The paper could provide excellent teaching material in elementary epidemiology classes, it demonstrates so many problematic aspects in much current population health research. It is yet another contribution to “Today’s random medical news”.”
1 https://www.youtube.com/watch?v=8IgpTT5ZXXU&t=2s
2 https://www.mdpi.com/2071-1050/12/14/5586
3 https://academic.oup.com/aje/article-lookup/doi/10.1093/aje/kwm165
Professor Baukje de Roos at the Rowett Institute, University of Aberdeen said:
“This is a first very large-scale analysis of an association of chocolate intake and self-reported Type 2 diabetes across three prospective cohort studies in the US. The distinctiveness of this research is that they specifically assessed which type of chocolate was linked to diabetes risk.
“As dark chocolate has a much higher cocoa content, this means that bioactive compounds in cocoa, and in dark chocolate, such as flavanols, may contribute to the lowering of diabetes risk, possibly by increasing insulin sensitivity. However, if and how flavanols on their own reduce diabetes risk remains to be established – a recent large scale randomized controlled trial (the COSMOS study) found that cocoa flavanol supplements did not affect the risk of type 2 diabetes.
“The authors found that milk chocolate consumption was associated with weight gain, whereas dark chocolate intake was not. Weight gain is an important risk factor for type 2 diabetes development, so it is important to control for this in the analysis, which is something the authors did.
“Interestingly, the inverse association between eating dark chocolate and the risk of type 2 diabetes was mostly observed in those that were younger than 70 years old, and the association was stronger in men than in postmenopausal elderly women. Other studies have found that chocolate consumption lowers the risk of type 2 diabetes in men but not in women, and in one of our own studies we found that eating dark chocolate improved platelet function in men but not in women. It is unlikely that sex differences in absorption or metabolism of the cocoa flavanols are responsible for the differences in observed effects between men and women. It is more likely that sex hormones may differentially modify the association between chocolate consumption and the risk of type 2 diabetes, or platelet function.”
‘Chocolate intake and risk of type 2 diabetes: prospective cohort studies’ by Lui et al. was published in British Medical Journal at 23:30 UK time on Wednesday 4th December.
DOI: 10.1136/bmj-2023-078386
Declared interests
Prof Kevin McConway “None”
Prof George Davey Smith “None”
Professor Baukje de Roos “I don’t have any conflict of interests to declare in terms of consultancies or partnerships. I do have research projects that are funded by industry, but none related to chocolate/cocoa research.”