select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A study published in European Heart Journal looks at air pollution levels, physical activity and risk of cardiovascular disease in young adults.
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“This study does raise the interesting possibility that where you do exercise, in terms of the air pollution there, might affect what happens to your health if you change how much exercise you do, but there are several problems with the interpretation of the results. The researchers mainly end up concluding that more research is needed to identify what’s going on, and I’d certainly agree that more needs to be found out before any changes in recommendations on exercise could be made. I’m certainly not saying that it’s impossible that pollution levels could have an effect on what exercise does to the body, indeed it would be unlikely if there were no such effects at all – but this study has not really been able to clarify what exactly is going on.
“The research involved asking a large number (nearly 1.5 million) young South Korean adults, aged 20-39, about how much exercise they did, on two occasions two years apart. The participants were then followed up for six years, and the researchers recorded if and when any of them had a cardiovascular disease (CVD) event (heath attacks, strokes, etc.) that was serious enough to put them in hospital for at least two days. The researchers also recorded two measures of air pollution close to where each participant lived. These standard measures are of the levels of very small particles in the air, called PM2.5 and PM10. The smallest particle sizes, measured by PM2.5 are generally considered to be most dangerous to health, but PM2.5 measurements were not available for all participants, only those who lived in three very large cities (but PM2.5 data were nevertheless available for almost a million people). The researchers wanted to investigate associations between the amount of exercise people took and their risk of CVD, and to see whether that association varied with the air pollution level. That kind of study has been done before – but what makes this new research interesting and (I’d say) more difficult to interpret is that the researchers’ focus was on changes in the amount of exercise taken on the two occasions, on how these changes might be associated with CVD risk, and on whether any associations varied with the pollution level.
“There’s an overall and unavoidable issue stemming from the fact that the study is observational – that is, the researchers did not aim to change or manipulate what the participants did, but just recorded what happened. That’s inevitable, particularly in relation to air pollution, because you can’t just make people move to areas with different air pollution levels. But it means that people will differ in many other ways than the ones that the researchers are interested in. People who do different amounts of exercise, or live in areas with different pollution levels, will vary in other ways than their exercise and pollution levels. Any differences in CVD risk between groups of people might actually be caused by these other differences, and not by exercise or pollution at all. It’s possible to make statistical adjustments to try to allow for these other differences, and these researchers did that in a sensible way for many quantities that could affect CVD risk, levels of exercise, the pollution levels where they lived. However, there’s no way of knowing whether these adjustments have dealt with everything important, and that’s why the press release points out that conclusions can’t be drawn from this study on what causes what. That applies to all observational studies, but perhaps particularly so when (like this) something relatively new is being studied, because of the concentration on changes in exercise levels. Also, people would very often have some reason for changing exercise levels, any differences in CVD risk with the amount of change in exercise levels could be caused by these reasons and not by the effects of the change itself, and it’s certainly not clear to me whether the researchers made statistical adjustments that might compensate for this possibility in relation to changes in exercise.
“In addition to this overall point about what causes what, there are some statistical issues. One is that almost half the participants did not change their exercise levels much. The researchers divided exercise levels into four groups (physically inactive, relatively low level, in the minimum recommended range, higher than the minimum recommended range), and almost half the participants were in the same group on both occasions when they were asked. Data on people who don’t change their exercise level is important as a basis for comparison with people who do change, but this does mean that less data are available to measure the effect of changes, particularly the largest changes in exercise levels. This may be related to a point mentioned in the press release, and the research report, that several of the comparisons in which the researchers were particularly interested were not statistically significant. That means that, while there was generally some difference between the groups being compared, it was of a size that was consistent with there being no real underlying difference, so that the observed association arose just because of chance variability. We’ve got to be a bit careful here – just because a difference could plausibly be nothing more than chance, because it’s not statistically significant, that doesn’t mean that it is definitely only due to chance. But this does make the findings considerably less persuasive. There were some statistically significant findings, but not many.
“A related issue is that the researchers carried out quite a large number of statistical tests to see which results might go beyond chance levels. Statistical tests can’t possibly give totally definite results – there’s always a chance that a result will be statistically significant even though there’s really no underlying difference, or indeed that it will not be statistically significant even though there really is a difference. But, other things being equal, the more statistical tests one does, the more likely it is that one or more of them will have a significant result, even if there aren’t any true differences at all. Adjustments can be made to allow for this consequence of doing many tests, but the researchers do not report that they made any such adjustments. That’s not necessarily wrong, but it’s another reason to be cautious about the findings.
“But anyway statistical significance is only part of the story, and not the most interesting part really. It’s usually more important to consider how big any effects might be. An important issue is that the people involved in this study were young (20-39) when the study began, and it’s not common for young or early middle aged people to have cardiovascular disease (though obviously it’s important if they do). Overall in the study there were 8,706 CVD events in 8,779,364 person-years of follow-up. That means that, on average, in a group of 10,000 people like those in the study, just under 10 of them would have a CVD event in a year – roughly one in a thousand. So we aren’t talking about a major public health risk here. Given that the length of follow-up was six years, in a group of 10,000 people like those in the study, there would be about 60 CVD events during follow-up.
“The paper and the press release report that, among the people who lived in an area of high PM2.5 air pollution who said they did no exercise on the first occasion but, two years later, said they had moved to the highest level of exercise (at least 1000 MET-min/week), the risk of a CVD event was increased by 33%. People that increased their exercise that much were not common in the study – on the basis of the numbers in Table 2, it looks as if only about 1 in 13 of the people who weren’t exercising at all on the first occasion increased to the highest group two years later. So they are probably not at all typical, and the adjustment for factors that might be confusing the picture wouldn’t necessarily work well for them. But, putting that to one side, I think the figure in the press release for the numbers affected (“an extra 108 people per 10,000”) is not correct. For 10,000 people living in a high PM2.5 area and doing no exercise on the first occasion, the number of CVD events expected in one year would be about 13, on the basis of data in Table 2 of the report, so about 75 over the period of follow-up. (That’s higher than the 60 average across everyone in the study, presumably because it isn’t good for one to do no exercise at all.) If that increases by 33% in people who have increased their exercise to the highest group by the second examination, that would lead to about 25 more CVD events during the follow-up period, by my calculation, not 108. I may have misunderstood, and an extra 25 CVD events is not good for the people affected – but it is 25 extra in 10,000 people, so not at all a large increase in risk. And there is a lot of statistical uncertainty about the 33% anyway – anything between about an extra 60 events during the six years, and about 3 fewer events, is consistent with the data. (It’s just about plausible that there could be fewer events in the people who increased their exercise this much, because the comparison isn’t quite statistically significant.) So a rather uncertain size of increase, that might just about not even be an increase at all, but quite a small increase in rather a small risk.
“I think there is a similar issue with the estimate in the press release of 49 for the reduction in CVD events in 10,000 people who move from no exercise on the first occasion to the highest group on the second occasion, but live in an area where the PM2.5 pollution is low or moderate. Over the follow-up period, I calculate the reduction as about 17, with a range, taking into account the statistical variability, from a reduction of about 30 to an increase of about 2. These are again small changes in a fairly small risk, and it’s not clear how important they are in public health terms as well as in statistical terms.”
Prof Alun Hughes, Professor of Cardiovascular Physiology and Pharmacology, Head of Department of Population Science & Experimental Medicine, and Associate Director of the MRC Unit for Lifelong Health and Ageing, UCL, said:
“The press release does not accurately reflect the science. Namely, it begins:
‘Physical activity is important in preventing heart and blood vessel disease in young people so long as they don’t undertake very strenuous activity on days when air pollution levels are high…’
“The first part of this statement relating to the benefits of physical activity is uncontentious and supported by prior evidence; however the second part of the statement is unwarranted. The current paper didn’t look at the impact of taking exercise on days when pollution was high or low; it compared the effect of physical activity changes in relation to people’s annual average cumulative level of exposure to particulate matter (PM). Its results therefore relate average annual air pollution levels, and, in my view, should not be extrapolated to predict the potential impact of daily variation in air pollution and its possible modification of the relationship between physical activity and cardiovascular risk.
“The press release also gives prominence to a 33% increase in hazard ratio when comparing individuals who most increased their physical activity (from 0 to >1000 metabolic equivalents (METS)) with those who remained inactive (0 METS on both occasions). The 95% confidence interval of this estimated hazard ratio was wide (0.96 to 1.84) and includes the null. The findings are therefore compatible with no association as well as a moderate increase in relative hazard. While the observation is interesting, the interpretation is insufficiently cautious (in the press release or the paper). The use of a hazard ratio will tend to exaggerate the apparent importance of risk of a rare event. As would be expected in such a young cohort the rate of cardiovascular disease events was low (approximately 1 event in 1000 person years).
“It is unclear what information and assumptions were used as a basis for the statement in the press release that ‘This means that an extra 108 people per 10,000 might develop cardiovascular disease during the follow-up period.’ This estimate does not appear in the paper.
“In my view it was inappropriate to adjust hazard ratios for potential mediators (e.g. blood pressure, body mass index) – inclusion of a factor that is potentially on the causal path linking the exposure to the outcome will tend to result in an underestimation of the true association.
“The choice of dichotomisation into low and high pollution groups seems rather arbitrary (it was selected on the basis of the value of the 66th percentile [3rd tertile]). Both resultant groups were exposed to annual average levels of air pollution that would generally be considered excessive (low/medium group PM2.5 (annual mean) = 24.11 (1.75) μg/m3 [mean (standard deviation)] and the high group PM2.5 (annual mean) = 29.70 (2.29) μg/m3. Evidently, the difference between the annual average exposure to PM2.5 in the two groups is quite small in percentage terms.
“Overall, the study provides some weak but inconclusive evidence that high levels of air pollution might attenuate or even reverse the benefits of a moderately large increase in physical activity. I think the conclusions in the press release tend to exaggerate the findings and extrapolate them beyond what is reasonable.”
Prof Anna Hansell, Professor in Environmental Epidemiology, University of Leicester, said:
“This follow-up study was conducted in South Korea. It analyses 1.5 million health records for younger people aged 20-39 years, who also had had health checks where they were asked about physical activity. It relates this information and data on particulate air pollution levels to risk of cardiovascular disease.
“The study shows that increasing physical activity reduces risk of cardiovascular disease when annual average particulate air pollution levels are low. UK annual average particulate air pollution levels would be in the ‘low’ category. Similarly, decreasing physical activity levels increases risk of cardiovascular disease.
“The study is harder to interpret when air pollution levels are high – which here is at least double the UK annual averages for particulates and also higher than our current Air Quality Standards. The study does find some suggestion that increasing physical activity levels when fine particulate PM2.5 levels are high (as in more than double the UK annual average) may actually be harmful for cardiovascular health. However, the study didn’t find a statistically significant effect when using a different measure of particulates, PM10. One issue affecting interpretation is that people may reduce their exercise levels when air pollution is high and the reduction in exercise alone is what is having the effect on cardiovascular disease.
“There are some good reviews of studies of air pollution and physical activity, which suggest that in healthy adults the benefits of exercise much outweigh adverse effects of air pollution except at very high levels of air pollution. Short-term studies in those with pre-existing disease suggest that the benefits of exercise on the body may be weaker or not seen if exercising at high air pollution levels, such as areas with very high levels of traffic.
“This and previous studies all confirm that reducing air pollution levels is beneficial for health and allows us to maximise the benefits of exercise on our cardiovascular health.”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
‘Association of the combined effects of air pollution and changes in physical activity with cardiovascular disease in young adults’ by Seong Rae Kim et al. was published in European Heart Journal at 00:05 UK time on Tuesday 30th March 2021.
doi:10.1093/eurheartj/ehab139
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. My quote above is in my capacity as an independent professional statistician.”
Prof Alun Hughes: “No conflicts of interest”
Prof Anna Hansell: I have no conflicts of interest to declare.