select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A study published in the European Journal of Population looks at the well-being of adolescents conceived through medically assisted reproduction.
Prof Carlos Calhaz-Jorge, Retired Full Professor of Obstetrics & Gynaecology, Faculdade de Medicina da Universidade de Lisboa, Portugal, said:
“This is a quite interesting and important topic centred in a population not yet studied and the authors deserve all congratulations for the huge work. In spite of the relevance of raising our attention to this possible issue, some points must be stressed:
1) The results refer to all medically assisted reproduction (MAR) techniques and not to IVF exclusively, as erroneously stated in the UCL press release. MAR includes ovulation induction (hormonal treatment for women that do not ovulate and get pregnant after intercourse), intrauterine insemination, IVF and ICSI.
2) The methodology used, based on administrative register data, has the weaknesses recognized by the authors and, according to them, the relatively small differences found “need for a cautious interpretation of the associations”. Even using the within-family evaluation, many crucial variables in the mental health dimension could not be considered (the authors state that they were “not able to control for potentially important factors not shared by siblings, such as personality, cognitive skills, or differences in parent-child relationships”).”
Dr Simon White, Senior Research Associate at the University of Cambridge, within the Department of Psychiatry and Medical Research Council Biostatistics Unit; and Statistical Ambassador for the Royal Statistical Society, said:
“The paper provides no direct evidence that IVF, or medically assisted reproduction (MAR) more broadly, cause differences in mental health problems at adolescence. All the analyses presented show associations rather than direct causation. The within-family analysis attempts to address this, by directly comparing siblings, and yet none of the findings (in Table 3) for the within-family fully-adjusted model (Model 4) are significant; hence once accounting for all the environmental differences available and controlling for within-family effects there are no statistically significant associations with an increase in any (or specific) mental health problems.
“The paper, and press release, do not present the uncertainty around many of the findings. In fact, there are very few probabilities mentioned within the text (nearly all the key findings are left to the tables). The uncertainty, shown in Figure 2 is substantial even for the key finding: Figure 2 any disorder between-family appears under fully-adjusted model shows the difference to be 0.1-1.4 (reading roughly from the figure), which puts the key finding of a one percentage point difference (really 0.693) into context. Further, the authors admit there are many aspects of family life and development that they do not observe or have any data on (for example parenting style, mental health of the parents, or aspects of the environment that change over the 16-18 years of development).
“Mental health problems have been summarised as four yes/no categories and combined into a a single “any yes/no” category. This is a sensible research question, however, mental health is a complex spectrum and the severity, duration, and other aspects of treatment have important impacts on the mental health of the individual and their family – not all four possible categories are equal in this regard. There is a question as to whether the authors should account for multiple testing within their analysis. They have five different outcomes, in statistical terms they have conducted multiple tests and then a combined outcome (it is unclear what effect this would have on the statistical significance of their findings).
“Potential issues in the paper:
(1) As an aside, I have a query around the results presented in Table 3. The footnotes indicate the standard errors are in parentheses, but a standard error must be positive and these numbers are negative? Either a serious typo or the table does not present the findings as I expect – meaning I’m unsure how to read the findings.
(2) The press release states the key numbers as 10% and 9% for the proportion of adolescents with mental health issues, for MAR and natural respectively. These numbers are not directly presented in the paper: Table 1 gives the unadjusted proportions and Table 3 only presents the difference. I assume they are summarising the difference for any mental health problem in the fully-adjusted between-family (Model 2), which is 0.693 (rather than a difference of one percentage point as stated in the press release, rounding 0.693 up to one).”
Prof Alastair Sutcliffe, Professor of General Paediatrics, UCL, said:
“The headline of the press release is misleading – the only alleged association with mental health with medically assisted reproduction occurred in the siblings of naturally conceived children, overall a very small proportion of the alleged study group. Furthermore the hard outcome of whether or not the adolescents took antidepressants in this sub group was non existent (i.e. there was no difference).
“In my view this is not high quality research – although the authors have used a population dataset, and they did control for confounding factors to some degree, their hypotheses were not supported at all by existing studies, which are extensive on assisted reproductive technology-conceived children.
“The authors have used proxies as measures of mental health rather than direct measurements – and the most powerful proxy looked at (use of antidepressants), which is a more reliable measurement, showed no difference between IVF-conceived and other children. In my view the ‘top line’ finding is not backed up by solid data.
“As it stands there are no real-world implications of this study, and it would be absolutely wrong to interpret this study as suggesting IVF leads to mental health problems in teenagers.
“The population of individuals conceived by ART (assisted reproductive technologies) increases year on year world wide. In this study the results were broadly reassuring, showing mental health was little different than the comparison group. A small difference in those who had naturally conceived siblings in some weak proxy measures of mental health would be most likely explained by residual confounding, meaning that other factors could have been behind this small difference, rather than the IVF itself.”
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“This is a potentially interesting new piece of research, using data from over 280,000 Finnish children born between 1995 and 2000 – the great majority of all the children born in Finland during that period. As in other Nordic countries, information is available there for public health research from comprehensive population registers. However, this is an observational study, and rather a complicated one, so there are problems in identifying what causes what.
“What’s more, the findings aren’t as precise as you might expect from the large number of children involved. That’s because what really matters statistically is not how many children there are in total, but how many had the poorer social and mental health outcomes, and, particularly amongst the children conceived through medically assisted reproduction (MAR), some of those numbers are not very large. And the differences in the risks of many of the outcomes, between children conceived by MAR and children conceived naturally, are mostly pretty small.
“The researchers looked at data on educational and social outcomes at age 16-18. Most were educational – school grades, whether the child went into academic or vocational education, whether they dropped out of school, whether they ended up with NEET status (Not in Education, Employment or Training), and, though this isn’t primarily educational, whether they left the parental home before the age of 18. They also looked at measures of mental health – whether the adolescents had had an antidepressant prescription, whether they had received in-patient or specialized outpatient care for various mental disorders, and whether there were records of high-risk health behaviours like some serious drug or alcohol use and self-harming.
“The researchers made, broadly, two different comparisons on these measures between children conceived by MAR and children conceived naturally. One comparison was in the whole group of children. The other compared used only data from families where there was at least one child conceived by MAR and at least one conceived naturally. The snag in an observational study like this is that there are bound to be many differences between the circumstances of children conceived by MAR and children conceived naturally. In this study, among other things, the researchers found that children conceived by MAR tended to have more advantaged families – higher income and higher parental education. Also their mothers were, on average, older at the time of birth, and less likely to have smoked during pregnancy. And children conceived by MAR were considerably more likely that the naturally conceived children to be the first-born in their family.
“Any difference in the social or mental health outcomes between the two conception types might be caused, in whole or in part, by one or more of these other differences (called potential confounders, in the jargon), and not by the difference in conception type. So it’s not possible to be certain about what is causing any differences in social or health outcomes. Of course the researchers were aware of this, and they made statistical adjustments to allow for several potential confounders, such as the child’s sex and birth year, where they were in the family’s birth order, the mother’s age, the family’s income and the level of parental education, and more. They also dealt (in a different but appropriate statistical way) with differences in birth weight, in prematurity, and in whether the child was part of a multiple birth (such as being a twin).
“But you can’t make adjustments for potential confounders on which you have no data, and one never has data on absolutely everything that could be relevant. So you can never be sure from a single observational study, like this, what is causing what. One aspect that can sometimes clarify matters to a certain extent is to work out details of the way in which the factor that you are interested in could actually affect the outcome. The research report mentions several possible mechanisms that might lead to differences in social or mental health outcomes between the different conception groups – possibly some long-term consequence of the medical treatment itself; possibly higher levels of parental stress stemming from subfertility and dealing with the MAR process, leading to different relationships between the child and the parents than with naturally conceived children; perhaps socio-economic differences between the parents of children conceived by MAR and others (though this was allowed for in this study, to some extent, by the adjustments). The researchers go into more detail about how relationships within the family could have led to differences in mental health outcomes between the two conception groups. In summary, though, this study could throw very little light on which, if any, of these mechanisms might actually have operated in the families they studied. It certainly can’t establish, for instance, that the differences were directly caused by the bodily effects of the fertility drugs that the mother took. So it’s very unclear, on the basis of these findings, what public health action, if any, might be most appropriate.
“There’s an interesting and potentially important point arising from comparing the results of the two different sorts of comparison that the researchers made. Other things being equal, you might expect their within-families comparison to be more precise than the comparison between all children conceived by MAR and all conceived naturally. That’s because the child conceived by MAR and the child conceived naturally in the same family will share the family environment, so any effect of general family environment on the outcomes will cancel out when their outcomes are compared. But here, other things aren’t equal – the comparisons between all the children regardless of family involved about 210,000 conceived naturally and about 10,000 conceived by MAR, while the within-family comparison involved only about 1,300 children of each conception type. The much smaller numbers in the within-family comparison means that, statistically, the differences in the outcomes are measured less precisely.
“But, that said, there remain some major differences between the two sets of comparisons, that seem to go beyond the question of statistical precision. On most of the outcomes, the estimated differences between the two conception groups are considerably larger in the comparison involving all the children than in the within-family comparison – though it’s not possible to be certain that that is the position, because of the statistical imprecision in the within-family estimates. There are a couple of exceptions to this, which I’ll come to. But it did raise in my mind the question of why there would be a difference between the two sets of comparisons. There are some differences in the detail of the statistical analysis, so it could be that, but I think it’s possibly more meaningful and important. There’s a potentially important difference between the two sets of comparison that is inevitable. The reason why there are so many fewer children, and indeed fewer children conceived by MAR, in the within-family comparison is because a family can be included only if it includes at least one child conceived by MAR and at least one conceived naturally, and both were born between 1995 and 2000. So any family that consists of an only child (or twins, born first in the family), or contains only children conceived by MAR, or only children conceived naturally, can’t be in the within-family comparison. Nor can a family where there are siblings in both the conception groups but not both born in 1995-2000. My concern is, in particular, that only children may well be more prevalent than usual (for Finland) amongst those conceived by MAR, and that being an only child was not adjusted for in the statistical adjustments, and that the family relationships may work differently in families with an only child and hence lead to different outcomes. I don’t know that this is the reason for the difference between the findings of the two comparisons, and I suspect that the researchers did not have full data on whether a first-born child was in fact the only child (given where they obtained their data), so could not make an adjustment.
“After the statistical adjustments, there were differences between children conceived by MAR and conceived naturally in the overall sample, after the statistical adjustments, in their grade-point average, whether they were on the academic or vocational education track, and in the chance of leaving home early. There were also some small but statistically significant differences in some of the mental health outcomes. There were no clear differences in the within-family comparison – that’s partly because the numbers were smaller, but also, quite possibly, because the differences were a lot smaller. If it’s because the differences are smaller, that could possibly be because there is one or more potential confounder operating in the overall comparison, but not the within-family comparison, that was not adjusted for. It could be whether a family has an only child, or it could be something else, but the difference between the two sets of findings makes it clear to me that we can’t even be sure that there are true differences at all between the two conception types in terms of the outcomes used in this study. The evidence for some differences in some of the mental health outcomes is slightly stronger, in that there’s a larger difference in the within-family comparison than in the overall comparison, particularly on anxiety and depression, and though it could be simply due to chance in that comparison, it does show up in the overall comparison too.
“It’s important to point out, in any case, that the differences between the two conception groups are mostly very small. In the overall comparison, after adjustments, the children conceived by MAR had a higher grade-point average and were more likely to be on the academic education track, and were less likely to have left home early, but only by very small amounts (between one and two percentage points for academic track and early home-leaving). They were more likely to have some of the adverse mental health outcomes, but the difference in the risk of these outcomes was less than one percentage point. In any case those differences aren’t necessarily caused by the difference in conception method – there could be confounder that weren’t adjusted for – but even if they are in fact caused by the conception difference, are they big enough to cause concern? And how might something be done about them, given that we don’t know how the cause works even if it is real? Given all the other worries and concerns for parents going through medically assisted reproduction, is this something to add to the list of things to worry about? I have to say my feeling is that it isn’t.”
‘The well-being of adolescents conceived through medically assisted reproduction: a population-level and within-family analysis’ by Hanna Remes et al. was published in the European Journal of Population at 00:01 UK time on Tuesday 28 June 2022.
Declared interests
Prof Carlos Calhaz-Jorge: “Clinician performing MAR treatments.”
Dr Simon White: “No conflicts.”
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. My quote above is in my capacity as an independent professional statistician.”
For all other experts, no reply to our request for DOIs was received.