May 13, 2024

expert reaction to global observational study of internet use and wellbeing

A study published in Technology, Mind & Behaviour looks at internet use and wellbeing.

Dr Nina Di Cara, Senior Research Associate, University of Bristol, said:

“This study used a large and globally diverse sample to analyse the relationship between well-being and internet use. The size and diversity of the sample makes it a very valuable contribution to this research area. 

“It is important to note that ‘internet use’ is a broad term that has a huge number of interpretations, and that this has led to conflicting evidence and confusion about the relationship between internet use and well-being. Here, internet use primarily refers to internet access, rather than what participants were specifically using the internet for. This distinction is important when considering the results. 

“It is also important to note that this study is not causal, and so can not claim that internet use causes positive well-being. To do this we need longitudinal research, which the authors explain.  

“As the authors highlight, well-being is also a complex and wide-ranging construct that can be measured in many ways. Several different well-being constructs were measured in the data used for this research, which is a strength of study, and consistency in the results across the different well-being measurements used gives more confidence in the results even though these were not measured using tools that academic researchers would typically consider ‘validated’. 

“The study uses a multiverse analysis which allows researchers to establish whether specifying the variables used in the analysis in a slightly different way would change the results. The consistency in the results found in the multiverse analysis also gives confidence that the positive effect they found between well-being and internet use is robust. 

“Overall the study is important because it provides robust evidence for the global picture of the relationship between access to the internet and psychological well-being. However, the evidence we need to make meaningful interventions and policy recommendations requires longitudinal research, triangulation across different methods, and a better understanding of what type of internet use is works for different people, and when.”

Prof Kevin McConway, Emeritus Professor of Applied Statistics, Open University, said:

“There has been endless debate and considerable speculation on the possible effects of internet use on well-being, in general across all ages, but more specifically in relation to children and young people. This interesting new study certainly extends the available information beyond simple speculation and beyond previous studies that used participants mostly in relatively rich Northern countries. But, as the researchers make very explicit, it doesn’t (and couldn’t possibly) make everything clear. The researchers wielded a very broad brush. It’s a starting point, and if nothing else it casts very serious doubt on the view, held by some people, that the Internet is bad for us all.

“There are several strong points of these new findings. First, the data source covers a huge number of participants (over 2 million), from 168 countries across the world, and across several years. And the data analysis works at the level of individual participants, not at the level of averages of internet in whole countries.

“The statistical analysis is, in my view, sensible and generally appropriate. It takes into account, as far as it can, the inevitable shortcomings of the available data.

“Those shortcomings are quite marked. The study is observational. So the positive associations between the amount of internet use and the measures of well-being could have arisen because the internet use somehow caused better well-being, at least in part, but they could also be caused by other factors that happen to be associated, separately, with both internet use and perceived well-being. You just can’t tell for sure in this kind of observational study. That’s why the researchers, rightly, don’t talk in terms of cause and effect but only in terms of association and prediction. (Predictions can still be accurate even if the pattern of cause is complicated and doesn’t directly involve the quantities that happen to be correlated – as long as the causal associations that do exist have not changed much.)

“It’s very common, in working with data from observational studies, to make statistical adjustments to try to allow for other factors that might be involved. However, that doesn’t deal with the issue of cause and effect, because researchers won’t have data on every possibly relevant quantity, and you can’t adjust for something on which you don’t have data.

“These researchers used an interesting approach to dealing with this kind of adjustment, and at looking at associations in subsets of the data (for example, for a specific age group). They have used this approach before in other studies. It’s not seen very commonly, though I think it’s very useful.

“In any statistically-based research, choices have to be made by the researchers on which factors to adjust for, which subgroups to look at, and so on. But typically only one set of choices is published, or just a few sets of choices. Instead, in this research, the researchers considered a huge range of almost 34,000 different sets of choices.

“So they can say that a large majority of those sets of choices showed a positive association between internet use and well-being, and they can also identify some groups of participants which appear not to follow that general pattern. This doesn’t by any means answer all the interesting questions about cause and effect, but at least it makes it clearer what the important questions may be. Then, as the researchers propose, further work can and should be done to look more closely into these potentially important details.

“The data source is, essentially, cross-sectional – that is, the level of internet use and the levels of well-being for each individual are measured at the same time. This means, for instance, that the data is consistent with higher internet use, in individuals, being caused (in part) by higher levels of well-being, as well as with the higher internet use causing (in part) the higher well-being – or indeed there being no cause and effect relationship between them. You just can’t tell.

“In order to find out more about this, we’d need what’s called longitudinal research, where the same individuals are followed up over a period of time, with changes in their level of internet use and in well-being over time being compared. The researchers again propose that this should be done.

“The data source that they used for this study does at first glance appear to be longitudinal, because it covers the years 2006 to 2021, but it does not follow up individual participants from one year to another over this period, so it isn’t longitudinal in the important sense here.

“A key feature of the statistical modelling used in this study is that its measures of difference in well-being between internet users and non-users are estimated separately for each country. That is, they compare the well-being of internet users in a country with the well-being of non-users in the same country. I think that makes good sense, since country average levels of internet use and of well-being do differ a lot between countries – but it means that some possibly interesting comparisons between countries can’t be made. However, the patterns of difference in well-being between users and non-users of the Internet are anyway extraordinarily consistent between countries.

“The researchers found that for girls and young women (aged 15 to 24), there was evidence that the association between internet use and well-being generally went in a different direction than in pretty well all the other subsets of participants. That’s potentially concerning, because if it’s a valid pattern, it could possibly have arisen because internet use in females in this age group is, on average, associated with lower well-being, rather than with higher well-being as in other age groups and in males. 

“But again, from this study we can’t be sure of the pattern of cause and effect – for example one possibility remains that young women who do not have high levels of perceived well-being chose, as a consequence, to use the internet more. And this pattern in any case applies only to one of the eight different well-being measures that were used in the research (the one that measures ‘community well-being’) and to the measure of the amount of internet use, not just whether the young women in question had access to the internet. That particular measure of well-being was available in the database only for a few years (2013 to 2015) – maybe things would have been different in earlier or later years, but we can’t tell because the data does not exist. So, while the researchers are right to point out that this finding needs more investigation. I wouldn’t by any means describe it as a red flag about Internet use.

“Finally, it’s important to understand that none of the well-being measures used in this research has been properly validated by experts in psychological measurement. That doesn’t mean they are no good – they were used in the original data source for reasons other than being validated measures. But I certainly do not know of another global source of data on these things that is properly validated. As a researcher, you have to work within available resources, and these researchers are admirably clear about the limitations of their data source.”

Professor Pete Etchells, Professor of Psychology and Science Communication, Bath Spa University, said:

“To my mind, this is one of, if not the, largest studies of its kind: data from nearly 2.5 million people, from 168 countries, was analysed in order to assess the impact of using the internet on wellbeing on a truly global scale. This is a question that is both timely and important; societally, we are increasingly concerned about the potential detrimental impacts that screen-based technologies might be having on us.

“One of the great strengths of this study is that it acknowledges the global penetration of internet and digital technology use; to date, the vast majority of studies on the psychological impacts of digital tech are based on convenience samples in Western populations. On that global scale, the results suggest that access to the internet is generally a good thing, with associations with greater life satisfaction and wellbeing.

“The study moves beyond that worldwide level though, to look at potentially varying impacts on different subgroups of people. Here for example, the authors note some clear negative associations, specifically for women 15-24 years of age, between internet use and community wellbeing – that is, wellbeing attached to liking where you live, and feeling safe there.

“As with many studies in this area, we are looking at associations – correlations, effectively – that tell us very little about any causal links. Moreover, broad-level definitions of internet access don’t get to the specifics of types of technology use that we particularly worry about. That being said, the fact that the top-line finding from this study is that we’re not seeing drastic declines in wellbeing across the board should give us pause to think about how we’re framing current conversations about the impact of tech use. It’s also clear to me that we need to start thinking really hard about how move this field beyond correlational analyses alone.

“In a way, the most important aspect of this study comes in the final paragraph: the authors acknowledge that in order to overcome the persistent limitations that we find in these sorts of studies, we need to have access to data from digital technology companies. These are data which already exist, and which could really get to the heart of understanding how our relationships with digital tech impact and influence our mental health.”

‘A multiverse analysis of the associations between internet use and well-being’ by Matti Vuorre and Andrew Przybylski et al. was published in Technology, Mind and Behaviour at 00:01 UK time on Monday 13^th May.

DOI: https://tmb.apaopen.org/phttps:/tmb.apaopen.org/pub/a2exdqgg/ub/a2exdqgg/

Declared interests

Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee.  My quote above is in my capacity as an independent professional statistician.”

Prof Pete Etchells: is the author of Unlocked: The real science of screen time (and how to spend it better) and Lost in a Good Game (Why we play games, and what they can do for us).

For all other experts, no reply to our request for DOIs was received.

This Roundup was accompanied by an SMC Briefing.