Here’s a popular press report of a study based on actual experimental data that shows no link between clinical depression and usage levels of Facebook. Fine, it’s a study with real data (unfortunately the fluff articles at Huffington Post are not actual journal reports with the real data to look at, so it’s tricky evaluating the study with such skimpy information). But, this is the kind of information the public gets since they rarely read actual journal articles, esp. as journals still insist on hiding their articles behind paywalls.
Now I thought, even without the journal article, I might be able to evaluate the reasonableness of this finding. This turned out to be a difficult challenge. From the sparse information in the article the number of subjects was: a) rather small, b) only from a narrow band of ages, c) no other demographic information was provided, and, d) no apparent control group or inference from other studies as a control. In particular, item a indicated that, assuming even distribution, there would only be about 60 individuals in each usage level (high, medium, low – not defined in popular press). So what would be a statistically significant finding for that sample size?
So I tried to find some incidence data on depression and right away you run into the issue of “what is depression?” or “what subtype of depression”. I found few good sources and answers all over the map for what the levels of depression is in the general population (and this itself seems to be dependent on geography (for instance, U.S. is higher than Japan, but knowing Japan I think that could be a reporting difference since any mental distress is more shameful in Japan), dependent on age (although the reports vary, either it’s most common in 30-40 or older people vs ???, or another report that shows it more common in teens (but that gets back to what kind, the report that claimed 25% in teens was saying “showing a symptom” and at least once in teen years – oh, terrific, what about something more solid, like actually diagnosed by a professional for n% in a given year (not the vague sometime). And then there are gender differences, socio-economic differences, and some hints at ethnic background differences. So, somewhere between 3% and 25% seems to be the range of 2-20 people in each of the groups – of that what would be a statistically significant variation. No idea, since the popular article had none of this data.
So, basically, ugh. I know the Net is definitely not a “reference” and so information you find will always be spotty, but frankly this is the worst I’ve ever done trying to get my hands on some hard numbers. I’ve looked at a lot of medical data, a bunch of psychopharmacology data, economic data, other technical data, and this is the mushiest search results I’ve ever found. That makes it difficult to evaluate this report about link of depression and Facebook.
But here’s a few interesting points: a) it appears to be consistently true that woman are more likely to be depressed (again, exactly what does this mean?, hard to tell) and simultaneously women are slightly more likely to be users of some of the social media – connection or coincidence, no idea since I couldn’t find any data on this. But I’m even a little skeptical of the initial part, higher incidence in women since I can explain some of that away by the cultural issue that men are less likely to report (at least openly) various mental distress that might be perceived as weakness or having adverse effect on job performance. b) reports of teen depression are definitely less clearly professionally diagnosed (vs more just anecdotal stuff), but perhaps that’s another reporting bias that adults are more likely to actually seek professional help and therapy (that report of 30-40 band is also interesting, because this is the first divorce and beginnings of middle age crisis crowd), c) symptoms of depression vs diagnosed “major depression” – give me a break, look at any of the depression tests on the Net or even more serious psychiatric sites for symptoms and show me any person that probably hasn’t felt a couple of these – diagnosing depression, as an actual clinical condition is tough, and asking a bunch of young adults if they’re depressed about something is about as useful as asking them if they’re sexual satisfied, whatever answers you get, IMHO, are likely to be highly inaccurate.
Now you might say, well, you have zero background in this area so who are you to say. Absolutely, I agree completely. The trouble is this is a relatively difficult area to actually find any “experts”, any consistent definitions, or any consistent methodology, so frankly it’s guessing (I sure hope the literature behind paywalls is a lot better than the stuff publicly available). If you don’t believe me, here’s a good example. This article, published by presumably a reputable organization (American Academy of Pediatrics) is, well, not to put to fine a point on it – mush. It doesn’t supply any data or even really a hint the conclusions are even based on data. So how can someone like me actually get the facts. And besides, I believe firmly that all of us should be a bit skeptical because there is a lot of wrong information out there (esp. all the fake denialist stuff pretending to be science). So if someone like me can’t make some sense of this who (in the public) can.
And here, if you didn’t believe me on the difficulty of pining down numbers, look at these, from a clearly non-authoritative source (a WordPress (hosted) blog, not a journal, and so with who knows what bias):
- About 20 percent of teens will experience teen depression before they reach adulthood.
- Between 10 to 15 percent of teenagers have some symptoms of teen depression at any one time.
- About 5 percent of teens are suffering from major depression at any one time
- As many as 8.3 percent of teens suffer from depression for at least a year at a time, compared to about 5.3 percent of the general population.
The only bit of this that would be useful in the study I mentioned at the top of the post is the 5% suffering from major depression, which appears to be right in the middle of the range reported in various sources.
SO. Does this report tell us anything, esp. since it is contradicted by other reports? The authors (shown on the paylink) certainly look credible. But still, without proof or even really good counter-argument, I just don’t find this report very convincing. What I do find is how hard it is to do a convincing study. I am well acquainted with a person (unnamed to protect the innocent) who actually helps run drug studies in a psychiatry research department and also does all the statistical analysis for the subsequent papers. I asked what might limit this study to such a small group of people with such little breadth by other demographic factors – in a word, money. Even a study like this is relatively expensive to conduct and unlike a drug study where big bucks await the drug company on approval this isn’t the kind of research that would get big funding.
So I think this will be a real challenge of actually getting facts about impact of social media, getting enough money, getting a broad enough sample (and where are you going to get a control group?), running the study for long enough, and getting statistically valid results. Plus it’s clear, at least with Facebook, that different groups of people use it for different purposes, so teens probably are most interested in what all teens area, i.e. sex, whereas adults are probably interested in family (although I believe that would be regionally influenced) or more educated and in professions will be more interested in classmates. So it isn’t just hours a day that defines social media usage, it’s what is actually being done. Tons of variables to consider and that means a large sample size is needed.
So simply this is a complex problem and it’s not going to be easy to get any data. Depression, while vague (even under clinical diagnostic circumstances) is still easier to measure than development issues in teens, that might take years (for sure) or even decades to manifest. So good luck I say to those who will try, but in reality I think we are just charging ahead and doing an uncontrolled experiment on ourselves and it may be a long time before we really see the result.
We’ve been doing an unplanned experiment in another situation, greenhouse gas emissions, and even though the science now is definitive, less than half the public accepts the true results because it’s so easy for special interests to lie through paid propaganda. I can just imagine the resistance there would be if tomorrow some researchers definitively proved Facebook (or some other social media) was causing horrible effects whether anyone would do anything about it (for that matter have we really ever settled the issue, or regulated, violence in TV and video games for its effect on young males; or what about all the nonsense over vaccines – both of these seem simpler and yet still no accepted answers). So even when there is some clear truth it’s going to be hard for it, especially if it means someone losing some money, for it to emerge.
So again I hope we make progress on this, but it looks like a real challenge.