Read the study, don’t believe headlines

Here’s an incredible example of how the popular press, especially one with attitude, provides misleading and/or exaggerated reporting of studies. The headline trumpets:

High-Protein Breakfast Helps Prevent Unhealthy Snacking Later, Study Says

Let’s pass over the editorial comment “unhealthy snacking” (how do we know it is “unhealthy”, did the study actually say that? or is this the author’s POV creeping in? So let’s look at the actual study title).

Beneficial effects of a higher-protein breakfast on the appetitive, hormonal, and neural signals controlling energy intake regulation in overweight/obese, “breakfast-skipping,” late-adolescent girls

Admittedly a cumbersome title to be quoting in a popular press article, but I don’t see anything about “unhealthy snacking” in there. But let’s move on to the bulk of this.

At least the popular press article provides a link to the actual journal article, conveniently hidden behind a paywall, however, so we can’t really take a look at the critical parts, the statistical analysis. But even without the actual statistics let’s just look at a couple of critical bits:

  1. the study involved 20! subjects (wow, couldn’t they afford more)
  2. the study only involved adolescent girls so obviously this must apply to the rest of us
  3. the study involved three tracks, meaning about 7 people in each track (think about the statistical significance of such tiny groups)
  4. the study lasted a whopping seven days, with six days establishing the protocol and only a single day! to measure the results

These are all red flags indicating that there can’t possibly be any statistically meaningful results, which on top of the fact that none are reported in the brief of the article (often, even when full articles are hidden behind paywalls at least a quick summary of the statistical analysis technique is presented, so this article is “hiding” its results more than usual).

But just imagine this. A completely unrepresentative sample of subjects in such tiny numbers that even small variations will not average out conducted over such a brief period of time that completely random effects will swamp any meaningful results.

In short, this study is utterly worthless and one wonders why the journal even published such a ridiculous study (anecdotal data would be almost as significant). Are they hard up for legitimate studies with reasonable sample sizes and statistical results.

But finally the kicker (remember this is a study designed to test the efficacy of high protein breakfasts). Here’s the source of funding:

Supported by the Beef Checkoff and the Egg Nutrition Center/American Egg Board.

Oh yow,are these authors primed to decide “HP egg- and beef-rich (35 g protein) breakfasts” is better than skipping breakfast. Are you surprised? A tiny study got a result the sponsors wanted!

But, so what. This is obviously a mediocre (at best) study of no great significance. Why then did the popular press even bother using this study as a basis for an article? Because of their attitude and probably studies to support that attitude are rare (or quite possibly contradictory). But the public is not served by this approach. This is the kind of “result” that almost certainly will be reversed or contradicted. Referencing all this stuff uncritically just lowers the credibility of science (the credibility of HuffPo’s Healthy Living section can hardly get any lower since it’s almost entirely quacks and charlatans).

People need good and correct and useful information about nutrition, not junk that just fits the biases of some author. No wonder the public tunes all this out and believes nonsense like the anti-vaxers and anti-GMO and supplements woo peddlers put out.


About dmill96

old fat (but now getting trim and fit) guy, who used to create software in Silicon Valley (almost before it was called that), who used to go backpacking and bicycling and cross-country skiing and now geodashes, drives AWD in Wyoming, takes pictures, and writes long blog posts and does xizquvjyk.
This entry was posted in musing, rant and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s