5-minute calculation

Of all the things I thought I would be blogging about, this did not cross my mind.

If you’ve watched videos online, you’ve doubtless been bombarded by the annoyingly tacky commercials for 5-Hour Energy, an energy shot that is supposed to contain B-vitamins, amino acids and nutrients, and is supposed to provide benefits that are not confirmed by the Food and Drug Administration.

Sometime in the past few weeks, the folks who run 5-Hour Energy have been flooding the internet with a less tacky commercial in which a polished-looking model describes the results of a survey of primary care physicians:

We asked over 3000 doctors to review 5-Hour Energy and what they said is amazing! Over 73% who reviewed 5-Hour Energy said they would recommend a low-calorie energy supplements to their healthy patients who used energy supplements. 73%!

[…stuff about the product’s low-calorie property and how millions use it …]

Is 5-Hour Energy right for you? Ask your doctor. We already asked 3000!

There are the usual advertising gimmicks at work here. At first glance you would think that 73% of the 3000 doctors, i.e., 2190 doctors, recommended the product. However, as the lady (honestly!) points out, they only recommended a low-calorie energy supplement, not necessarily 5-Hour energy. To find out how many doctors recommended the product, we have to read the fine print on the page, which is not at all difficult given how many times the advertisement shows up in videos of the Daily Show. This fine print states:

Of the 73% of primary care physicians who said they would recommend a low-calorie energy supplement to their healthy patients who use energy supplements, 56% would specifically recommend 5-Hour Energy …

This means that 56% of 2190, i.e., 1226 doctors truly recommend the product being advertised. That’s still a significant number, and a significant fraction of 3000, until you read more of the fine print about how the survey was conducted. Apparently, some doctors were interviewed online, and some were visited by 5-Hour Energy representatives (emphasis mine).

Two surveys were conducted to determine the opinions of primary care physicians regarding energy supplements and 5-Hour Energy: (1) An online survey of 503 participants, and (2) An in-person survey of 5-Hour Energy representatives of 2500 participants (50% of those approached)”. In both, participants agreed to review materials regarding 5-Hour Energy consisting of label and basic description of its ingredients.

Now, anyone with the slightest curiosity would wonder: What happened with the other 50%, i.e. 2500 doctors? Did they refuse the survey? If so, why? Did they not recommend 5-Hour Energy? And if you’re even slightly familiar with data collection and surveys, one would ask: How did you choose which 50% to report results on, and which 50% to reject? Why is your data-point selection criterion not given in the fine print. What is to stop me from believing that you reported the most favorable 50% and summarily rejected the other half?

At any rate, it is clear that the total number of physicians (assuming that they were truly primary care physicians) approached by the 5-Hour Energy representatives was not 3000, but roughly 500 + 2500 + 2500 = 5500. Of these, 1226 doctors, or 22% recommended 5-Hour Energy. Even though these folks choose their words carefully, the advertisement clearly wants you to believe that 73%, i.e., nearly 3 out of 4 doctors recommend the product, when in reality, slightly more than 1 out of 5 doctors do so.

If you want to nitpick further, you might ask: Why should I trust that one doctor out of 5 who makes a recommendation based on the “label and basic description of its ingredients”?. That is not how doctors recommend medicines in general. There has to be at least a proper double-blind study with placebos*. No wonder, the statements of 5-Hour Energy are not approved by the FDA. Still, it is apparently used 9 million times a week. That is a lot of suckers.

5-Hour Energy is owned by this guy, a businessman and philanthropist. He’s just doing what marketers in the lucrative supplement industry do over and over again. They want to mislead, so they choose their advertising monologues carefully while flashing the fine print just to keep their hands clear of the law. One wonders, do they need a 10-Hour-Sleep supplement to get through the night?

* In one of the comments below, a reader, Christy, has provided a link to the webpage containing more information about the test. It shows the label provided to the doctors being surveyed, and claims that a double-blind study was conducted in 2009 but that it is still in peer review. If you are familiar with studies of this sort, I would appreciate knowing from you  whether it is normal for peer review to take 3 years. [Edit added on August 13, 2012].
[Afterward: Just realized that I mentioned 5-Hour Energy so many times that the context-sensitive advertisement robot¬†may want to advertise the product on this page. If you, faithful-reader-without-a-wordpress-account, find that this is indeed the case, do let me know in the comments, for that would be one messed-up irony in the age of the internet. This is when I really hate the fact that I still use the free version of wordpress.com, providing my implicit consent to the serving of ugly ads to non-wordpress netizens in order to pay Matt Mullenweg’s bills.]