5-minute calculation

Of all the things I thought I would be blogging about, this did not cross my mind.

If you’ve watched videos online, you’ve doubtless been bombarded by the annoyingly tacky commercials for 5-Hour Energy, an energy shot that is supposed to contain B-vitamins, amino acids and nutrients, and is supposed to provide benefits that are not confirmed by the Food and Drug Administration.

Sometime in the past few weeks, the folks who run 5-Hour Energy have been flooding the internet with a less tacky commercial in which a polished-looking model describes the results of a survey of primary care physicians:

We asked over 3000 doctors to review 5-Hour Energy and what they said is amazing! Over 73% who reviewed 5-Hour Energy said they would recommend a low-calorie energy supplements to their healthy patients who used energy supplements. 73%!

[…stuff about the product’s low-calorie property and how millions use it …]

Is 5-Hour Energy right for you? Ask your doctor. We already asked 3000!

There are the usual advertising gimmicks at work here. At first glance you would think that 73% of the 3000 doctors, i.e., 2190 doctors, recommended the product. However, as the lady (honestly!) points out, they only recommended a low-calorie energy supplement, not necessarily 5-Hour energy. To find out how many doctors recommended the product, we have to read the fine print on the page, which is not at all difficult given how many times the advertisement shows up in videos of the Daily Show. This fine print states:

Of the 73% of primary care physicians who said they would recommend a low-calorie energy supplement to their healthy patients who use energy supplements, 56% would specifically recommend 5-Hour Energy …

This means that 56% of 2190, i.e., 1226 doctors truly recommend the product being advertised. That’s still a significant number, and a significant fraction of 3000, until you read more of the fine print about how the survey was conducted. Apparently, some doctors were interviewed online, and some were visited by 5-Hour Energy representatives (emphasis mine).

Two surveys were conducted to determine the opinions of primary care physicians regarding energy supplements and 5-Hour Energy: (1) An online survey of 503 participants, and (2) An in-person survey of 5-Hour Energy representatives of 2500 participants (50% of those approached)”. In both, participants agreed to review materials regarding 5-Hour Energy consisting of label and basic description of its ingredients.

Now, anyone with the slightest curiosity would wonder: What happened with the other 50%, i.e. 2500 doctors? Did they refuse the survey? If so, why? Did they not recommend 5-Hour Energy? And if you’re even slightly familiar with data collection and surveys, one would ask: How did you choose which 50% to report results on, and which 50% to reject? Why is your data-point selection criterion not given in the fine print. What is to stop me from believing that you reported the most favorable 50% and summarily rejected the other half?

At any rate, it is clear that the total number of physicians (assuming that they were truly primary care physicians) approached by the 5-Hour Energy representatives was not 3000, but roughly 500 + 2500 + 2500 = 5500. Of these, 1226 doctors, or 22% recommended 5-Hour Energy. Even though these folks choose their words carefully, the advertisement clearly wants you to believe that 73%, i.e., nearly 3 out of 4 doctors recommend the product, when in reality, slightly more than 1 out of 5 doctors do so.

If you want to nitpick further, you might ask: Why should I trust that one doctor out of 5 who makes a recommendation based on the “label and basic description of its ingredients”?. That is not how doctors recommend medicines in general. There has to be at least a proper double-blind study with placebos*. No wonder, the statements of 5-Hour Energy are not approved by the FDA. Still, it is apparently used 9 million times a week. That is a lot of suckers.

5-Hour Energy is owned by this guy, a businessman and philanthropist. He’s just doing what marketers in the lucrative supplement industry do over and over again. They want to mislead, so they choose their advertising monologues carefully while flashing the fine print just to keep their hands clear of the law. One wonders, do they need a 10-Hour-Sleep supplement to get through the night?

————————
* In one of the comments below, a reader, Christy, has provided a link to the webpage containing more information about the test. It shows the label provided to the doctors being surveyed, and claims that a double-blind study was conducted in 2009 but that it is still in peer review. If you are familiar with studies of this sort, I would appreciate knowing from you  whether it is normal for peer review to take 3 years. [Edit added on August 13, 2012].
————————
[Afterward: Just realized that I mentioned 5-Hour Energy so many times that the context-sensitive advertisement robot may want to advertise the product on this page. If you, faithful-reader-without-a-wordpress-account, find that this is indeed the case, do let me know in the comments, for that would be one messed-up irony in the age of the internet. This is when I really hate the fact that I still use the free version of wordpress.com, providing my implicit consent to the serving of ugly ads to non-wordpress netizens in order to pay Matt Mullenweg’s bills.]

Advertisements

37 thoughts on “5-minute calculation”

  1. I found this article because I found the ad utterly ridiculous and deceptive too.

    One other hilarious aspect of this ad, is this giant pile of papers with supposed signatures from doctors. In this pile there are well over 3000 pages. I would guess the pile has upwards of 50,000 pages or maybe more.

    1. 500 sheets of office depot paper is slightly under 2 inches thick, I have to stock tons of reams for my electronics store. Average womans torso would be about 2 feet, crotch to shoulder, so I’d guess more specifically, 6000 sheets of paper in her stack.

  2. Well, 30 years ago, when there were still electronics devices that could be repaired, I was an apprentice in a TV repair store and had to go out to pick up TVs, etc. One day, we had to go to a Doctor’s office. The Doctor had a short break from his patience to let us in and at the same time received a representative from a drug company to inform him about the drug they were selling, give him flyers etc.
    The representative was the most stunning woman I have ever seen, and I had seen a lot at that time, being a frequent Disco visitor and girl hunter 🙂
    My point is: When drug corporations visit a doctor, they make sure they do it in a way to influence him in the most serious way possible, either with beautiful women or invitations to conferences in medically relevant locations like Acapulco etc. That’s probably why they send out more people to visit doctors than they did that online thingy.

    Besides, when I hear that sort of thing in an ad, the old Camel commercial comes to mind: “Which cigarette do you smoke, Doctor?”

  3. Actually your numbers are way off. The statement is about patients that “already use energy boosters.” If only 7 doctors out of the 3 thousand had patients who used energy boosters and 5 of them recommended 5 hour energy that would make the 73%. The 3000 number does not factor in to the equation. Through false logic they lead you to believe it is recommended by over 2,000 doctors.

    1. Hi Justin,

      Thanks for your comment. I conservatively assumed that _each_ of the 3000 doctors was asked the question, “Would you recommend a low-calorie energy supplement to your patient, if that patient is already using an energy supplement?”

      Of course, if the doctors with no patients on energy boosters were first eliminated, then what you say would be exactly correct. That would be an even worse fabrication than the one that I have elaborated above.

      Polaris.

      1. This was the part of the commercial that really bothered me. The woman clearly states that the doctors were only commenting about whether they would recommend a LOW Calorie supplement to their patients who ALREADY use supplements. The question very well could have gone something like this: “If you have a patient who is already taking energy supplements and you knew they were still going to take energy supplements regardless of any other recommendations you might make, would you suggest that the patient at least take a low calorie supplement (not even 5-Hour Energy, by the way) instead of a high-calorie supplement?” That’s kind of like asking “If you knew your patient was going to hit himself in the head with a hammer, would you recommend that the patient at least use a rubber hammer instead of a metal hammer?”

  4. I saw this commercial earlier, and when I heard what the lady said, I had to rewind and listen carefully. I was appalled and searched to find if anyone else felt the same as I did.

    Are consumers really so inept that this kind of advertising works? Really? Are we collectively so stupid?

  5. In fairness, I think that the 50% of those who were approached bit means that 50% of those who were approached agreed to participate in the survey, not that 5000 doctors participated and they used the half that were more favorable.

    Also, I suspect that more than 500 were “approached” online, but only 500 actually took the survey.

    1. The thing is though that you don’t mention who you approached and didn’t answer a survey in a research article. You just mention the people that you did talk to. It is research stats 101, they should have hired someone who went to college to do their research.

      1. I entirely disagree.

        The more information that you provide, the better it is for your audience and the easier it is for them to judge how meaningful your results are. Every decent report of a statistical survey should report the response rate. If a report does not mention the response rate, that calls into question the integrity of the report.

        That’s why I think that the 50% figure that’s quoted is the survey response rate.

    2. donboc,

      Thank you for your comment. It is possible that you are correct. Indeed, I mentioned the possibility in the blog post above, when I asked: “Did they refuse the survey?”.

      However, I still share Tim’s skepticism because:
      (1) Even if you are correct, we do not know the grounds on which the other 50% did not participate, i.e., we do not know whether they declined because they did not have time, or because they were skeptical about the product or because of some other reason.
      (2) The figure of 50% seems a little too convenient. It reeks of selection bias to me. This has happened before in clinical trials, and it will unfortunately happen again.

      At any rate, even if what you say were true, the “73% of the 3000 doctors recommend …” is disingenuous because – for people who don’t read the fine print – the advertisement conveniently masks the 2000 doctors who did not participate in the survey.

      Polaris

      1. Hi Polaris,

        “the advertisement conveniently masks the 2000 doctors who did not participate in the survey.”

        Well, true, but isn’t that true of any survey you read about? When you hear about a political poll, you hear 47% support Obama; you don’t hear what fraction of the people who were approached participated (unless you scan the fine print).

        I’m not suggesting that this “survey” was run anywhere near as well, nor reported as honestly, as Gallup does. But I think that this particular point of criticism might not be completely fair.

  6. At no point did the commercial claim 3000 doctors had replied; 3000 were asked. Perhaps 100 replied and 73 said “Yeah, I’d recommend your crap over someone else’s crap”.

    Enery is measuerd in Calories.

    1. Nelson, thanks for your comment. I assumed that since the fine print said “50% of those approached”, they meant that this 50% and the 500-odd online folks actually responded to the survey. The language is unclear though, and you may be correct; it could be even worse than I thought.

  7. I also find it incredibly unnerving when these doctors “suggest 5 Hour Energy”, but they are not willing to share their names. If you look when she’s flipping through the stack of signatures, all but the top has been blurred out.

    I know I wouldn’t want my name plastered on a commercial, but if they’re so consenting of 5 Hour Energy … why would THEY care? Just supports the idea that they do not actually support 5 Hour Energy.

    Very sneaky 5 Hour Energy.

  8. I have a background in social science statistics, and this commercial made me cringe because of the obvious problems with the data analysis. Thank you for your own detailed analysis.

  9. The biggest thing that bothered me about it was that the fine pring specified that the doctors were surveyed about energy supplments “and” 5 hour energy, and the results were about energy supplements in general and not 5 hour energy specific. I get the feeling the FTC will be shutting this advertising down soon because it is BLATANTLY misleading.

    P.S. I found this article after googling the commercial to make sure I wasnt the only one who found it completely appalling.

    1. Hi Renee,

      Thank for your comment and visit. It is indeed an appalling ad.

      Unfortunately, this may be outside the jurisdiction of the FTC (or even the FDA). The rationale for putting in the fine print is for them to shrug their shoulders later and say: “See, we included all the true information. What could we do if our viewers are daft enough to believe our beautiful actress while ignoring the fine print!?”

      Polaris.

  10. I can’t express just how much I loathe deceptive advertising.
    If I showed up in a physicians office, showed the doctor a brochure on this product and asked the question: “Doctor, as I am sure you are aware, some of your patients use energy boosting drinks. Would you recommend a high calory or low calory product?”. I would put money on it that 73% would respond “Low calorie”.
    If I then presented a list of energy boosting drinks on the market with each of their ingredients listed below them and then asked the question “Doctor, as you can see the majority of the products sold for this purpose have exactly the same ingredients. If your patient is healthy and is planning on using an energy booster anyway, would you feel comfortable recommending our product?”. I would again bet that at least 47% of the doctors would say “sure”. Probably far more if they hired the right representatives to ask the question.
    5 Hour Energy is taking advantage of people that cannot read fast enough to read the fine print on their TV ad (like me) or cannot read between the lines of the script that is being read. They are very careful not to make any health or nutritional claims which would force a review of the product by the FDA.
    Hopefully something will go viral and alert the masses of their deceptive practices. Even if the product turns out to be safe (if we ever find out) companies that resort to deception should not be supported. If you believe in more government involvement and regulation please talk to your representatives in Washington, if you believe in less tax dollars spent on Federal regulation please vote with your checkbook and don’t buy this product. Either way please do your part to expose these practices to others.
    When companies have to resort to deception we must ask, what are they hiding?

    1. I totally agree they should not be supported. The funny thing is, I used to very occasionally by 5 hour energy drinks, but now I will make a point not to because of this ad.

    2. Brian,

      You are definitely correct in your observation that careful choice and ordering of questions can prime the respondents (doctors in this case) so that they provide the desired response with high probability.

      Regarding the policy reform that will prevent this from happening, I am not sure which course is best. It does appear that stricter regulation might help but the problem is that when governments create a regulation, the parties most likely to suffer from the regulation lobby to simultaneously create a loophole around it. This is really the problem in my view – Allowing an FDA-unapproved product to share airtime (or webtime these days) with ads for approved products, gives these companies time to beguile the public and significantly negates the effect of regulation that seeks FDA approval.

      Polaris

      1. Christy: Thanks for the link to the review website. Later today, I shall cite your comment in the post above.

        Yes, they mentioned in the fine print that the doctors reviewed just the product label. It is quite ludicrous. I find it mildly comforting that there was a double-blind study to compare the effects with that of a placebo. However, given that the study has been under peer review since 2009, I wonder what has been holding it up.

        Polaris.

  11. One thing you seemed to overlook is even MORE blatant and stupid: 73% of doctors recommend a low-cal supplement for those already taking an energy supplement. So essentially they’re saying “well if you’re gonna do this, you might as well do it with less calories so MAYBE it won’t be quite as terrible for you.”

    1. jC24, yes I did notice it, but my intention while writing this post was to focus only on the misrepresentation of the statistics and the discrepancy between what the ad said and what the fine print revealed. You are — of course — correct in noticing how the question is blatantly framed to prime the doctors for a certain kind of response.

  12. The doctors who didn’t respond probably chose not to because (as jC24 mentioned) 5-hour Energy was trying to corner them into saying what 5HE wanted them to say: “IF a patient is already taking an energy supplement, should they make sure it’s a low-calory one? Yes or No?” What doctor would say no?? It’s essentially a trap — manipulative and completely unethical. I’d like to believe that that’s the reason for only 50% participation…that some of those doctors have their heads screwed on straight and want no part in that bullshit. And I’m pretending that of the 50% who did not participate, 25% told 5-hour Energy to go shove this “survey” up their ass.

  13. regarding your afterword – go ahead and let them! Some of their ill-gotten profits can end up going to you, for taking the time to counter their immoral methods of advertising. In fact, I’ll click through, just to help make that happen!

    1. Hehe, thanks for the kind words, jacobaziza.

      However, if there were any ads on my blog, the people making money out of them are WordPress (Automattic). WordPress.com does not allow me to host ads. To do that, one needs to use the self-hosted wordpress platform.

  14. All it is is another caffeine product. They do have a “decaf” version, which they say has about half the caffeine of a normal cup of decaf coffee, but I doubt it’s a big seller.

    http://www.doctorsreview5hour.com/review.php
    “There are three versions of 5-hour ENERGY: Decaf, Regular and Extra Strength. The caffeine in the Decaf version is about as much as a half cup of decaffeinated coffee. In the Regular version it is comparable to a cup of the leading premium coffee. In the Extra Strength version it is comparable to 12 ounces of the leading premium coffee. Decaf has Choline Bitartrate (not Citicoline) and no Niacin.”

    A cup of black coffee would qualify as a “low-calorie energy supplement” as well, and keep you as awake as a regular strength bottle of their product. The coffee is a helluva lot cheaper too.

    A fool and his money…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s