Jan 142012
 

Bad science: have a pet theory, manipulate the results to suit it.

Marketplace mimicked bad science well this past week. Their theory: COLD-FX does not work. Their results: don’t fit.

A little manipulation was in order using panning camerawork, fun quizzes, people-on-the-street interviews journalists are addicted to, jerky camerawork as they follow some poor target, lowered voice, clever camera cuts, grainy footage, undercover-type footage, selective submission of papers for scientific analysis (truly, have doctors and researchers not yet cottoned on to how journalists manipulate them?), highlight preferred statements over blasphemous one, present conclusions as mind-blowing, etc., etc.

Well, I don’t like it when my health is threatened, and so it’s time for a little fun. After all CBC’s Marketplace can’t hog it all.

But first: I take COLD-FX, have done for several years, as a preventative measure. I started because since my brain injury, I’ve become quite susceptible to colds; worse it takes me a month to recover from one. Going from cold to cold while trying to cope with the fallout of brain injury is extremely unpleasant. I took Flonase for awhile, but the side effects aren’t great. COLD-FX allowed me to stop the Flonase and for the first time in ages, I went a whole year last year without one respiratory infection. Hallelujah! I take COLD-FX as a preventative or prophylaxis because if I took it immediately in response to a scratchy throat, my body would still take weeks to recover no matter how efficacious COLD-FX is because that’s how it rolls these days with any illness I contract. I figure not getting one in the first place is better.

And so in the cause of health, I brave watching Marketplace. I haven’t watched it in years, ever since it went from trustworthy straightforward journalism to the gotcha kind. The old Marketplace may’ve been staid, but I trusted and respected it. New Marketplace makes me roll my eyes and switch the channel.

Anywhoo…

I sit back and watch … a mom-child convention. Huh? I don’t know what a COLD-FX luncheon for hockey moms has to do with a market report on a product. But it sure looks suspicious! Marketplace has set the mood and begins to reel us in with choice words.

Brilliant marketing idea” — sounds like COLD-FX was all about marketing, not about helping people fight the bane of our lives: colds. (Let rolling eyes commence.)

take a natural product, ginseng and get some science behind it.” — tsk, tsk, imagine makers of a natural health product standardizing their product and using the scientific method. What will they do next? Conduct and publish more than one study?

just like a pharmaceutical drug” — the nerve!

“research pays off” — damn, it sounds dirty, having solid research backing their product.

Marketplace then capitalizes on something no lay person is going to know, that Health Canada takes years — and years and years — to approve new products, and it isn’t always for kosher reasons either. Imagine a company that decides it’s had enough of Health Canada’s notorious foot dragging and, gasp, puts on political pressure to light a fire under the bureaucrats to actually work on it. Tut, tut. Bet all companies wish they could do that. What would be better though is if the politicians reformed Health Canada to approve — or reject — new products in a timely manner based solely on science.

Oh look, now we have the person-on-the-street interviews. It’s interactive, snazzy, and provides a we’re-here-for-you backdrop to the “expert” interview. And here’s where the manipulations get awesome.

Erica Johnson asks their chosen expert from a prestigious Toronto hospital about the claim for immediate relief for colds and flu. Erica asks Dr. Andreas Laupacis, a general internal medicine specialist: “Is there any research that’s been done showing that Cold-FX helps stop colds in their tracks?

He answers, the camera moving and panning, weaving and zooming on him, on her, on both: “Certainly all the [camera cut to Andreas only] clinical trials I’ve looked at there’s no such [camera cut to Erica only] evidence. They’ve studied patients with [camera cut to Andreas only] Cold-FX to prevent flus. I didn’t see any studies to show whether Cold-FX works or not in people that notice a flu coming and then take Cold-FX.

Erica: “That’s right. The pitch: to stop a cold in its tracks.

Uh, no, not right, he said “flu.” You Erica said “cold.” Two different viruses; two different topics. Just like the flu vaccine has zero effect on a cold and some effect in preventing flu, any product that can prevent a cold may not necessarily prevent the flu. Your expert, Marketplace, did not say COLD-FX does not prevent colds. He said flu, and only flu. (That’s probably why there were separate studies for colds and flu, more below.)

But a little repetition by Erica nicely masks that distinction. Gotta admire the manipulation.

What the heck did the Health Canada letter to Marketplace actually say? A few words pulled out say nothing and cannot be relied upon. I mean if movie companies can pull out glowing excerpts from bad reviews… If you want to know, check out their website for Health Canada’s statements (more below).

On to the “undercover” work! The better to make COLD-FX look like a big, fat fraud. Jerky camera work. Blurred faces. Closed captioning of what pharmacists say. The pitch: “Remember: there’s no published evidence for [taking COLD-FX for immediate relief].

For some reason, I keep hearing the Twilight Zone theme.

More experts! This time Marketplace sends a select list of published articles on COLD-FX for analysis by Andrew Lane Ilersich, MSc, BScPhm, RPh at the Univeristy of Toronto. But it’s kind of boring just saying what they said. Quiz time! Grand revelation after each question and answer session! But did the analysis really say what Marketplace asserts it said?

As Syd Baumel wrote on the Marketplace website: “To begin with, the scientist didn’t do an independent search of the literature in case there were other studies of Cold FX out there. He only analyzed the four submitted to him by Marketplace.”

Andrew Lane Ilersich, MSc, BScPhm, RPh did put in his short summary headline of the meta-analysislimited scope.” (More below.)

COLD-FX has ten citations and it looks like about eight clinical trials listed on its website. Cherry picking is sweet.

Baumel again: “Cold FX enjoyed a 15% reduction in cold frequency compared to those who took a placebo. Very modest effect, but statistically significant. To the individual user, this suggests that if you take Cold FX, it’ll spare you from getting a cold about one time out of 7.

I personally didn’t understand this whole “once in seventeen years” of taking COLD-FX assertion on Marketplace. I’ve never heard statistics interpreted that way before, not in stats classes or research I participated in or studies I’ve read. Fifteen percent is one in seven and would be a standard way of putting it.

Anyway, how many people would bother reading the entire meta-analysis (PDF) to get the correct picture? Don’t your eyes glaze over at the very thought? So it’s pretty safe for Marketplace to reproduce only one paragraph from the plain-language summary and not the paragraph that states clearly that the studies “demonstrated a reduction in the risk of getting a cold.” It’s that old pull out one statement, ignore the other trick to make it sound like it’s saying what you want to. Here’s the entire summary:

“COLD-FX is effective for preventing colds in adults. Research findings from 4 experimental studies (randomized controlled trials) that compared COLD-FX to a placebo (dummy treatment) including over 1000 adults demonstrated a reduction in the risk of getting a cold. In all studies, the COLD-FX was used in a dose of 400mg/day. The duration of treatment ranged from between 2 months and 6 months.

Relative to placebo, the risk of getting a cold was reduced by about 15% when COLD-FX was used. The absolute risk reduction was about 6% (this means that if the overall chance of getting a cold is, for example, 50%, then taking COLD-FX reduces it to 44%). Altogether, 17 people need to be treated to prevent 1 person from getting a cold.

For those who contracted a cold, there was insufficient evidence that the duration or severity was reduced.

This analysis did not explore the effects of age, dose and/or duration of therapy on the effectiveness of COLD-FX, nor the cost-effectiveness of COLD-FX.”

How interesting: the analysis did not explore effects of duration of therapy or dosage taken, one or both of which would have large effects on COLD-FX’s efficacy, one would surmise.

Onto the gotcha journalism CBC enjoys. It makes their target look like they’re up to no good, even when s/he has a clear, legal reason for not answering their in-their-face questions (I mean, what journalist doesn’t know how lawyers make people shut up, even for the silliest of reasons? The buy-out seems to be the reason here. Oh, but perhaps journalists figure most people wouldn’t know how effective lawyers are at silencing people? I feel for target Shan, caught between a lawyer and a journalist. Gak.).

So Erica asks the big question. And Jacqueline Shan answers: “[I was just talking about Cold-FX inside.]

Erica: “We didn’t hear you talk inside.” Really? They were able to track her down but were unable to make it in time for her talk?

Shan: “Our company was bought by Valeant. So I’m not allowed to make any public statement… You need to contact the company.” Pretty clear to me. It must suck for a journalist to be stonewalled by a large company, so take it out on an individual instead, eh?

Onward!

Oh hey, the lowered voice method! A lowered voice hints at nefarious doings, hints there was a bacteria cover-up even though Health Canada said there is no health risk in its last statement to Marketplace.

Marketplace quotes: “Based on currently available information, the presence of E. hermannii in a finished natural health product would be unacceptable.” Health Canada clarifies:

“Our earlier language was perhaps too black and white and did not accurately convey the science behind acceptable levels

After laboratory assessments were conducted by Health Canada scientists of the product on the Canadian marketplace, a low level of the bacteria Escherichia hermannii was found. Following a thorough assessment by Health Canada Scientists, it was determined that the level found presented the lowest risk to health and safety of Canadians [my emphasis] and, as such, no recall was initiated.

It is important to note that all health products have benefits and risks. When health products are found on the market that pose an unacceptable level of risk to health, Health Canada takes appropriate steps to mitigate and manage these risks.”

To be sure, I don’t recall any mass deaths or hospitalizations from COLD-FX-related E. coli contamination back in 2008 or 2009. Do you? But who needs to prove a dangerous bacteria contamination when all you have to say is “bacteria” to spring suspicion and fear into every viewer’s breast?

I feel for Marketplace. They really had to work hard to prove their theory about this product, going here and there, running all over the planet, from city to city, from expert to expert, using cameras that produced grainy pictures in China while using excellent ones for the scenes in which Erica appears.

On to the good stuff: an interview with Don Cherry.

Love the Don Cherry interview: one science-illiterate person talking to another, talking about two totally different things. One about immediate relief, the other about prevention. Neither notices. You don’t see this in the Markeplace piece, but in the extended Cherry piece I’ve embedded above.

Cherry begins by saying he doesn’t work for them anymore and he’s a little ticked off with COLD-FX, the company. Yet, get this, he still takes four COLD-FX capsules a day and ten a day, like the hockey players, if he feels a cold coming on. He relates in the extended Cherry piece that after a lifetime of being plagued by colds, after he began taking COLD-FX, he’s had just three colds in eight years. I don’t think anyone, least of all, Marketplace, or anything, like being fired, is going to pry the product out of his hands. He likes being cold-free too much. Me too, actually.

In all the hoo-hah, Marketplace forgot to mention an important point: “in the United States alone at least 1 billion colds per year have been reported” (from Predy et al, 2005 CMAJ article) with each person catching on average two to six colds. We know each cold costs several days of lost work or reduced productivity, never mind that it makes one feel lousier than hell. This is not peanuts. Dissing an effective remedy for colds harms public health.

The meta-analysis they had done listed four studies. I took a gander at them. (Note: in the meta-analysis, they were not identified in proper reference format, but I’m pretty sure I found the ones looked at as there can’t be more than one in the same year by the same authors on the same topic.)

2004 study: Elderly nursing home residents, 90 percent of whom had received the flu vaccine, had fewer cases of flu when taking COLD-FX for 8 weeks and 12, that is, 1 of 97 versus 7 of 101 who took a placebo. Taking COLD-FX reduced the risk of a fragile, elderly person from catching flu by 89 percent. By the way, flu kills the elderly.

2005 peer-reviewed study in CMAJ (Canadian Medical Association Journal): Healthy adults who took the same dose as in the 2004 study but for 4 months caught 0.68 colds versus 0.95 for placebo and also only 10 percent caught more than one cold while 22.8 percent in the placebo group got multiple colds.

“These results are similar to those reported for zanamivir and oseltamivir therapy. These antiviral agents have been reported to reduce the severity and duration of illness by 1.5-2.5 days. In comparison, the ginseng extract treatment was found to reduce the duration of a cold by 2.4 days.”

2006 study: A variation of the 2004 study, in which after two months of use, COLD-FX reduced the risk of contracting a respiratory infection by almost half (48 percent) and the duration by 55 percent. I assume the infections were colds because they state that there was no influenza in the community during the study.

2011 study: A larger version of the 2005 study.

“data indicate that CVT-E002 at a dose of 400 mg/day or 800 mg/day is safe and well tolerated and results in a reduction in the number, severity, and duration of Jackson-confirmed URIs (upper respiratory tract infections) when taken as seasonal prophylaxis by healthy, community-dwelling older adults. Further studies with larger sample size are warranted to determine possible dose-related effects of CVT-E002.”

Ilersich concluded: “In summary, these results support the effectiveness of COLD-FX for preventing colds. There is insufficient evidence of a reduction in severity or duration of colds.Insufficient evidence is science-speak for do more work, we don’t know one way or the other yet.

By the end of the twenty-two-odd minutes, Marketplace’s entire piece, when read between the lines and engendering Herculean effort not to be distracted by the bells and whistles, boils down to COLD-FX prevents colds. The claim it provides immediate relief needs further study; the China connection is no different than every other product we buy (have you checked where your frozen veggies are grown lately?), thus is not COLD-FX specific and is a separate topic; the bacterial contamination is old news and a non-starter. In other words, Marketplace told its alert viewers to take COLD-FX daily if you want to prevent colds.

Perhaps that’s why it ends its piece in the bathroom – with a shot of Erica and another expert washing their hands with soap, claiming that it’s more effective than COLD-FX. Washing hands with soap is effective in reducing colds. But what’s their published evidence proving their theory right? Where’s the double-blind randomized trial that compares the two methods side-by-side in reducing severity, duration, and frequency of infections, one for colds, one for flu?

 

——————-

References:

McElhaney JE, Gravenstein S, Cole SK, Davidson E, O’neill D, Petitjean S, Rumble B, Shan JJ. “A placebo-controlled trial of a proprietary extract of North American ginseng (CVT-E002) to prevent acute respiratory illness in institutionalized older adults.” J Am Geriatr Soc. 2004 Jan;52(1):13-9. Erratum in: J Am Geriatr Soc. 2004 May;52(5):following 856.

Gerald N. Predy, Vinti Goel, Ray Lovlin, Allan Donner, Larry Stitt, Tapan K. Basu. “Efficacy of an extract of North American ginseng containing poly-furanosyl-pyranosyl-saccharides for preventing upper respiratory tract infections: a randomized controlled trial.” CMAJ October 25, 2005 vol. 173 no. 9.

McElhaney JE, Goel V, Toane B, Hooten J, Shan JJ. “Efficacy of COLD-fX in the prevention of respiratory symptoms in community-dwelling adults: a randomized, double-blinded, placebo controlled trial.” J Altern Complement Med. 2006 Mar;12(2):153-7.

Janet E. McElhaney, Andrew E. Simor, Shelly McNeil, and Gerald N. Predy, “Efficacy and Safety of CVT-E002, a Proprietary Extract of Panax quinquefolius in the Prevention of Respiratory Infections in Influenza-Vaccinated Community-Dwelling Adults: A Multicenter, Randomized, Double-Blind, and Placebo-Controlled Trial,” Influenza Research and Treatment, vol. 2011, Article ID 759051, 8 pages, 2011. doi:10.1155/2011/759051

Share
  • Cindy

    Great article. I was also perplexed after viewing that Marketplace episode. I’ve been taking Cold-fx on and off for the past 4 years, and I have not had one severe cold or flu since. My instint told me the stuff was working for me, but after seeing that episode I thought I was possibly wasting my money. So I stopped taking it. Three weeks ago, my 20 month old daughter came down with a cold. Normally I would have taken the cold-fx to avoid catching her virus (she is in daycare and catches something every 2 months), but this time I didn’t. Well lo and behold, I got so sick I missed an entire week of work. It’s been 2 weeks and I am still coughing and blowing my nose non-stop. Needless to say I will resume the Cold-fx. Shame on you Marketplace.

  • http://jeejeebhoy.ca Shireen

    I’m so sorry about your bad cold. :( I concur: shame on Marketplace! It’s great to go after companies when gypping consumers but not when their products are vital to people’s health and do work. Colds (moreso flu) are not innocuous.

  • BaldyDaniels

    Thank you for bringing clarity to this issue. I had missed the airing of the marketplace episode on cold fx, but heard from a colleague of mine that it had been reported as a scam. Being a daily user of cold fx, and also a firm believer in it’s benefits, I had to see the marketplace claims for myself. Although I do not watch hockey and am not a fan in the slightest of Don Cherry, I am with him in regards to this matter. It works very well for me, and although there may be some who say it doesn’t work for them, well then sorry about their luck. Considering the marketplace’s pick and choose method in the gathering of their alleged evidence against cold fx, and I believe I read that CBC had aired cold fx commercials at one time, I think maybe CBC has a different motive for the attack on cold fx. As I already stated, it works well for me, and that’s what I care about. I’ve had so many pharmacists recommend advil to me for relief of a migraine headache, but I can take 5 or 6 with no affect, but two Tylenol and in 15 to 20 mins…gone! Doesn’t mean that advil is a scam, it just doesn’t work for me. Thanks again for posting what people should see about this product.

  • ret ret

    Thank you Marketplace. I do believe the product is a scam and a carrier of ecoli.

  • http://jeejeebhoy.ca Shireen

    You don’t believe in science, eh? No idea how research works, the concept of evidence and looking at the whole of it, not just cherry picking to suit your subjective view? And of course the complete illogic of saying COLD-FX carries a bacterium in an era when products that spread E. coli, etc. are always on the news, including CBC.

  • ret ret

    Ooh Shireen what bit your bum? You must use or sell this crap! Sorry but FX is what it is, a scam.

  • http://jeejeebhoy.ca Shireen

    You obviously didn’t read what I wrote.

  • http://twitter.com/RJWaterworth Robert Waterworth

    Come on, it is a major marketing blitz. This stuff dose not work. I fell for the ads a few years ago, and took the junk for a long time. Made no difference at all. After looking into it, I realized they play games with what is claimed and what is in the fine print. Snake Oil nothing more, save your money. Wash your hands, get lots of sleep, stay fit, eat better, etc etc. All of which is free.

  • http://twitter.com/RJWaterworth Robert Waterworth

    Come on, it is a major marketing blitz. This stuff dose not work. I fell for the ads a few years ago, and took the junk for a long time. Made no difference at all. After looking into it, I realized they play games with what is claimed and what is in the fine print. Snake Oil nothing more, save your money. Wash your hands, get lots of sleep, stay fit, eat better, etc etc. All of which is free.

  • http://jeejeebhoy.ca Shireen

    Nonsense. Did you read the research I cited? Furthermore, Snake Oil isn’t standardized. Cold-FX is. Health Canada doesn’t give out a Natural Product Number unless it is:
    “The amount of medicinal ingredient(s) per dosage unit. It is always required for a product, as it is the amount of medicinal ingredient in the product.” (See http://webprod3.hc-sc.gc.ca/lnhpd-bdpsnh/info.do?licence=80002849)
    Also, if we go by your criterion that drug x doesn’t work because it didn’t work for me, then every drug on the market from aspirin to zoloft is Snake Oil because no drug is 100% effective for 100% of people.
    As for fine print, they’re as upfront as every other drug manufacturer out there, if not more so. They’re careful in what they claim it can do, if for no other reason than Health Canada won’t give them the Natural Product Number. This is like complaining that KD plays games because it doesn’t list tartrazine on the front of their package and only lists it in fine print in the tiny ingredient list no one reads. Before taking anything, it behooves you to read the label fully, including what the non-medicinal ingredients are. Washing hands with soap goes without saying; but it too is not 100% foolproof. Does that make all the health care professionals who espouse it Snake Oil marketers?

  • Roy H.

    You have to laugh at people who think this stuff works. CBC is was right..and a many other RESPECTED SCIENTISTS (more later) think its not that great either. The product itself..is not unlike corn sugar. It is poly-furanosyl-pyranosyl-sacchardies (this is sugars) that has been extracted by using a aqueous method (soaked in water). Your comment about it being standardized is also a joke sir. This is marketing too. OF COURSE IT IS! It has to be. You can not make a product that has no standardization.

    Your reference to Healthcare Canada is also a joke. Cold FX tried to pull the wool over our eyes for years. They sell sold packages intended to be take :”at the first sign” of colds or flus. The dosage was 3 pills 2 times a day for one day, then 2 pills twice a day..then one….it total, about $12 worth. Funny thing they were never allowed to prescribe such dosages..yet they did..to make money. When they were told by Heath Canada to stop..it took 3 years for them to actually stop. Very ‘trustworthy” eh? So reading the label fully..is clearly a stupid place to gain your information.

    Siereen, i DO have an idea how research works and I CAN tell you that the data they provide is misleading. Their studies are way small, have a poor P factor and are statistically insignificant. The site relative, not absolute differences in order to mislead.

    On a broader scale..what a great marketing position eh? Here is the joke of it all!!!! Follow me on this:

    You take a pill every day to, supposedly avoid getting something you don’t have. What you are avoiding, your body will fight off most of the time anyway. Hey, we all have felt a cold coming on..the tickle ion the throat…a couple sneezes…and our body’s acquired and innate immune systems fights it off. The people that take it say “Look how well it works”…when it was their own body doing what it does…protecting you.

    Then when they DO get a cold, these people say..well..no one pill is 100% effective. Sound familiar…Shireen?

    The CSO and founder of Cold FC is at it again with a new product and the marketing is exactly the same. This is where we laugh a little more. Jackie Chan cold FX company was called Afexa Life Sciences…Her new one… Afinix Life Sciences. Sound familiar?

    Her old standardization process was called ChemBioPrint…LOL. THis time its called…Afinity Technology and the wording surrounding it is also the same. THis time, she is selling Omegas. At least the product has a sound scientific backing…but nothing revolutionary.

    Before I sign off, I wanted to let you know about one last study that was done..the largest to date…partially sponsored by U.S. National Cancer Institute. This IS a credible body. In laymen’s terms, the NCI wanted to see if Cold FX could stop colds from occurring in their cancer ( chronic lymphocytic leukemia) , as a colds can and do kill them!! If Cold FX worked…. would SAVE LIVES!!!!

    No shocker here…..It didn’t. The trail concluded “Conclusion: CVT-E002 (Cold FX’s clinical name) was well tolerated. It did not reduce the number of ARI days or antibiotic use…”

    Feel free to take it Shireen. it won’t hurt you and if you think it helps you…then by all means…dig in.