Bad science: have a pet theory, manipulate the results to suit it.
A little manipulation was in order using panning camerawork, fun quizzes, people-on-the-street interviews journalists are addicted to, jerky camerawork as they follow some poor target, lowered voice, clever camera cuts, grainy footage, undercover-type footage, selective submission of papers for scientific analysis (truly, have doctors and researchers not yet cottoned on to how journalists manipulate them?), highlight preferred statements over blasphemous one, present conclusions as mind-blowing, etc., etc.
Well, I don’t like it when my health is threatened, and so it’s time for a little fun. After all CBC’s Marketplace can’t hog it all.
But first: I take COLD-FX, have done for several years, as a preventative measure. I started because since my brain injury, I’ve become quite susceptible to colds; worse it takes me a month to recover from one. Going from cold to cold while trying to cope with the fallout of brain injury is extremely unpleasant. I took Flonase for awhile, but the side effects aren’t great. COLD-FX allowed me to stop the Flonase and for the first time in ages, I went a whole year last year without one respiratory infection. Hallelujah! I take COLD-FX as a preventative or prophylaxis because if I took it immediately in response to a scratchy throat, my body would still take weeks to recover no matter how efficacious COLD-FX is because that’s how it rolls these days with any illness I contract. I figure not getting one in the first place is better.
And so in the cause of health, I brave watching Marketplace. I haven’t watched it in years, ever since it went from trustworthy straightforward journalism to the gotcha kind. The old Marketplace may’ve been staid, but I trusted and respected it. New Marketplace makes me roll my eyes and switch the channel.
I sit back and watch … a mom-child convention. Huh? I don’t know what a COLD-FX luncheon for hockey moms has to do with a market report on a product. But it sure looks suspicious! Marketplace has set the mood and begins to reel us in with choice words.
“Brilliant marketing idea” — sounds like COLD-FX was all about marketing, not about helping people fight the bane of our lives: colds. (Let rolling eyes commence.)
“take a natural product, ginseng and get some science behind it.” — tsk, tsk, imagine makers of a natural health product standardizing their product and using the scientific method. What will they do next? Conduct and publish more than one study?
“just like a pharmaceutical drug” — the nerve!
“research pays off” — damn, it sounds dirty, having solid research backing their product.
Marketplace then capitalizes on something no lay person is going to know, that Health Canada takes years — and years and years — to approve new products, and it isn’t always for kosher reasons either. Imagine a company that decides it’s had enough of Health Canada’s notorious foot dragging and, gasp, puts on political pressure to light a fire under the bureaucrats to actually work on it. Tut, tut. Bet all companies wish they could do that. What would be better though is if the politicians reformed Health Canada to approve — or reject — new products in a timely manner based solely on science.
Oh look, now we have the person-on-the-street interviews. It’s interactive, snazzy, and provides a we’re-here-for-you backdrop to the “expert” interview. And here’s where the manipulations get awesome.
Erica Johnson asks their chosen expert from a prestigious Toronto hospital about the claim for immediate relief for colds and flu. Erica asks Dr. Andreas Laupacis, a general internal medicine specialist: “Is there any research that’s been done showing that Cold-FX helps stop colds in their tracks?”
He answers, the camera moving and panning, weaving and zooming on him, on her, on both: “Certainly all the [camera cut to Andreas only] clinical trials I’ve looked at there’s no such [camera cut to Erica only] evidence. They’ve studied patients with [camera cut to Andreas only] Cold-FX to prevent flus. I didn’t see any studies to show whether Cold-FX works or not in people that notice a flu coming and then take Cold-FX.”
Erica: “That’s right. The pitch: to stop a cold in its tracks.”
Uh, no, not right, he said “flu.” You Erica said “cold.” Two different viruses; two different topics. Just like the flu vaccine has zero effect on a cold and some effect in preventing flu, any product that can prevent a cold may not necessarily prevent the flu. Your expert, Marketplace, did not say COLD-FX does not prevent colds. He said flu, and only flu. (That’s probably why there were separate studies for colds and flu, more below.)
But a little repetition by Erica nicely masks that distinction. Gotta admire the manipulation.
What the heck did the Health Canada letter to Marketplace actually say? A few words pulled out say nothing and cannot be relied upon. I mean if movie companies can pull out glowing excerpts from bad reviews… If you want to know, check out their website for Health Canada’s statements (more below).
On to the “undercover” work! The better to make COLD-FX look like a big, fat fraud. Jerky camera work. Blurred faces. Closed captioning of what pharmacists say. The pitch: “Remember: there’s no published evidence for [taking COLD-FX for immediate relief].”
For some reason, I keep hearing the Twilight Zone theme.
More experts! This time Marketplace sends a select list of published articles on COLD-FX for analysis by Andrew Lane Ilersich, MSc, BScPhm, RPh at the Univeristy of Toronto. But it’s kind of boring just saying what they said. Quiz time! Grand revelation after each question and answer session! But did the analysis really say what Marketplace asserts it said?
As Syd Baumel wrote on the Marketplace website: “To begin with, the scientist didn’t do an independent search of the literature in case there were other studies of Cold FX out there. He only analyzed the four submitted to him by Marketplace.”
Andrew Lane Ilersich, MSc, BScPhm, RPh did put in his short summary headline of the meta-analysis “limited scope.” (More below.)
Baumel again: “Cold FX enjoyed a 15% reduction in cold frequency compared to those who took a placebo. Very modest effect, but statistically significant. To the individual user, this suggests that if you take Cold FX, it’ll spare you from getting a cold about one time out of 7.”
I personally didn’t understand this whole “once in seventeen years” of taking COLD-FX assertion on Marketplace. I’ve never heard statistics interpreted that way before, not in stats classes or research I participated in or studies I’ve read. Fifteen percent is one in seven and would be a standard way of putting it.
Anyway, how many people would bother reading the entire meta-analysis (PDF) to get the correct picture? Don’t your eyes glaze over at the very thought? So it’s pretty safe for Marketplace to reproduce only one paragraph from the plain-language summary and not the paragraph that states clearly that the studies “demonstrated a reduction in the risk of getting a cold.” It’s that old pull out one statement, ignore the other trick to make it sound like it’s saying what you want to. Here’s the entire summary:
“COLD-FX is effective for preventing colds in adults. Research findings from 4 experimental studies (randomized controlled trials) that compared COLD-FX to a placebo (dummy treatment) including over 1000 adults demonstrated a reduction in the risk of getting a cold. In all studies, the COLD-FX was used in a dose of 400mg/day. The duration of treatment ranged from between 2 months and 6 months.
Relative to placebo, the risk of getting a cold was reduced by about 15% when COLD-FX was used. The absolute risk reduction was about 6% (this means that if the overall chance of getting a cold is, for example, 50%, then taking COLD-FX reduces it to 44%). Altogether, 17 people need to be treated to prevent 1 person from getting a cold.
For those who contracted a cold, there was insufficient evidence that the duration or severity was reduced.
This analysis did not explore the effects of age, dose and/or duration of therapy on the effectiveness of COLD-FX, nor the cost-effectiveness of COLD-FX.”
How interesting: the analysis did not explore effects of duration of therapy or dosage taken, one or both of which would have large effects on COLD-FX’s efficacy, one would surmise.
Onto the gotcha journalism CBC enjoys. It makes their target look like they’re up to no good, even when s/he has a clear, legal reason for not answering their in-their-face questions (I mean, what journalist doesn’t know how lawyers make people shut up, even for the silliest of reasons? The buy-out seems to be the reason here. Oh, but perhaps journalists figure most people wouldn’t know how effective lawyers are at silencing people? I feel for target Shan, caught between a lawyer and a journalist. Gak.).
So Erica asks the big question. And Jacqueline Shan answers: “[I was just talking about Cold-FX inside.]”
Erica: “We didn’t hear you talk inside.” Really? They were able to track her down but were unable to make it in time for her talk?
Shan: “Our company was bought by Valeant. So I’m not allowed to make any public statement… You need to contact the company.” Pretty clear to me. It must suck for a journalist to be stonewalled by a large company, so take it out on an individual instead, eh?
Oh hey, the lowered voice method! A lowered voice hints at nefarious doings, hints there was a bacteria cover-up even though Health Canada said there is no health risk in its last statement to Marketplace.
Marketplace quotes: “Based on currently available information, the presence of E. hermannii in a finished natural health product would be unacceptable.” Health Canada clarifies:
“Our earlier language was perhaps too black and white and did not accurately convey the science behind acceptable levels
After laboratory assessments were conducted by Health Canada scientists of the product on the Canadian marketplace, a low level of the bacteria Escherichia hermannii was found. Following a thorough assessment by Health Canada Scientists, it was determined that the level found presented the lowest risk to health and safety of Canadians [my emphasis] and, as such, no recall was initiated.
It is important to note that all health products have benefits and risks. When health products are found on the market that pose an unacceptable level of risk to health, Health Canada takes appropriate steps to mitigate and manage these risks.”
To be sure, I don’t recall any mass deaths or hospitalizations from COLD-FX-related E. coli contamination back in 2008 or 2009. Do you? But who needs to prove a dangerous bacteria contamination when all you have to say is “bacteria” to spring suspicion and fear into every viewer’s breast?
I feel for Marketplace. They really had to work hard to prove their theory about this product, going here and there, running all over the planet, from city to city, from expert to expert, using cameras that produced grainy pictures in China while using excellent ones for the scenes in which Erica appears.
On to the good stuff: an interview with Don Cherry.
Love the Don Cherry interview: one science-illiterate person talking to another, talking about two totally different things. One about immediate relief, the other about prevention. Neither notices. You don’t see this in the Markeplace piece, but in the extended Cherry piece I’ve embedded above.
Cherry begins by saying he doesn’t work for them anymore and he’s a little ticked off with COLD-FX, the company. Yet, get this, he still takes four COLD-FX capsules a day and ten a day, like the hockey players, if he feels a cold coming on. He relates in the extended Cherry piece that after a lifetime of being plagued by colds, after he began taking COLD-FX, he’s had just three colds in eight years. I don’t think anyone, least of all, Marketplace, or anything, like being fired, is going to pry the product out of his hands. He likes being cold-free too much. Me too, actually.
In all the hoo-hah, Marketplace forgot to mention an important point: “in the United States alone at least 1 billion colds per year have been reported” (from Predy et al, 2005 CMAJ article) with each person catching on average two to six colds. We know each cold costs several days of lost work or reduced productivity, never mind that it makes one feel lousier than hell. This is not peanuts. Dissing an effective remedy for colds harms public health.
The meta-analysis they had done listed four studies. I took a gander at them. (Note: in the meta-analysis, they were not identified in proper reference format, but I’m pretty sure I found the ones looked at as there can’t be more than one in the same year by the same authors on the same topic.)
2004 study: Elderly nursing home residents, 90 percent of whom had received the flu vaccine, had fewer cases of flu when taking COLD-FX for 8 weeks and 12, that is, 1 of 97 versus 7 of 101 who took a placebo. Taking COLD-FX reduced the risk of a fragile, elderly person from catching flu by 89 percent. By the way, flu kills the elderly.
2005 peer-reviewed study in CMAJ (Canadian Medical Association Journal): Healthy adults who took the same dose as in the 2004 study but for 4 months caught 0.68 colds versus 0.95 for placebo and also only 10 percent caught more than one cold while 22.8 percent in the placebo group got multiple colds.
“These results are similar to those reported for zanamivir and oseltamivir therapy. These antiviral agents have been reported to reduce the severity and duration of illness by 1.5-2.5 days. In comparison, the ginseng extract treatment was found to reduce the duration of a cold by 2.4 days.”
2006 study: A variation of the 2004 study, in which after two months of use, COLD-FX reduced the risk of contracting a respiratory infection by almost half (48 percent) and the duration by 55 percent. I assume the infections were colds because they state that there was no influenza in the community during the study.
2011 study: A larger version of the 2005 study.
“data indicate that CVT-E002 at a dose of 400 mg/day or 800 mg/day is safe and well tolerated and results in a reduction in the number, severity, and duration of Jackson-confirmed URIs (upper respiratory tract infections) when taken as seasonal prophylaxis by healthy, community-dwelling older adults. Further studies with larger sample size are warranted to determine possible dose-related effects of CVT-E002.”
Ilersich concluded: “In summary, these results support the effectiveness of COLD-FX for preventing colds. There is insufficient evidence of a reduction in severity or duration of colds.” Insufficient evidence is science-speak for do more work, we don’t know one way or the other yet.
By the end of the twenty-two-odd minutes, Marketplace’s entire piece, when read between the lines and engendering Herculean effort not to be distracted by the bells and whistles, boils down to COLD-FX prevents colds. The claim it provides immediate relief needs further study; the China connection is no different than every other product we buy (have you checked where your frozen veggies are grown lately?), thus is not COLD-FX specific and is a separate topic; the bacterial contamination is old news and a non-starter. In other words, Marketplace told its alert viewers to take COLD-FX daily if you want to prevent colds.
Perhaps that’s why it ends its piece in the bathroom — with a shot of Erica and another expert washing their hands with soap, claiming that it’s more effective than COLD-FX. Washing hands with soap is effective in reducing colds. But what’s their published evidence proving their theory right? Where’s the double-blind randomized trial that compares the two methods side-by-side in reducing severity, duration, and frequency of infections, one for colds, one for flu?
McElhaney JE, Gravenstein S, Cole SK, Davidson E, O’neill D, Petitjean S, Rumble B, Shan JJ. “A placebo-controlled trial of a proprietary extract of North American ginseng (CVT-E002) to prevent acute respiratory illness in institutionalized older adults.” J Am Geriatr Soc. 2004 Jan;52(1):13-9. Erratum in: J Am Geriatr Soc. 2004 May;52(5):following 856.
Gerald N. Predy, Vinti Goel, Ray Lovlin, Allan Donner, Larry Stitt, Tapan K. Basu. “Efficacy of an extract of North American ginseng containing poly-furanosyl-pyranosyl-saccharides for preventing upper respiratory tract infections: a randomized controlled trial.” CMAJ October 25, 2005 vol. 173 no. 9.
McElhaney JE, Goel V, Toane B, Hooten J, Shan JJ. “Efficacy of COLD-fX in the prevention of respiratory symptoms in community-dwelling adults: a randomized, double-blinded, placebo controlled trial.” J Altern Complement Med. 2006 Mar;12(2):153-7.
Janet E. McElhaney, Andrew E. Simor, Shelly McNeil, and Gerald N. Predy, “Efficacy and Safety of CVT-E002, a Proprietary Extract of Panax quinquefolius in the Prevention of Respiratory Infections in Influenza-Vaccinated Community-Dwelling Adults: A Multicenter, Randomized, Double-Blind, and Placebo-Controlled Trial,” Influenza Research and Treatment, vol. 2011, Article ID 759051, 8 pages, 2011. doi:10.1155/2011/759051