2012-08-24 18:00:00.348883+00 by Dan Lyke 11 comments
Charlene and I have been talking about how to read a scientific paper, and I thought this would be a good start: Echinacea for treating the common cold: A randomized controlled trial.
comments in descending chronological order (reverse):
#Comment Re: made: 2012-08-27 18:51:47.22221+00 by: ebradway
I don't mind fooling myself as long as the desired outcome is real. I would prefer the outcome to be repeatable, at least for me. But I've encountered enough health issues that seem to have "wandering cures" that I'm happy just to find the next cure.
Probably the biggest cause of health issues is just being alive. I deal with an insane level of self-inflicted stress and anxiety. Some of the symptoms are increased weight, poor sleep, and occasional depression. But I remember experiencing the same symptoms about having to go to pre-school. There's probably only one "cure" for these symptoms. At the same time, I've managed to develop a level of tolerance for stress and anxiety that allow me to accomplish things.
It's kind of like losing a couple toes to frostbite while climbing Mount Everest. ..
#Comment Re: made: 2012-08-27 18:04:15.968028+00 by: Dan Lyke
I think that "...with many holistic approaches is that success can depend heavily on the relationship between the caregiver and the patient..." is exactly what makes evidence based medicine so difficult: Outcomes can take longer than human lifespans ("how many of your patients died?") to measure, and much of what we're looking for from medicine is a qualititative "I want to feel better".
But I'm also one of those people who wants to know when I'm fooling myself, even if fooling myself leads to the desired outcome.
#Comment Re: made: 2012-08-26 22:36:34.542689+00 by: ebradway
That slip of the tongue actually creates a nice allegory:
"Not statistically significant means the same conclusion would be reached even if the numbers are transposed."
I sometimes wonder if the reason certain fields have abandoned quantitative methods is that they want to draw conclusions absent statistical significance. The biggest problem leading to a lack of statistical significance is sample size.
For many holistic health approaches, the sample size can be very small (e.g., It worked for me!). Does lack of statistical significance mean it didn't work? There are many holistic treatments that I continue to use because I find relief. There are also many treatments I avoid because I do not believe I will find relief and there is little evidence suggesting I should.
The other challenge I've noticed with many holistic approaches is that success can depend heavily on the relationship between the caregiver and the patient. Western Medicine is based on the scientific premise of repeat-ability. If two doctors administer the same treatment in the same manner, there should be no difference in outcome. This is not the case at all with holistic approaches.
#Comment Re: made: 2012-08-26 18:26:31.803606+00 by: spl
If you want more practice, use Google Scholar alerts to receive regular emails on recent papers of interest.#Comment Re: made: 2012-08-26 04:15:42.158152+00 by: Dan Lyke
Yes, agreed.
#Comment Re: made: 2012-08-25 22:57:51.399153+00 by: ebradway
Sorry. I meant "transcribed" but "transposed" is actually telling. What "not statistically significant" means is that the numbers could be transposed and you would reach the same conclusion.
#Comment Re: made: 2012-08-25 17:33:27.963497+00 by: Dan Lyke
When I dig through it with Charlene I'll see how she responds, my take was that in that sentence they were explicitly making a qualitative judgement based on their readings of the other previous studies. But, yeah, it seems a little extreme to be making that statement.
And which numbers did I transpose? Paper says:
Mean illness duration for the blinded and unblinded echinacea groups was 6.34 and 6.76 days, respectively, compared to 6.87 days for blinded placebo and 7.03 for no pills.
#Comment Re: made: 2012-08-25 04:42:29.040826+00 by: ebradway [edit history]
Unless the authors made an error with their statistics, the statement "The observed shorter illness duration and lower severity seen in the echinacea groups were not statistically significant" means you cannot interpret any conclusions from the presented statistics. The authors were mistaken in their interpretation that there is a small beneficial effect.
If done correctly, statistics are quite simple. If there is significance, then you can make interpretations based on the numbers. If there is no significance, then the numbers are meaningless. The values you carefully transcribed in the table above are statistically equivalent. There is no difference between 7.03 and 6.34.
The "higher than expected variability" just means they need to start over. It doesn't mean you get to interpret statistics that lack significant differences.
#Comment Re: made: 2012-08-24 21:48:24.297554+00 by: Dan Lyke
Oh, from the conclusions:
In conclusion, our own interpretation is that there is likely a small beneficial effect attributable to echinacea’s pharmacological activity. ... Nevertheless, if there is indeed underlying benefit, it is not large, and is not clearly demonstrated by this trial’s results.
#Comment Re: made: 2012-08-24 21:28:35.055375+00 by: Dan Lyke [edit history]
I haven't dissected it, but 719 participants enrolled, 713 completed, data longer than 2 weeks thrown out, 4 random groups:
| Group | Illness duration (days) |
| blinded echinacia | 6.34 |
| unblinded echinacia | 6.76 |
| blinded placebo | 6.87 |
| no pills | 7.03 |
Although blinded echinacea had a half-day advantage, it was deemed not significant because the overall variability of the data was far higher than the researchers expected. My own commentary: Especially since the unblinded was so close to placebo.
But far more important than the results will be reading through the methods, looking at the cites (there have been a number of studies on echinacea with different results, and it will be interesting to compare and contrast), and then looking at how the press release and the popular press covered the paper.
#Comment Re: made: 2012-08-24 20:46:12.736283+00 by: Jack William Bell
What's the TL;DR?