One study is never good enough, because medicine and statistics are complex, scientists are fallible and external pressures are swaying – so here is a way to present the bulk of literature in a handy visual way
Science based medicine, a term coined by Dr. Steven Novella (of neurologica, skeptics guide to the universe and SBM blog fame), is the idea that medicine should be based on scientific prior knowledge and not only evidence, when evaluating new concepts. This challenges a few notions in regular clinical practice but it is world shaking when it’s woo time. The idea is such: if you claim that water has memory, that carries the energy of a single lost molecule of a substance that magically removes the symptoms it would cause were it in a large dose than you can’t just do a study to prove it right. For it to be right you need to overturn all physics, all chemistry, all biology and all medicine, so you would need an enormous effect size, conclusive evidence under the most rigorous of examinations and then still you would need to explain all the science that says it can’t be. Science based medicine is a tall shield against inane pseudo-scientific bullshit.
There is one problem, and that it convincing patients that their favorite brand of woo is not based on anything real. This problem stems from an axiom: Disproving any bullshit is an order of magnitude more difficult than presenting it was. For any silly claim you need the body of evidence, rigorous reading of the scientific literature and parsing it for the willfully ignorant, and that takes time and effort. There isn’t just “one study that shows X works” for any given X be it woo or science. You need preliminary studies that become better studies as knowledge advances that become high quality, blinded, randomized clinical trials with many participants. You need to reproduce your results a few times. You need to see the big picture. One study is never good enough, because medicine and statistics are complex, scientists are fallible and external pressures are swaying.
So here is one possible, interesting solution I was presented with, and I would like to take the time to present to you. It is called Treatment Scores, and the whole explanation can be found in this blog. I will be using this format from now on to present the summary of my results. It is an attempt to transform clinical decision-making into a more scientifically based process, that is easier to understand and easier to explain to patients. What I love about this is the way it presents information in an easy to understand manner, that is transparent to clinicians and patients alike. I like my patients to be able to understand the way I came to a conclusion on the best management for their illness and I think that doctors would like to introduce more transparency of the process they took. I think transparency inspires trust and better doctor-patient communication. This application is far from finished, and should become more automated as time goes by, but I think that for now it is a very interesting idea, worthy of time and effort to support and spread the word about. Hopefully, in the future, this will become an almost automated tool for quick review of the scientific basis for the various treatments available for most diagnoses.
And now, lets get started and discuss Black cohosh for menopausal symptoms. Black cohosh is a plant originating in north america that was allegedly used by native amercians to treat infections, kidney problems and depression. Today it is a popular herbal supplement used to treat menopausal symptoms in women, especially the bothersome hot flashes. I include a few studies here, as usual, and in the end I will present the summary through Treatment scores.
First is a study of black cohosh on 99 women (with 16 who dropped after baseline and another eight who dropped out after 5 weeks) which is double blind, placebo controlled crossover study. The symptoms were evaluated using symptom diaries, and a questionnaire called the Greene Climacteric scale (GCS). There was no difference between the groups, nor was there difference per patient on both arms of the cross-over. So this is a generally high quality study albeit with a large drop-out rate with a negative result. This study, of course, gives black cohosh a score of 0.
Another study of black cohosh for menopause, this time regarding anxiety in menopause. It is a small (30 patient) study, randomized, double blind and placebo controlled and its population was post-menopausal women with diagnosed anxiety without other psychiatric comorbidities . The total follow-up was 12 weeks and 75% of participants completed the study, one was due to side effects but the general side effect rates were not significantly different from placebo. The results – Black cohosh was no different from placebo and was actually worse on one scale, the GCS!. The treatment score here is 0, and this goes as a secondary statistic.
A four arm study of black cohosh vs red clover vs placebo vs hormone replacement therapy for hot flashes. It was double blind, randomized, and had 22 patients in each arm for a total of 88. The symptoms were evaluated using a symptom diary and the GCS, previously mentioned, and the follow up was 12 months. The results – only hormone replacement therapy had significant benefit, black cohosh occasionally did significantly worse than placebo and was worse in a non significant manner throughout follow-up, so a treatment score of 0 is alloted. There was no change in bone density or any side effects between groups. This had a daughter study that tested cognitive functions on the same population and found no benefit over placebo for black cohosh or red clover. This is a secondary statistic with a treatment score of 0.
Another study, published in a low impact korean journal, compared a St. John’s wort and black cohosh preparetion vs placebo for all symptoms of menopause. It was double blind, placebo controlled, and was supported by the company that produces the supplement. The results are rather positive on the general menopausal KI score measurement, but the hot flash results, though significant and positive, are not very clinically impressive: on a scale of 1-10, both groups began at around 7.5 and finished at 1.5 ± 1.97 for the supplement and 3.54 ± 3.38 for placebo so there is a large overlap and very large placebo response. This 20% improvement over placebo is suspect due to conflict of interest, and the supplement is made of two different plants. I can still give it a careful score of +5.
A double blind, placebo controlled study, published in the annals of internal medicine used a naturopathic (eye roll) approach and gave patients either black cohosh, multi-herb supplements, herbs and soy rich diet, placebo or hormone replacement therapy. This study measured all menopause symptoms using a questionnaire and measured hot flash frequency and severity, and had 351 participants. The results for all interventions except for the hormone replacement therapy were as good as placebo, and the HRT was significantly better at reducing hot flash frequency and ameliorating all symptoms. This well done study gives a treatment score of 0. No significant adverse events were observed.
At this point I was satisfied with the results. I found no large, high quality studies other than those included. I decided to go check the side effect profile:
A longitudinal prospective study with 12 months of follow up saw no liver toxicity in 87 post-menopausal women.
A review of published cases reported a few tens of severe autoimmune hepatitis cases linked to black cohosh consumption. These are rare, severe side effects, that are correlated with the herb, but making a cause and effect statement here is difficult.
Not much besides that and what was already covered.
So, my summary would be: Black cohosh gets a treatment score of 0 due to multiple high quality studies showing no effect for ameliorating hot flashes in menopause. It has been linked to a few rare cases of hepatotoxicity inconclusively, and has monetary cost, so it has negative secondary scores. I summarized it as a treatment with a total score of -10 due to no effect, possible side effects and cost. The way it is displayed in the treatment score application:
As you see, Treatment scores provides a handy visual guide to aid in decision making. I think this program has a great future ahead of it.
Hope you enjoyed this demonstration and review, I am, as always,
A wide eyed skeptic.