Evidence-based medicine under attack
Here we go again. Evidence-based medicine (EBM) has once more come under fire, only this time the accusations are even more brutal. The EBM movement, says a group of academics at the University of Ottawa and the University of Toronto, is "outrageously exclusionary," "dangerously normative," and "a good example of microfascism at play." As an EBM believer, I guess this would make me a microfascist.
The academics, led by Professor Dave Holmes from University of Ottawa's School of Nursing, made their accusations in a dense, jargon-rich paper called Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. It appears in the International Journal of Evidence-based Healthcare.
As you can guess from the title, it's a pretty tough read, full of words like "territorialised," "interpollated," "hegemony," and "post-positivism." In essence, the authors suggest that EBM has become overly dominant within the health sciences and has excluded other ways of thinking about what treatments might work for a patient. Holmes and colleagues say that the language of EBM has become "an ossifying discourse," and that, as the main torch bearers for EBM, the Cochrane Collaboration "supplants all heterogeneous thinking with a singular and totalising ideology."
Archie Cochrane a fascist? The Cochrane Collaboration a totalitarian regime? At first I thought this paper was a spoof. But apparently not. In fact, the former director of the National Institutes of Health, Bernadine Healy, cites the paper in her latest column for US News and World Report, while putting her own boot into EBM ("By anointing only a small sliver of research as best evidence," she writes, "and discarding or devaluing physician judgment and more than 90 percent of the medical literature, patients are forced into a one-size-fits-all straitjacket").
I take a rather dim view of these attacks. It is a fallacy to suggest that EBM forces health professionals to behave like automatons. Indeed, in the classic article Evidence based medicine: what it is and what it isn't, published in the BMJ, David Sackett and colleagues made it clear that EBM involves the “compassionate use of individual patients' predicaments, rights, and preferences in making clinical decisions about their care.” If you get sick, don't you want a doctor to apply the very best evidence on what will work for you as an individual?
The alternative to EBM is a return to the bad old days when doctors used certain treatments because they seemed to make intuitive sense, or because experts told them to, or because observational studies suggested benefits, or because they had seen a case in which the treatment worked. But, as I have previously argued in an essay called Subjectivity can be inhumane, published in wjm, history is littered with treatments that seemed to make sense at the time, but that have now been proven to be useless or, worse still, damaging.
A good example is debriefing for treating psychological trauma. In their Health in Action article in PLoS Medicine, Prathap Tharyan and colleagues describe how, in the wake of the December 2004 Asian tsunami, aid agencies rushed in to offer forms of quick debriefing to survivors. Unfortunately, a Cochrane systematic review on debriefing found no evidence that brief single-session debriefing reduces psychological morbidity. In fact, the review found evidence that those who receive debriefing have a higher risk of developing post-traumatic stress disorder one year later. So it is quite possible that these well-meaning aid agencies ended up causing harm. A systematic review published today in PLoS Medicine found no evidence that any of the treatments given to patients with SARS were effective. And then, of course, there's hormone replacement therapy, tonsillectomies, episiotomies…which, for many patients, were useless or harmful.
I'll leave the last word to the wonderful Ben Goldacre, who writes the Bad Science column in The Guardian. Goldacre points out that Archie Cochrane was "a prisoner of war for four years in Nazi Germany, who has, from his abstracted position, probably saved more lives than any doctor you know." And to Dave Holmes and his coauthors, Goldacre says: "in 1936, he [Cochrane] went to Spain to join the International Brigade, and fight the fascists of General Franco. Now, what did you do with your summer holidays?"
Just discovered our blog; right on about EBM. The consumer coalition of the Cochrane Collaboration is meeting later this month in Washington, and I’m sure Bernadine Healey’s outrageous attack on EBM will be high on the agenda.
Interesting that Gavin Yamey’s last word, and next to last word, on evidence-based medicine (EMB) involve anecdotes. In order to convince, he needs emotion and a story. It also seems that terms such as “hegemony” are too high fallotin’ for him. Rather than deal with the substance of Holmes’ paper, he is content to ridicule the language from a clubby, anti-intellectual, know-nothing perspective. (“I guess I am a microfascist” he joshes. Well…..)
I don’t know the personal background of Gavin Yamey. I don’t know whether he is a hero, as is Archie Cochrane, whether he personally suffered for freedom, or not. But when it comes to this argument, such moral history seems not very compelling. It is rather like giving Chairman Mao a pass because he DID go on that long march. Since this was for freedom, so is everything else that followed. Of course, I would never compare Archie Cochran to Mao or anyone else. I suppose if you know the man you have a right to defend him personally. It is obvious, however, that all sorts of bad ideas and practices spring from heroic and wonderful intentions.
Yamey sites Sackett’s describing “evidence-based medicine” in his original article as “compassionate” etc. and we are to conclude that therefore it is. The tone of this Blog is not at all “microfascistic” but it certainly is Orwellian. I would say the same about EBM, despite the compassionate language of supporters such as Sackett (or precisely on account of such language.)
I will add that there are other approaches to science and to evidence that do not require a return to the bad old past. We are cut off these by EBM in precisely the manner that Holmes accurately describes, albeit in a post-modern idiom. I refer, for starters, to a article I wrote on this topic long ago for “Perspectives in Biology and Medicine” — the central arguments are as relevant as ever — available in two parts on the web at part one and part two) Jargon is always a problem, and I would agree that a more judicious use of language might have strengthened Holmes argument, made it appeal to a larger audience, or a least immunized it to a degree against the sort of classic ad-hominem attack mounted by Yamey. But sometimes new terms are useful. Some, such as “evidence-based-medicine” might be catchier than others — not necessarily a benefit, and certainly one of the problems Orwell was getting at.
I’ve been doing science for my entire career. It’s not a popular view, but what I’ve seen of the medical profession does not engender in me the same kind of trust that I have for other sciences.
Based on knowledge acquired and refined repeatedly and relentlessly, I trust the materials scientist. I trust that a simulation set up to test the vibration of a fighter jet at Mach 3 will give back reliable, repeatable results.
I trust that gravity will take the remote out of my hand and throw it to the floor. I trust that a laser will lase, that a computer will compute, and that my anti-lock braking system will stop my car before it crashes.
I trust that the avionics in the 737 I took from Detroit to Atlanta will get it there with a probability of well in excess of 99.9999% (1 in a million). (In fact, six 9’s is a wholly unacceptable safety rate for commercial aircraft.)
Doctors, on the other hand, scare me. True, the root cause of the lack of surety is that living systems are vastly complex and that experimenting with life and death is not acceptable. However, this does not mean that experience and a confident, sage demeanor give me anything like the trust I have in a flying tube of aluminum.
In my view, evidence based medicine is a baby step toward the same mechanistic toward the ideal of understanding living organisms that the paper derides.
Medicine should be science. It should be engineering. What a doctor does to a patient should be viewed as maintenance and repair, not a treatment or values. As it is, there is far too much guesswork and loose probabilities and “let’s try this to see how it works out”. We should know with high confidence how it will work out.
The thing is, we are already creating stable systems across many fields that approach the complexity of biology and they work. While our knowledge of how we work is incomplete, all mysteries will of course be penetrated, it’s only a matter of when.
Let’s make it sooner rather than later. A concerted, systematic effort to understand everything about ourselves can only succeed in the same way a concerted, systematic effort produced a nuclear bomb or a space shuttle.
Momentum holds us back. I don’t want explanations why my L5S1 disk disintegrated or councelling or pain relief; I want it repaired, and if the design is lacking, I want it corrected and refitted. I want the control to have it done the way I choose, and I want the power to diagnose it myself with readily available sensors and tools.
It’s not well and good to engage in any activity that holds back our understanding in any way.
If I understand the arguments of the paper, the principle worry regarding EBT is that the regimented and structured approach takes away choices from the caregiver/patient, which really means “takes choices from the caregiver”, because the patient is usually fed information to lead them to the choices the caregiver wants him to make.
Another principle worry seems to be that procedures and best practices developed will not undergo the type of refinement that corrects most scientific models, but instead mechanize the doctors. This doesn’t mean that the approach should be scrapped or backed away from, it means that it should be refined — each application of a best practice should be recorded in every detail for feedback into the system.
In any case, we shouldn’t be led down the path away from ideal of complete understanding in the name of traditional relationships.
The paper under examination here is an example of exactly the wrong way to go. It argues using philosophy, such as “what is evidence?” (Science answered this question centuries ago). It argues uncertainty and ignorance, “what does the woman with 40% chance of developing breast cancer do.” (A model that predicts an outcome with 40% probability is a completely unacceptable.) It argues emotion and comfort. It argues organization and authority.
What it should be doing is presenting results and scientific models (with real math and physics and chemistry in them). It should point out weaknesses in existing models and propose new ones.
That’s what science is, not arguing with sophistry and politics from a basis of an emotional reaction to perceived loss of freedom.
Enjoyed the post and responses. I hope you will send your responses to US news for rebuttal/publication.
My 2 cents: The idea that we’re even using EBM in a meaningful way is pretty laughable. That which we do have evidence for, physicians rationalize themselves out of; the vast majority of issues we simply have little evidence on what is good practice.
Hippocrates knew that “Experience is delusory,” and we are just now getting our house in order and we have a long way to go.
Best practices and protocols need to be implemented across the board in states as a standard mode of practice.
The medicals sciences have sorely departed from the basic initial screening systems of care that use to consist of the standardized flowcharting documenting that would have flagged many conditions in the past.
Now many newly graduated medical doctors just glance at a patient’s chart or rely or some computerized output where as in the past older physicians collected detailed H&Ps: history and physical with a breakdown of the patient’s review of the systems and systemic testing and comprehensive labs and EKG, UAs, and so forth.
A physician with limited practice experience is highly unlikely to catch many border line abnormal labs, EKGs, unless he develops the practice experience by doing the detailed H&Ps on patients and eliciting the information from the patients in order to correlate the symptom to a problem.
One has to COLLECT the evidence in order to locate
If one does not bother collecting the evidence, obviously one will never find the source of the problem, nor aid the patient.
The controversy over EBM, why and why not, is easy to understand reading the comments about Gavin Yameyâ€™s blog, certainly the comments about trusting materials science, etc.
It is easy to take a simplistic approach to what should be done in medicine, if ones background has been in a narrowly specialized science environment. It would be a wonderful day if medicine could be done by passing a sensor over a patient. Upon getting a perfect diagnosis, she could be treated with another high tech gadget, or nanobots or some such.
The truth is that the human body is an incredibly complex system of interdependent subsystems encompassing every other scientific discipline. This unthinkably complex organism is controlled by software that is only very slowly yielding to scientific inquiry. This inquiry is hampered by the fact that it is individualized for each organism, based on the largely unknowable experience of each organism and on the still imperfectly knowable genetics of the organism. Until we are able to completely understand the how and why these systems interact as they do, and how the â€œmindâ€ is able to regulate and control their interaction, the â€œevidenceâ€ in evidence based medicine will be at best only a rough guide to treatment.
The knowledge we need is too dependent on things we do not know or understand for our current EBM to be taken as gospel or absolute truth. Until we collectively, in all the sciences, can speak to each other in language we can all understand and can work collectively toward common goals, EBM will only be an unfulfilled promise and hope of things to come. It may be the best information we have, but as Dennis Schmitz said about airline safety statistics it is, â€œnot good enough.â€
Well said, drjerr. The dichotomy of evidence based versus treatment and procedure based is from a second, third and so on persons perspective, the meaning of the discord lost without the first person, the client/patient.
With that first person again viewed neither a basis of evidence nor one of procedure, classification and treatment should prevail at the cost of sacrifice of the other. This is, after all, medicine that we discuss, and not mere academics.
However reiterated, these age old arguments evidently bring to light semantics, the word medicine used with different meanings, with fuzzy, non-definitive focus on the conditionals practice and research. Common conditionals that offer further definition, failure to clearly incorporate the greater definitions makes the discussion one of semantics.
The attacks on EBM are unsurprising. EBM is hard and not many have the capability to do the work. Professions other than medicine feel this particualrly as they do not have the infrastructure of training and history of research. All this sits beside the more fundamentalist opposition to the special claims of science as the most reliable route to uncovering nature’s secrets. In the end science will win as the findings of alternative aproaches are unreliable and largely ineffectual.
Of course, there are many valid critiques to the way in which the medical establishment does itâ€™s business, perhaps starting with the fact that it is a business. However, in my view, the last thing we should do is to pay any serious attention to a post-modern critique of medicine, or science, or literature for that matter. It is a bankrupt philosophy that substitutes cleverness and obfuscation for clarity and honesty. In my experience living in the Bay Area, other â€œways of knowingâ€ are usually a substitute for the hard work of actually proving something.
In order to find out whether or not a given treatment works, researchers have to obtain evidence. That evidence should be as reliable as possible; as in, if I take this drug it is likely to make me better. It should not rely on what we want to be true, but is what is actually happening. We should not draw broad conclusions based on a limited number of cases, because there is a great deal of variation in all biological systems and there are exceptions to every rule. Conclusions based on that evidence should be drawn based on an awareness of statistical probability so that we avoid being fooled into seeing patterns where there are none. Conclusions must always be tentative and should be altered based on new, more reliable evidence. Given how little we know about complex biological systems, broad, overarching theories should also be avoided, since they are almost certainly wrong.
Now all of these statements seem to me to be reasonable, practical and even obvious. Insisting that people who make claims about medicine follow their guidance is no more â€œfascisticâ€ than are traffic laws. There simply is a physical and biological reality that makes certain demands on those of us that want to understand it, and medicine will never progress unless we understand more about our bodies and how they work. Itâ€™s just that simple. Now, having said that, it also should be clear that, since we really donâ€™t know nearly enough about biology, we should accept that evidence based medicine will always be deeply flawed. We will often be wrong, and this will cost lives. But the alternative, to abandon an insistence on evidence, would cost far more lives.
The article was written by people from Ryerson University, not the University of Toronto. Ryerson just happens to be in Toronto and so the article contains a reference to Toronto, Canada.
No, medicine is NOT a science. It is a profession whose purpose is to heal the sick. It certainly uses science, but its primary mission is to make people better, not to understand their biology. Of course, understanding biology (a real science) is key to understanding medicine, but most doctors are emphatically not scientists. In most cases, I would say that they care far less how a given medicine works, that that it works. That is why so many doctors used medicines “off label”. They think it won’t do any harm (which is the first part of the approval process of any medicine), and they guess that it might do some good. If it seems to, they keep using it. However, there is a problem here. Let’s say that the off label use really does help, but in the particular group of folks they are giving it to, one in a thousand patients dies. Well, no single doctor is going to treat a thousand such patients, so they will be completely unaware of the potential danger of this use of the medicine. Further, if the occasional patient dies, they will have no idea that this use of the medicine is the problem. Of course, as the author says, there are a huge number of variables in human biology, and consequently nearly all conclusions are tentative, but that is no reason not to try our best to determine scientifically which treatments work and which don’t. As the author of the last post says, much of what doctors do is make educated guesses, which is fine. That is what most scientists do as well. The difference is that scientist have to be very clear about when we are guessing and when we actually know the answer. The same is not always true for doctors. Most of us who have been patients have had the experience of having doctors state something with complete confidence even when they really are just guessing, or doing something that seemed to work in the past, or that an other doctor said worked for them. Of course they have to do that, and they have always done that (even when they were bleeding patients and force feeding them arsenic) because no one wants to be treated by someone that doesn’t seem to know what they are doing. Some people call this arrogant, but I think it’s an unavoidable requirement for being a doctor. Unfortunately, I think this can also lead to an unwillingness to be confronted by evidence that their personal favorite treatment actually doesn’t really work. So, let’s not be naïve about what science can and can’t do, but let’s also not be naïve about how scientific most doctors actually are.
Sure, Evidence Based Medicine sounds great, but the words are not synonomous with their meaning. There are thousands of variables in medicine: For example certain condions are treated differently on different parts of the gobe. Also different ethinicities often require different treatment protocols. This term evidence based medicine is a politicized term andis no substitute for the well educated provider, who uses rigourous medical studies as part of his decision making in coming to the right assessment. The term was invented ten years ago by managers and policymakers to try to guide Medical Doctors, Physician assistants,etc, in their decision making.
“Actual MD”, you may be a qualified doctor, but that does not make a qualified scientist, and that is my point. What you do, day to day, is much more similar to what a mechanic does than what a scientist does. The point was not that there is no such thing as medical science, as a branch of the biological sciences, but that most doctors may not be particularly scientific in their approach. I agree, that given the complexity of biology, it is unlikely that evidence based medicine will be the kind of panacea that it’s advocates expect, and doctors are often left with a best guess, or what you call “probabilistic reasoning”. It’s also pretty clear that the “evidence” is so often contradictory that a doctor really is left relying on his or her own experience.
So be it. But that only reinforces my point. In fact, doctors can’t be scientists, because their primary goal is not some universal truth, it is to cure their patients. This is why so many studies performed by doctors are essentially anecdotal case studies, because that is the world that they live in, at least the good ones.
So, to summarize. Being a good doctor is like being a good mechanic. The more you know about the engineering behind the cars the better, but in the end the goal is not understanding, it is effective repair. You don’t have to be an expert at everything, and it is certainly more important that you are effective as a doctor than that your are effective as a scientist, but maybe you should be open to what real scientist have to say about the effectiveness of your treatments, even if it make you uncomfortable.
Apparently, this author can’t even wrap his mind around the possible fact that there may be some serious toxicities to the dominance of EBM.
The author’s apparent first order of business is to dismiss this attack on EBM because of its language. Apparently words such as “hegemony” are too complex and terms such as “intention-to-treat analysis,” and “logistic regression -derived odds ratio” are straightforward. Come now.
Next, the author presents the rather obvious rationale for EBM. If well constructed studies clearly show a treatment to be ineffective or harmful why would anyone choose the treatment? Come now. A 6th grader could figure this out. It is not EBM that is being attacked per se. It is the dominance of EBM that is being criticized. The author completely misses this point and consequently, instead of actually learning something goes into “counterattack mode”.
The dominance of EBM is clearly having widespread toxicities on the production of new knowledge and creative thought and secondarily, promotes a myopic biomedical approach to certain problems. For example, how many dozens of new antidepressants does EBM have to discover before we realize that depression is a much deeper problem, linked to our social and cultural toxicities, than to 5 of 9 symptoms for 2 weeks. Come now.
The biggest issue I have with EBM is showing something achieves a desired result without taking into consideration the totality of the situation. Acid blockers, a great example, do they reduce irritation of the esophagus, yes but at what price? the reflux is not reduced, in fact increased and the side effects of having an abnormal ph of the stomach are great. Statins, do they reduce the total ldl number, yes but in many cases by effecting the liver in such a way that IR is increased and the density profile is worse. The price to pay for the single desired result is not looked at in most EBM. We need to start looking at patients as complex adaptive systems and take into consideration how the complex system adapts to the treatment being studied.