This blog was originally published on 7th September 2020 on the Psychology Today website as: https://www.psychologytoday.com/us/blog/our-turbulent-minds/202009/should-ect-be-judged-the-standards-homeopathy
Extraordinarily, we now live in a world in which the best evidence [https://www.rcpsych.ac.uk/mental-health/treatments-and-wellbeing/ect] that the Royal College of Psychiatrists has been able to muster for the real-world effectiveness of ECT exactly parallels the claims for homeopathy.
ECT is defended energetically by traditional psychiatrists, and many claims are made, but systematic reviews of the available evidence conclude that there is no reliable evidence [https://connect.springerpub.com/content/sgrehpp/early/2020/04/02/ehpp-d-19-00014] that ECT helps lift depression significantly after the end of the treatment (although there is some evidence for a significant placebo effect and the induction of a convulsion through the passage of electricity through the brain may lead to a temporary dissociative and euphoric state [https://connect.springerpub.com/content/sgrehpp/early/2020/04/02/ehpp-d-19-00014]).
This leads to a problem – for a profession and those professionals who would like to believe in the effectiveness of the procedure. It appears, as an outsider, that ECT holds a special place. In general, psychiatry tends [https://academic.oup.com/shm/article-abstract/32/2/395/4833896] to defend the effectiveness of any particular class of drugs in vogue at any particular time (before realising the harms done, changing the narrative, and then defending the new status quo). Despite the claims to be practicing evidence based medicine, there seems to be more effort expended in defending practices than developing more effective and humane alternatives.
We could speculate that this may be because psychiatry needs to defend its place as a subspecialty of medicine (and therefore requires the use of conventional diagnosis and medical, physical, treatments), perhaps because it is felt that there must always be a back-stop, a treatment which has the twin benefits of drama and (it is claimed) “miraculous” effects [https://theconversation.com/electroconvulsive-therapy-does-work-and-it-can-be-miraculous-76381] or perhaps guilds merely tend to defend their members.
But in any event, ECT requires defending. And ECT needs to demonstrate its effectiveness… in the regrettable absence of any real evidence.
In 2013 a different group of people were lobbying for the effectiveness of their practices. The UK Parliament was debating the management of ‘long term conditions’ and specifically which forms of care should be commissioned by the NHS. The Health Select Committee held hearings and invited submissions from interested parties.
One of the organisations supplying evidence [https://publications.parliament.uk/pa/cm201415/cmselect/cmhealth/401/401vw60.htm] was the Society of Homeopaths. Part of their evidence included the claim that 68% of people receiving homeopathic treatments were reported that their problems were “better” or “much better”.
This is quite a claim, given the widespread view in the medical profession that homeopathy is little more than an expensive placebo.
It turns out that this evidence was gathered in a study [https://www.liebertpub.com/doi/abs/10.1089/acm.2005.11.793] in which the clinicians (the homeopaths) rated their subjective judgments as to whether they believed their patients (who had received the homeopathy treatment) were; “much better”, “better”, “slightly better”, “no change”, “slightly worse”, “worse”, or “much worse”. The authors reported that 65.8% of children were reported as “better” or “much better” (it seems that the exact figure slipped a little in reporting). To be clear; these were not the judgments of the patients as to whether their treatment had helped. These were the judgments of the clinicians.
The Society of Homeopaths were unconvincing. Their ‘evidence’ did not feature in the Select Committee’s report [https://publications.parliament.uk/pa/cm201415/cmselect/cmhealth/401/40102.htm] and, a few months later, the Chair of the Select Committee, Dr Sarah Wollaston (a practicing GP as well as MP) tweeted; “It’s homeopathy awareness week. Will @rpharms issue clear advice that there is no evidence that homeopathy is anything other than a placebo” [https://twitter.com/sarahwollaston/status/610072583669682176].
In other words – and I agree – it simply isn’t good enough to base healthcare decisions on the subjective opinion of the treating clinicians. There are significant vested interests that could give rise to unconscious bias. And there are risks of artefacts; if treatment continues until, whether causally related or not, the patient reports an improvement, mathematics tells us that the outcome is likely to be “improved” if clinicians report back when treatment ends.
The scientific methodology applied to healthcare is intended to address these issues… and I agree with Sarah Wollaston and the Royal Pharmaceutical Society “no effect beyond placebo” [https://twitter.com/rpharms/status/610082078743523328].
Interestingly, the Royal College of Psychiatrists recently revised their guidance [https://www.rcpsych.ac.uk/mental-health/treatments-and-wellbeing/ect] on ECT. In their leaflet for the public, they also claimed that “around 68% of patients were “much-improved” or “very much improved” following ECT.
The data for the supposed effectiveness of the controversial treatment comes from a database compiled by members of the Royal College of Psychiatrists’ ECT Accreditation Service (ECTAS). Their members routinely collect and compile key information on the use of ECT, and publish [https://www.rcpsych.ac.uk/docs/default-source/improving-care/ccqi/quality-networks/electro-convulsive-therapy-clinics-(ectas)/ectas-dataset-report-2016-17.pdf] the findings.
The methodology used in compiling these reports relies on ‘CGI’, or ‘clinical global impression’, scores. Here, a clinician rates their subjective judgments as to whether they believe their patients (in this case, people who had received ECT) were; “very much improved”, “much improved”, “minimally improved”, “no change”, “minimally worse”, “much worse”, or “very much worse”. The Royal College of Psychiatrists reported that 68% of people receiving ECT were judged by their clinicians as “much-improved” or “very much improved”.
In other words, the evidence for the effectiveness of ECT is functionally equivalent to the effectiveness of homeopathy. In both cases, the proponents of the treatment rely on the subjective judgments of the prescribing clinicians to evaluate outcomes, and, remarkably, on almost identical scales, with the same outcome – that homeopaths believe that homeopathy has resulted in 68% of their patients recovering and psychiatrists believe that ECT has resulted in 68% of their patients recovering.
When homeopaths present evidence such as this, quite rightly, senior doctors are scornful, pointing out that “there is no evidence that homeopathy is anything other than a placebo” [https://twitter.com/sarahwollaston/status/610072583669682176].
Despite the professional packaging and the protective carapace of a Royal College, the evidence for the effectiveness of ECT remains decidedly dubious.