In this response we will clearly evidence why Hayes and Jauhar’s blog critique of our systematic review was often factually incorrect and was replete with misrepresentations (and/or misunderstandings) that will lead some readers to conclude wrongly that our findings are not robust.
The seven points we will make are that Hayes and Jauhar:
1) allege, without justification, incompleteness in our search strategy, while it was as least as thorough as other systematic reviews in this area
2) misrepresent or simply misunderstand our exclusion/inclusion criteria
3) appear entirely unconcerned by the bias in the RCTs, while overstating the bias in the surveys
4) wrongly imply we did not declare our conflicts of interest
5) misrepresent or simply misunderstand why RCTs and naturalistic studies did not inform our severity estimates
6) allege data extraction errors we did not make
7) by raising the issue of differences between the drugs, commit the all-too-common fallacy of criticising a study for not doing what it never set out to do
Before we consider these points in turn, we would first like to explain why we are posting our response here.
On the same day as Hayes and Jauhar’s published their blog critique (18 Oct 2018), we invited them to submit it to the journal in which we published our article, Addictive Behaviors, so we could respond to each of their points in the appropriate peer-review setting. As they only submitted to the journal on 2 Nov 2018, and as the peer-review/publishing process will now take additional unspecified time, we have taken the decision to respond to their blog post here, given that there is growing appetite in the professional and service user communities for our response, and that we are anxious not to endure any further delay in setting the record straight. We have also sent this response to Mental Elf for publication.
1) Search strategy
Hayes and Jauhar (2018) allege that it is ‘highly likely’ that our search strategy did not find all relevant studies. We are confident that our strategy, which included MEDLINE/PubMed, PsycINFO, Google Scholar, previous reviews and the bibliographies of 20 relevant papers, is at least as thorough as most systematic reviews with regard to identifying published studies. We have subsequently searched for relevant unpublished theses, dissertations and conference proceedings, in ProQuest and OpenGrey, and found none.
2) Inclusion and exclusion criteria
Hayes and Jauhar point out that we did not register our inclusion criteria in advance. In response, aside from PRISMA not obliging such criteria to be preregistered (see item #5), we clearly state our eligibility criteria (which complies with PRISMA guideline item #6, while further address PRISMA items 7-10# in the methods section). Additionally, we surpassed most systematic reviews on depression treatments as only a minority (around 30% – Chung et al., 2018) include lists of included and excluded studies, as we do.
Hayes and Jauhar then suggest that we should have identified the length of follow-up, length of antidepressant exposure, and drug-company funding as reasons for exclusion before undertaking the search and data extraction. Firstly, we went beyond both what PRISMA requests and the procedure of previous systematic reviews in this area, by explicitly stating in each relevant section a rationale for excluding each of the studies omitted. Because we did so, any diligent reader will clearly see that none of the three variables that Hayes and Jauhar raise were the sole reason for excluding any study at all (hence our not identifying them as sole reasons for exclusion before our search and data extraction). For example, as the tables make clear, six of the included 24 studies were clearly identified as drug-company funded. Furthermore, the five drug-company studies excluded from our estimate of incidence, while indeed reporting artificially short durations, were excluded on the quite obvious ground that they failed to report incidence rates. This was all plainly stated, even though Hayes and Jauhar wrongly imply otherwise.
Hayes and Jauhar also take issue with our excluding two studies from our estimates of incidence “because they assessed only 9 withdrawal symptoms”. This is, once again, misleading. We did not exclude them on this basis alone but, as explicitly stated, on a variety of methodological considerations. For example, both studies were ‘chart-reviews’ of medical notes, which are notoriously weak owing to their reliance on practitioners being aware of, and recording, withdrawal reactions, while one study, oddly enough, excluded any withdrawal reactions commencing three days after discontinuation.
Finally, we note that Hayes and Jauhar only find reasons to challenge the exclusion of studies with relatively low incidence rates, but do not find fault with, or even acknowledge, our exclusion of a study with a 97% incidence rate.
3) Assessment of study quality
Hayes and Jauhar state ‘This review does not attempt a traditional assessment of bias in the studies they include’. The methodology of every one of the 24 studies is described, in text and tables, so that readers can assess for themselves their quality, including any sample biases. Furthermore, our ‘Limitations’ section acknowledges the potential minimising bias of the RCTs because of their artificially short treatment and follow-up durations (about which Hayes and Jauhar express no concern), and the possible maximising bias of the surveys because they may attract a disproportionate number of people unhappy with their drugs (about which they express grave concern). We also pointed out, however, that surveys can be prone to bias either way – e.g. one of the largest surveys included contained unusually high proportions of people who thought the drugs had helped them, so it is feasible, in this case, that the sample bias may have been towards people with a generally positive attitude about antidepressants, and therefore the study underestimated adverse effects such as withdrawal. While the RCTs had extremely artificial samples and conditions (and small numbers) the large online surveys, while not necessarily representative of all users (like the RCTs), represented the real life experiences of several thousand people with a range of treatment durations (from weeks to years) and various speeds of withdrawal.
4) Conflict of interest
Hayes and Jauhar appear to imply that we may have undisclosed ‘ideological’ conflicts of interest, something that can presumably be alleged of any researcher, author or, indeed, blogger in this area. We fully abided by the Conflict of Interest policy of the journal in which we published (Addictive Behaviors, 2018).
5) Outcome measures
Hayes and Jauhar rightly state that in three of the incidence studies that we reviewed some withdrawal symptoms (as identified by the DESS) were also present in some of those continuing to take antidepressants. However, Hayes and Jauhar wrongly claim that in the Zajecka (1998) study withdrawal incidence is ‘higher’ in those continuing to take antidepressants compared to those stopping. The difference between the two overall rates was clearly stated in the study as not statistically different. Furthermore four specific withdrawal effects (dizziness, dysmenorrhea, rhinitis and somnolence) were significantly more common in participants who had come off the drugs. No specific effects were significantly more common in the participants who had stayed on the drugs.
6) Measuring severity
Here Hayes and Jauhar either misrepresent or simply misunderstand the reasons the RCTs and naturalistic studies did not inform our overall severity estimates. Had they read these studies carefully they would have realised that these studies did not provide any data on the severity of withdrawal effects. The one RCT that did provide severity data (Sir et al., 2005) was, as stipulated, excluded for two reasons: it only covered eight weeks treatment (which would lower severity rates), and because it was a clinician-rated rather than self-report. Hayes and Jauhar choose not to acknowledge that our review also stated that even if we had included this outlier the weighted average of people who described their withdrawal effects as severe would have reduced only slightly, from 45.7% to 43.5%. (i)
Furthermore Hayes and Jauhar misrepresented our 45.7% weighted average by falsely stating that we concluded that ‘The severity of these symptoms is severe in over half of cases’.
7) Mixing study designs (opposition to including surveys)
Hayes and Jauhar argue that it is questionable to combine data from randomised controlled trials and naturalistic studies with survey data. As RCTs and naturalistic studies are regularly covered in systematic reviews (Guyatt et al. 2008; Egger et al. 2001), they obviously object to our inclusion of surveys. In making this objection, however, Hayes and Jauhar are simply confusing a methodological preference with a methodological law. There is no law prohibiting the inclusion of experiential survey data in a systematic review. In fact, given that the design of the majority of RTCs lead to the incidence, severity and duration of withdrawal being minimised (completely failing to capture real world experience of patients), it would have been unethical to omit such experiential data. Psychiatry has too often been guilty of devaluing the importance of experiential knowledge in its evaluation of interventions, which has in turn undermined our capacity to positively intervene.
Even so, given that surveys are open to specific forms of bias, how different would the estimates have been had we omitted surveys from our analysis? The answer is found in noting that the three types of studies, when grouped, did not differ greatly in terms of withdrawal incidence. The weighted averages turn out as follows:
- The three surveys – 57.1% (1790/3137),
- The four naturalistic studies – 50.4% (115/228)
- The seven RCTs – 51.4% (353/687)
As getting similar findings from different methodologies is typically seen to strengthen confidence in an overall, combined estimate, it is safe to conclude that at least half of people suffer withdrawal symptoms when trying to come off antidepressants.
Hayes and Jauhar report 3 very minor errors in presentation, which do not impact our estimates. Where we are concerned, however, is that they purport to identify two further ‘errors’, which are clearly not errors at all. For example, their assertion that the ‘total number experiencing withdrawal in the study by Sir and colleagues is 83 rather than 110’ is wrong. They reached this figure by unjustifiably removing all withdrawal symptoms rated ‘minimal’, while not informing readers they did this. This unwarranted decision not only minimised the rate of incidence but also created the false impression that we were in error.
Their second error concerns their suggestion that we misrepresented the incidence rate of another study (Montgomery et al. 2005) by presenting the incidence of withdrawal following escitalopram treatment as ‘27%, when it is 16%.’ Here Hayes and Jauhar again mislead the reader. The 27% rate we reported was at one week and the 16% they reported was at two weeks. Given we were calculating for incidence, it was absolutely correct for us to use the 27% figure in our calculations. These are unfortunate errors for Hayes and Jauhar to make, which, if left uncorrected, would wrongly undermine confidence in our study for some readers.
Combining data from different types of antidepressants
Hayes and Jauhar point out that because medications with longer half-lives will be associated with ‘less’ withdrawal effects “it is puzzling that the results are presented for all antidepressants combined”. Firstly, in the paper we explicitly acknowledge that “differing half-lives affect timing of withdrawal onset”, so they are telling us nothing we don’t already declare. Furthermore, ‘combining results’, or, more accurately, advancing global estimates was both necessary and appropriate given the central aim of our review. To reiterate, our aim was to assess whether NICE guidelines (2009) on antidepressant withdrawal were evidence based, not to guide clinicians in what drugs to prescribe nor to illuminate the particularities of different pharmacokinetic properties. Here Hayes and Jauhar commit the all-too-common fallacy of criticising a study for not doing what it never set out to do.
For the reasons stated we believe Hayes and Jahaur’s commentary to be inaccurate and misleading overall. In some cases the critiques they offer are based on obvious misrepresentations of study findings.
We fully accept that our overall estimates of 56% incidence, with 46% of those being severe, are only estimates. They may move by a few percentage points in either direction. However, even if the actual incidence is towards the lower end of the 50% to 57% range, when grouping study types, this will still constitute over half of all antidepressant users. It is crucial that amid the complexities of academic disagreement we do not lose sight of the scale of the problem.
In the light of this, it is also interesting to note the absence of any acknowledgment that we are discussing a public health issue involving millions of people worldwide. Hayes and Jahaur also fail to comment on the primary finding of the review, namely that national guidelines in the USA and the UK significantly misjudge the true extent of the problem.
In the sprit of seeking some common ground between us, we can agree with one concluding statement Hayes and Jahaur make:
‘It reflects negatively on the whole of the field of psychiatry that there is not better, clearer evidence from high quality studies on the incidence, severity and duration of any symptoms related to antidepressant cessation.’
Given that 16% of our adult population was prescribed an antidepressant last year alone, this professional oversight, and its significance, is hard to excuse.
While better research is indeed desirable (with respect to a diversity of issues pertaining to withdrawal), the millions experiencing withdrawal effects cannot wait for psychiatry to determine whether they represent 56%, 51% or 61% of those withdrawing, or what will be the best methodologies to assess that. They need accurate information and proper support now. And the millions more who will consider starting antidepressants in the coming years are entitled, unlike those who have gone before, to receive accurate information about all the adverse effects including the difficulty they are very likely to encounter when they try to stop; difficulties that in far too many cases will be protracted and severe. A crucial step forward will be for government bodies and professional organisations to update their guidelines so as to render them evidence-based.
Dr James Davies
Professor John Read
Addictive Behaviors (2018) Guide for Authors (Declaration of Interest). Website: https://www.elsevier.com/journals/addictive-behaviors/0306-4603/guide-for-authorsAccessed Oct 2018.
Chung, V.C.H., et al. (2018) Methodological quality of systematic reviews on treatments for depression: a cross-sectional study. Epidemiology and Psychiatric Sciences, 27 (26): 619-627
Egger M. et al. (2001) Systematic reviews in health care: meta-analysis in context. 2nd ed. London (UK): BMJ Publishing Group.
Guyatt G. et al. (2008) Users’ guides to the medical literature. 2nd ed. New York (NY): McGraw Hill Medical.
Hayes, J. & Jauhar, S. (2018) Antidepressant withdrawal: reviewing the paper behind the headlines. Mental Elf. Website: https://www.nationalelfservice.net/treatment/antidepressants/antidepressant-withdrawal-reviewing-the-paper-behind-the-headlines/. Accessed Oct 2018.
Montgomery, S. A. et al. (2005) ‘A 24-week randomized, double-blind, placebo-controlled study of escitalopram for the prevention of generalized social anxiety disorder’, The Journal of Clinical Psychiatry, 66(10): 1270–1278.
The National Institute for Health and Care Excellence (NICE) (2009) NICE Depression in adults: recognition and management. Website https://www.nice.org.uk/guidance/cg90/resources/depression-in-adults-recognition-and-management-pdf-975742638037, Accessed Jul 2018.
Price, J. et al. (1996) A comparison of the post-marketing safety of four selective serotonin re-uptake inhibitors including the investigation of symptoms occurring on withdrawal, British Journal of Clinical Pharmacology, 42: 757–763.
Rosenbaum, J. F. et al. (1998) Selective serotonin reuptake inhibitor discontinuation syndrome: a randomized clinical trial, Biological Psychiatry, 44(2): 77–87.
Zajecka, J. et al. (1998) Safety of Abrupt Discontinuation of Fluoxetine: A Randomized, Placebo-Controlled Study, Journal of Clinical Psychopharmacology, 18(3): 193.
(i) Hayes and Jauhar may have confused the severity data in Rosenbaum et al. (1998) for data on withdrawal severity. To be clear, such severity data pertained to depression severity pre- and post-discontinuation not withdrawal severity. Secondly, Price et al. (1996) indicate that 79% of withdrawal reactions were rated either ‘moderately severe or severe’. As Price et al. offer no scales or definitions by which to assess precisely what ‘moderately severe or severe’ means, we had no way of confidently placing their data (e.g. was ‘severe’ divided into three categories: ‘mildly severe’, ‘moderately severe’ and ‘severe’? If their data were included in our systematic review on the basis of such assumed divisions, then our overall severity estimates would have been higher).