Guest blog: NICE depression guideline – cautious optimism?

NICE depression guideline: cautious optimism?

Susan McPherson, University of Essex

This commentary is a revised and updated version of a Rapid Response letter to the BMJ in September 2019 posted prior to NICE’s decision letter of December 2019: https://www.bmj.com/content/365/bmj.l2356/rr-0

In a joint stakeholder position statement concerning the NICE 2018 draft depression guideline, six methodological issues were raised. These were that patient experience research had been omitted; as had long-term follow-up data; chronic forms of depression had been inappropriately separated into chronic, treatment-resistant and complex depression; that depression severity in trials and subsequent interpretations of ‘recovery’ had been misrepresented; that an experimental and ‘confounded’ network meta-analysis (NMA) had been used as a primary rather than supplementary basis for recommendations; that quality of life outcomes had been left out. (See www.nsun.org.uk/news/stakeholder-position-statement-on-the-nice-guideline-for-depression-in-adults)

Following ongoing discussions within the NICE depression Guideline Committee and engagement with stakeholder representatives, NICE responded with their decision sent to all registered stakeholders in December 2019. Below I summarise each issue and the latest response from NICE in their letter to stakeholders. 

Inclusion of patient experience research

The additional work on patient choice, which was begun in February 2019 will include a systematic review of primary studies of service user experience. This work focuses specifically on qualitative studies informing questions around choice of treatment, rather than more general studies of people’s experiences of depression (not linked to choice of treatment), which were not felt to be sufficiently relevant to the question. The findings of this work will be considered alongside quantitative results when recommendations are being written and quality assured and presented as such in the final guideline. 

NICE, December 2019

The wording of this decision is slightly ambiguous and may not directly address stakeholder concerns. The initial position taken by NICE on the question of patient experience appeared to conflate three different issues: patient involvement in the depression guideline development process, patient choice and patient experience research (see McPherson & Beresford, 2019). By choosing not to conduct a systematic review of patient experience research, a significant body of literature was being excluded. Stakeholders therefore asked NICE to conduct a review of studies of patient experiences of treatments for depression. It remains ambiguous what NICE mean by “studies informing questions around choice of treatment”.

It will be important that the review to be undertaken by NICE does focus on studies which examine experiences of treatment rather than “choice of treatment” since a better understanding of the former may then be used to support the latter. In a completed metasynthesis currently under peer-review on patient experiences of psychological treatments for depression, myself and colleagues identified 38 eligible studies. These studies should be reviewed by NICE in order to inform patient choice of psychological treatments. Alongside this, a further review should be carried out examining patient experiences of medical treatments. The guideline should also take into account experiences of carers. In a recent metasynthesis examining the experiences of people caring for loved ones with depression (Priestley & McPherson, 2016), we included 15 studies which together provide a useful perspective on carer involvement in depression treatment. It is not clear from the wording in their letter to stakeholders that NICE intend to look at any of this evidence.

Inclusion of long-term follow-up data

NICE will now include long-term follow-up data in all its treatment reviews. The committee previously considered that long-term follow up data were not available across all comparisons of interests in the guideline and consequently long-term effectiveness would be subject to large potential biases. However, we accept that long-term effectiveness is an important outcome and we will now look for it in all treatment reviews. The committee will consider all long-term follow-up data found in making its recommendations and the relative importance attributed to it will be a judgement made by the committee based on the availability and quality of the long-term data.

NICE, December 2019

This decision appears to be a good one. However, it is important to reiterate that while ‘relative importance’ might be judged on quality, it should not be judged based on quantity. As noted by stakeholders repeatedly, if we acknowledge that long-term data is a better form of evidence than short-term data, then large amounts of short term (i.e. poor quality) data should not outweigh small amounts of long-term evidence. 

An analysis of long-term outcomes for the trials of treatments for long-term depression included in the guideline review illustrated how and why NICE should do so too (McPherson & Hengartner, 2019). Although long-term evidence from RCTs is limited in volume, the findings from this set of data (that psychological therapies are superior to antidepressants in the long-term) is supported by a range of systematic reviews of other forms of evidence including naturalistic cohort studies and discontinuation trials. NICE have produced recommendations for depression treatment for 15 years now, ignoring this data and potentially contributing to the crisis of prescribed drug dependency now recognised by Public Health England (https://phe-newsroom.prgloo.com/news/dependence-on-prescription-medicines-linked-to-deprivation). It is hoped the decision to ‘look for’ long-term follow-up data will therefore also lead to prioritisation of this data and also an openness to alternative research designs which give a more rounded picture of the long-term effects of treatment although the latter is at best optimistic.

Subgrouping chronic forms of depression and severity classifications

For the further-line treatment review, NICE will include studies of adults with depression showing an inadequate response to at least one previous intervention for the current episode and this will include the further-line treatment of psychotic depression, depression with coexisting personality disorder and chronic depression.

NICE, December 2019 

If this is to be understood as a decision to combine the reviews which were previously separated into treatment resistance, chronic and complex depression, then this also appears to be a good decision. NICE previously made fundamental philosophical errors in separating these supposed subgroups of depression, which I discussed in a recent article ‘A NICE Game of Minecraft’ (McPherson, 2019). This article argues that in making these distinctions without having full knowledge of individuals’ experiences, the suggestion that these sub-classifications of persistent forms of depression are somehow ‘clinically useful’ dissolves and becomes more like a game of Minecraft in which virtual cuboids are built up into logically impossible structures. This epistemological problem also applies to NICE’s separation of trials into categories of ‘mild’ and ‘severe’ depression. Concerning severity classifications, NICE have responded:

The categorisations of depression severity have been updated and are now based on published work by Wahl et al (2014) and others.

NICE, December 2019 

Whilst this approach to standardisation of severity scores across different psychometric measures (e.g. BDI, HRSD, MADRS etc) may be peer-reviewed as opposed to the earlier bespoke approach designed by NICE, it does not address the epistemological problem I identify in A NICE Game of Minecraft. Specifically, there is no good justification for artificially separating trials into categories of mild and severe based on a group mean average at baseline because it distorts experiences of individuals within those trials and doing so will continue to lead to misleading recommendations. Unfortunately, while NICE remains primarily structured around a diagnostic and categorical approach to ‘mental disorders’, it forces itself into approaches to evidence synthesis which are not grounded in patient experience and are philosophically unsound.

Full versus partial recovery

The guideline will look at continuous changes in scores on depression scales in every treatment question, which will capture changes for people who have both fully and partially recovered. This was agreed by the committee to be a better way to capture this data than the use of a dichotomous outcome for partial recovery.

NICE, December 2019

This decision appears to rectify the previous reliance on a ‘recovery’ metric which meant that trials where the majority of participants were clinically complex and severely depressed had a higher benchmark to reach in order to be considered to be demonstrating treatment efficacy. 

Quality of life outcomes

NICE will now include quality of life and functional status as outcomes in the treatment reviews. As with long-term data, the relative importance that will be attributed to this data cannot be stated upfront, since this will be a judgement made by the committee based on the availability and quality of data available for these outcomes.

NICE, December 2019

NICE previously claimed that it was not possible to look at quality of life outcomes because use of these measures is rare (NICE, 2018).I have examined the appendices of the draft guideline which detail the 124 trials reviewed in the treatment resistant and chronic depression categories and I have looked at the published papers of these trials. Fifty-eight (nearly half) used a quality of life or functioning measure. It is the case that the specific measures used vary a great deal; I counted 16 different such measures among these trials. However, there were also 13 different symptom outcome measures and NICE took all of these into account.  

The decision to include quality of life outcomes in reviews appears to be a good decision. It is important to examine quality of life outcomes not least because NICE have consistently stated in their own scoping documents and previous guidelines that quality of life is an important outcome; quality of life is also a central pillar of government policy (e.g. No Health Without Mental Health). Moreover, service users often note that outcomes such as better relationships and better functioning matter more than symptom improvement. Although some of the measures typically used may not perfectly reflect service user priorities (Connell, O’Cathain & Brazier, 2014), it indicates a move in the right direction to demonstrate commitment to examining patient preferred outcomes. It is also important to note that quality of life outcomes sometimes give different results to symptom outcomes. In a re-analysis of the first NICE depression guideline (McPherson, Evans & Richardson, 2004) we found that analysing quality of life outcome measures does produce different findings in reviews.

Network Meta-Analysis (NMA)

We do not accept that the use of NMA was inappropriate and we will continue to use NMAs both to assess clinical effectiveness and to inform the economic model in accordance with the NICE Guidelines Manual. However, pairwise data will be presented separately in the new version of the guideline to enable an easier comparison between direct and NMA results. There will also be a peer review of all NMAs by a NICE Technical Support Unit contractor and NICE will publish the code for its NMAs. NICE recognises that no statistical technique will ever lead to an indisputably ‘correct’ answer, since they all involve assumptions and extrapolations of the available data. Both the committee and quality assurance team will consider any limitations of the analysis and the confidence we have in it when making recommendations. The data from the NMA will also be considered alongside the other sources of data feeding into this question, including the pairwise data, economic model results and newly reviewed qualitative evidence. 

NICE, December 2019

Stakeholder responses to this last decision may be mixed. The NMA reflects an increasingly technocratic approach to guideline development at NICE which may be useful in some areas of medicine. In mental health, there are increasing concerns about the use of technical approaches which rely on certainty in classification and diagnosis, as in the Minecraft problem described earlier. As discussed, this approach to classification of ‘mental disorders’ is philosophically flawed. The acknowledgement that NMA reviews should be peer-reviewed may be welcomed but there may also be some questions raised about the degree of independence of the ‘contractor’. It is nevertheless a potentially positive sign that the results from the qualitative evidence will be fed into interpretations of these reviews.

Conclusions

Some may take the view that ‘interfering’ in NICE guideline development is antithetical to the nature of our UK guideline body. Indeed NICE was established to be free from political interference and lobbying from pharmaceutical companies.  In a commentary in the British Journal of Psychiatry, Prof Sunkin QC (Hon) and I argued that the stakeholder consultation process is a critical element of quality assurance in NICE guidelines (McPherson & Sunkin, 2019). We suggest that stakeholder consultation can only serve as a robust quality assurance process if legitimate methodological concerns raised by stakeholders during this process are taken into account and not brushed off as minor inconveniences. Moreover, as a public body, NICE is bound by English law to consult in good faith and not to have made decisions prior to the consultation. It is therefore appropriate that NICE appear to have taken on board a set of methodological concerns expressed by a wide section of the patient and professional stakeholder community.  The next revision of the guideline will inevitably take some time given the amount of new work now to be done but there appears to be reason for some cautious optimism along with some remaining concerns about the next draft which will be put out for stakeholder consultation in due course.

Connell, J., O’Cathain, A., Brazier, J. (2014). Measuring quality of life in mental health: Are we asking the right questions? Social Science & Medicine, 120, 12-20. 10.1016/j.socscimed.2014.08.026

McPherson, S. (2019). A NICE game of Minecraft: philosophical flaws underpinning UK depression guideline nosology. Medical Humanities. doi:10.1136/medhum-2019-011658

McPherson, S., & Beresford, P. (2019). Semantics of patient choice: how the UK national guideline for depression silences patients.Disability and Society, 1-7. doi:10.1080/09687599.2019.1589757

McPherson, S., Evans, C., & Richardson, P. (2009). The NICE Depression Guidelines and the recovery model: Is there an evidence base for IAPT?. Journal of Mental Health, 18(5), 405-414. Retrieved from http://dx.doi.org/10.3109/09638230902968258

McPherson, S. & Hengartner, M.P. (2019). Long-term outcomes of trials in the NICE depression guideline. BJPsych Open, 5(5). https://doi.org/10.1192/bjo.2019.65

McPherson, S., & Sunkin, M. (2019). The Dobson–Rawlins pact and the National Institute for Health and Care Excellence: impact of political independence on scientific and legal accountability. The British Journal of Psychiatry. doi:10.1192/bjp.2019.121

NICE. (2018). Consultation comments and responses – second consultation period. https://www.nice.org.uk/guidance/gid-cgwave0725/documents/consultation-comments-and-responses-2

Priestley, J., & McPherson, S. (2016). Experiences of adults providing care to a partner or relative with depression: A meta-ethnographic synthesis. Journal of Affective Disorders, 192, 41-49. doi:10.1016/j.jad.2015.12.011