Neuroscience
Its not often that we hear such phrases, but thus opens the 'highlights' section of the latest edition of the British Journal of Psychiatry, referring to our new meta-analysis examining Cognitive Behaviour Therapy (CBT) for the symptoms of Schizophrenia. This is the most comprehensive analysis ever undertaken, covering 50 Randomised Controlled Trials (RCTs) of this 'talk therapy' published over the past 20 years. The paper received press coverage and is, of course available for subscribers at the British Journal of Psychiatry, but I would like to give an overview for the interested lay reader, service-users or anyone who cannot access the journal.
Looking at all trials regardless of quality, the paper reveals a small effect in terms of CBT reducing the symptoms of schizophrenia: effect sizes being 0.25 for positive and 0.13 for negative symptoms. To put these effect sizes into everyday language - the vast majority of patients in the CBT and control groups fail to differ at the end of the intervention: 82% and 90% of the CBT and control groups overlapped on positive and negative symptoms respectively.
But this is not the end of the story...
Study Quality
Studies vary in their quality (eg. studies with fewer methodological controls are more prone to bias). In this context, we draw attention to 'blinding' or 'masking' i.e. whether the person assessing symptoms at outcome knows if patients did or didn't receive CBT. We found that effect sizes were up to 7 times larger in nonblind than blind studies. And if you assess effect size in blind studies, the small effects totally disappear (see Table 1). In other words, when researchers know if the patients had received CBT, it massively inflates the positivity of the researchers ratings of patient benefit at outcome! In plain language, at the end of trials 94% and 97% of the CBT and control groups overlap on positive and negative symptoms respectively
Table 1. Comparison of effect sizes for blind (high risk) vs nonblind (low risk) studies
Whats happening in individual studies: Forest Plots
Forest plots show the effect size in each trial (the filled rectangle). The size of the rectangle represents the size of the sample tested in a study. The horizontal lines represent the 95% confidence intervals for each effect - these essentially tell us about the reliability of the estimated effect; shorter lines indicate that the estimate is more reliable; longer lines, less reliable. You will notice that larger CI lines emerge in studies with smaller samples and vice versa. The key thing to ask is ... Do the 95% CIs in any study cross zero? If they do, then the trial revealed a nonsignificant effect of CBT on symptoms.
Looking at Figure 1, we can see 25 of 33 studies document a non-significant impact of CBT on positive symptoms. Nonetheless, the overall effect across all 33 studies is significant i.e. ES= -.25 (95%CI -.37 to -.13). This reveals several things - that even when 75% of studies are nonsignificant, meta-analysis can produce an overall significant effect.
Figure 1. Forest plot of 33 studies examining the impact of CBT on positive symptoms
The picture for hallucinations is bleaker...with only 4 significant studies ever published
Figure 2 CBT for Hallucinations
And if it could be worse...it is for negative symptoms ...with no significant study since 2003
A few key take-home observations from the forest plots:
If anyone is interested in exploring the data and forest plots further, they may do so via a downloadable and interactive database on our website: http://www.cbtinschizophrenia.com/
Symptoms or Distress?
One response to me about our paper, from some UK clinical psychologists, has been to say that ...they use CBT not to reduce the symptoms of psychosis, but to reduce the 'distress'. In the context of the clinical guidance provided to UK clinicians by the National Institute of Clinical Excellence (NICE), this response raises interesting questions about the relationship between science and practice.
NICE do state state that CBT be used to reduce distress (see above); however, this is intriguing on multiple levels. First, NICE base their recommendations on the meta-analysis conducted for them by the National Collaborating Centre for Mental Health (NCCMH), in which all of the data examined relates to RCTs aimed at symptom reduction....and not distress
This is perhaps exemplified by the following paragraph from the NICE guide
The NICE guide states that distress is the target, but that outcomes in trials is not distress. Second, some UK clinicians are clearly taking NICE guidance at face value saying they use CBT to 'reduce distress' - this is effectively unevidenced or off-label use of CBT. Third, and crucially, the evidence does not suggest that CBT reduces distress. For example, they refer to Trower et al 2004 as an example - actually, the study shows no benefit of CBT for distress after one year.
Additionally, I would question the reference to CBT improving 'function' - the meta analysis in 2008 by Wykes et al showed that CBT has no significant impact on functioning in studies meeting their own minimally acceptable study quality. Fifth, they reference Garety et al regarding relapse prevention - our re-analysis of that study actually shows an increase in relapse for the CBT group. And finally, by the time of this NICE document in 2009, NICE had removed insight in psychosis as a target for CBT (following their 2002 recommendations), even though they had no evidence for it in the first place
These findings create a challenge for the guidance provided by Government organisations (in the UK, this is NICE) who advocate that "CBT be offered to all people with for schizophrenia".
CBT does not reduce positive symptoms, negative symptoms, or hallucinations; it does not prevent relapse, it does not reduce distress, it does not improve functioning, and it does not improve insight. In the paper we therefore call on NICE to reexamine their recommendation- especially as new guidance is due in 2014...in a matter of weeks!
- Meta-matic: Meta-analyses Of Cbt For Psychosis
Meta analyses are not a 'ready-to-eat' dish that necessarily satisfy our desire for 'knowledge' - they require as much inspection as any primary data paper and indeed, afford closer inspection...as we have access to all of the data....
- Satellites Of Love: Uninterpretable Results
In llama land there's a one-man band And he'll toot his flute for you Come on fly with me, let's take off in the blueFrank Sinatra (Come Fly with Me) Cognitive Behavioral Therapy for negative symptoms in psychotic disorders: a pilot...
- Its Just A Story: Transition To Psychosis & Cbt
I've been living through changes...And I could swing for you I can see the veins in my hands...Are showing throughBut if you disguise what...These things are doing to me If you criticize them...I'll know that you can see......
- Who Watches The Watchmen? Bias In Studying Bias
The coins are often very old by the time they reach the jeweller With his hands and ashes he will try the best he can He knows that he can only shine them Cannot repair the scratches The Jeweller (by Pearls Before Swine) Publication...
- In A Manner Of Speaking
In a Manner of speaking I just want to say That I could never forget the way You told me everything By saying nothingIn a Manner of Speaking (by Tuxedo Moon) I was recently asked to write an article on some of the problems associated with psychological...
Neuroscience
Blinded by Science
"The New Year starts with a test of an established tenet of treatment in schizophrenia."
British Journal of Psychiatry 'Highlights' January 2014
Its not often that we hear such phrases, but thus opens the 'highlights' section of the latest edition of the British Journal of Psychiatry, referring to our new meta-analysis examining Cognitive Behaviour Therapy (CBT) for the symptoms of Schizophrenia. This is the most comprehensive analysis ever undertaken, covering 50 Randomised Controlled Trials (RCTs) of this 'talk therapy' published over the past 20 years. The paper received press coverage and is, of course available for subscribers at the British Journal of Psychiatry, but I would like to give an overview for the interested lay reader, service-users or anyone who cannot access the journal.
Forbidden Colours (Sakamoto & Sylvian)
I’ll go walking in circles
While doubting the very ground beneath me
Trying to show unquestioning faith in everything
I’ll go walking in circles
While doubting the very ground beneath me
Trying to show unquestioning faith in everything
Looking at all trials regardless of quality, the paper reveals a small effect in terms of CBT reducing the symptoms of schizophrenia: effect sizes being 0.25 for positive and 0.13 for negative symptoms. To put these effect sizes into everyday language - the vast majority of patients in the CBT and control groups fail to differ at the end of the intervention: 82% and 90% of the CBT and control groups overlapped on positive and negative symptoms respectively.
But this is not the end of the story...
Study Quality
Studies vary in their quality (eg. studies with fewer methodological controls are more prone to bias). In this context, we draw attention to 'blinding' or 'masking' i.e. whether the person assessing symptoms at outcome knows if patients did or didn't receive CBT. We found that effect sizes were up to 7 times larger in nonblind than blind studies. And if you assess effect size in blind studies, the small effects totally disappear (see Table 1). In other words, when researchers know if the patients had received CBT, it massively inflates the positivity of the researchers ratings of patient benefit at outcome! In plain language, at the end of trials 94% and 97% of the CBT and control groups overlap on positive and negative symptoms respectively
Soft by Lemon Jelly (with added "If you leave me now" by Chicago)
Forest plots show the effect size in each trial (the filled rectangle). The size of the rectangle represents the size of the sample tested in a study. The horizontal lines represent the 95% confidence intervals for each effect - these essentially tell us about the reliability of the estimated effect; shorter lines indicate that the estimate is more reliable; longer lines, less reliable. You will notice that larger CI lines emerge in studies with smaller samples and vice versa. The key thing to ask is ... Do the 95% CIs in any study cross zero? If they do, then the trial revealed a nonsignificant effect of CBT on symptoms.
Looking at Figure 1, we can see 25 of 33 studies document a non-significant impact of CBT on positive symptoms. Nonetheless, the overall effect across all 33 studies is significant i.e. ES= -.25 (95%CI -.37 to -.13). This reveals several things - that even when 75% of studies are nonsignificant, meta-analysis can produce an overall significant effect.
The picture for hallucinations is bleaker...with only 4 significant studies ever published
And if it could be worse...it is for negative symptoms ...with no significant study since 2003
A few key take-home observations from the forest plots:
Positive symptoms - 25 of 33 (76%) studies are nonsignficant
Negative symptoms - 30 of 34 (88%) studies are nonsignificant
Hallucinations - 11 of 15 (73%) studies are nonsignificant
If anyone is interested in exploring the data and forest plots further, they may do so via a downloadable and interactive database on our website: http://www.cbtinschizophrenia.com/
You Cut Her Hair by Tom McRae
Symptoms or Distress?
One response to me about our paper, from some UK clinical psychologists, has been to say that ...they use CBT not to reduce the symptoms of psychosis, but to reduce the 'distress'. In the context of the clinical guidance provided to UK clinicians by the National Institute of Clinical Excellence (NICE), this response raises interesting questions about the relationship between science and practice.
NICE do state state that CBT be used to reduce distress (see above); however, this is intriguing on multiple levels. First, NICE base their recommendations on the meta-analysis conducted for them by the National Collaborating Centre for Mental Health (NCCMH), in which all of the data examined relates to RCTs aimed at symptom reduction....and not distress
This is perhaps exemplified by the following paragraph from the NICE guide
The NICE guide states that distress is the target, but that outcomes in trials is not distress. Second, some UK clinicians are clearly taking NICE guidance at face value saying they use CBT to 'reduce distress' - this is effectively unevidenced or off-label use of CBT. Third, and crucially, the evidence does not suggest that CBT reduces distress. For example, they refer to Trower et al 2004 as an example - actually, the study shows no benefit of CBT for distress after one year.
Additionally, I would question the reference to CBT improving 'function' - the meta analysis in 2008 by Wykes et al showed that CBT has no significant impact on functioning in studies meeting their own minimally acceptable study quality. Fifth, they reference Garety et al regarding relapse prevention - our re-analysis of that study actually shows an increase in relapse for the CBT group. And finally, by the time of this NICE document in 2009, NICE had removed insight in psychosis as a target for CBT (following their 2002 recommendations), even though they had no evidence for it in the first place
Hærra by Ásgeir Trausti
These findings create a challenge for the guidance provided by Government organisations (in the UK, this is NICE) who advocate that "CBT be offered to all people with for schizophrenia".
CBT does not reduce positive symptoms, negative symptoms, or hallucinations; it does not prevent relapse, it does not reduce distress, it does not improve functioning, and it does not improve insight. In the paper we therefore call on NICE to reexamine their recommendation- especially as new guidance is due in 2014...in a matter of weeks!
- Meta-matic: Meta-analyses Of Cbt For Psychosis
Meta analyses are not a 'ready-to-eat' dish that necessarily satisfy our desire for 'knowledge' - they require as much inspection as any primary data paper and indeed, afford closer inspection...as we have access to all of the data....
- Satellites Of Love: Uninterpretable Results
In llama land there's a one-man band And he'll toot his flute for you Come on fly with me, let's take off in the blueFrank Sinatra (Come Fly with Me) Cognitive Behavioral Therapy for negative symptoms in psychotic disorders: a pilot...
- Its Just A Story: Transition To Psychosis & Cbt
I've been living through changes...And I could swing for you I can see the veins in my hands...Are showing throughBut if you disguise what...These things are doing to me If you criticize them...I'll know that you can see......
- Who Watches The Watchmen? Bias In Studying Bias
The coins are often very old by the time they reach the jeweller With his hands and ashes he will try the best he can He knows that he can only shine them Cannot repair the scratches The Jeweller (by Pearls Before Swine) Publication...
- In A Manner Of Speaking
In a Manner of speaking I just want to say That I could never forget the way You told me everything By saying nothingIn a Manner of Speaking (by Tuxedo Moon) I was recently asked to write an article on some of the problems associated with psychological...