Neuroscience

CBT: She's Lost Control(s) Again

And of a voice that told her when and where to act,
She said I've lost
control again

Joy Division (Shes Lost Control Again)

Cognitive therapy for people with a schizophrenia spectrum diagnosis not taking antipsychotic medication: an exploratory trial

Morrison, Hutton, Spencer, Barratt, Brabban, Callcott, Christodoulides, Dudley, French, Lunley, Tai & Turkington

This study, published recently in Psychological Medicine marks a departure from previous CBT for psychosis studies as it involves administering CBT to individuals who are unmedicated - and alongside the media attention (All in the Mind), it is important to look more closely at the study itself.

Morrison and 11 colleagues examined a small sample of 20 participants with schizophrenia spectrum disorders (actually 18 and one also didn't complete the therapy - so 17). All were outpatients - who had not been taking antipsychotic medication for at least 6 months. Morrison et al measured the impact of CBT on symptomatic outcome measures: Positive and Negative Syndromes Scale (PANSS), which was administered at baseline, 9 months (end of treatment) and 15 months (follow-up). Secondary outcomes were dimensions of hallucinations and delusions, self-rated recovery and social functioning. Rather than dismiss missing data, the authors chose to impute the missing data in the following analyses:

Their main findings were stated as:

"significant beneficial effects on all primary and secondary outcomes at end of treatment and follow-up, with the exception of self-rated recovery at end of treatment. Cohen’s d effect sizes were moderate to large [for PANSS total, d=0.85, 95% confidence interval (CI) 0.32–1.35 at end of treatment; d=1.26, 95% CI 0.66–1.84 at follow-up]."

The authors conclude that the

"study provides preliminary evidence that CT is an acceptable and effective treatment for people with psychosis who choose not to take antipsychotic medication. An adequately powered randomized controlled trial is warranted."

Now, I do think this is important because some individuals may see this study and attendant media as a basis to decide not to take antipsychotic medication. I have no qualms about personal choice when based on evidence - so let us examine the evidence.

Joy Division: Shes Lost Control

So, what does the study show? Well, the answer comes from the design of the study. It is a pretest - posttest design i.e. there is no control group. So, a within-group analysis with no control group. Without a control group of any description, we cannot know if any change is a generalised consequence of the added interaction (rather than anything about CBT itself). And without even a Treatment as Usual (TAU) control group, we cannot exclude the possibility that any change would have occurred regardless of therapy e.g. regression to the mean.

Second, the trial is an Open Trial i.e. the outcome measures are not made blind. All participants are evaluated by members of the team who were involved in the administering the trial. The lack of blinding is the biggest drawback to the evaluation of therapy and is well documented by all meta-analyses in this area (Lynch, Laws & McKenna 2010; Wykes et al 2008).

So, the study suffers from a host of threats to validity: a) regression to the mean i.e. extreme scores at start simply move toward the (lower) mean on second testing (with no control we cant estimate or eliminate this); b) Hawthorne effect i.e. any change is due to the special circumstances patients find themselves in (with no control, we cant eliminate this); c) lack of blinding i.e. those measuring outcome were involved in the study and aware that all patients had received CBT - the authors don't provide detail on this, but presumably some or all of the 8 different therapists administering the CBT to the participants and/or those who designed the study

Losing Control adds up!

To summarise - no control group and no blinding of outcome measures.

Can we see what happens in such designs with medicated patients? Well, below is a table from a meta-analysis by the CBT guru Aaron Beck looking at effect size for pretes-posttest analyses in medicated patients (Rector & Beck 2002; so 'good' the journal republished it in 2012). Now Morrison et al found d=.85 for overall symptoms -Rector and Beck report a much larger mean effect size of d=1.31 for pre-post comparison (CBT-RC). So, the effect with unmedicated is substantially less - consider the size in some studies here - Pinto is almost three times that reported by Morrison et al!

This doesn't alter the fact that Morrison et al do report a substantial effect in unmedicated patients of .8+. However, the second thing to note about the table above is the comparison of CBT with a so-called active control (ST-RC) i.e. a condition that controls for the generalised impact of just interacting with another, receiving attention etc - here the controls are supportive therapy, befriending and so on. This shows that an effect size of .63 emerges in pretest postest designs for something as simple as befriending - not that much smaller than Morrison et al claim to attribute to CBT.

The final and key point though concerns the lack of blind evaluation. As Morrison et al note in their discussion:

"CBT for psychosis trials that attempt masking were reported to be associated with a reduction of effect sizes of nearly 60% (Wykes et al 2008)"

Actually, what Wykes et al say is

"There is a tendency for the unmasked studies to be overoptimistic about the effects of CBTp, with effect sizes of 50%–100% higher than those found in masked studies."

60% may be an average, but the reduction could range from 50-100%!

Crucially, the patients may not feel better. At the end of the study, the patients rated themselves as experiencing no recovery (pre and post); although they did report a minor improvement at the follow-up. In other words, the non-blind researchers perceived a much greater change than the patients themselves - approximately double the effect size!

To conclude: the use of a pretest-posttest design with no blinding means the authors are unable to draw conclusions about the efficacy of CBT in the study. The effect size is much smaller than in pretest-postest comparisons with medicated patients; of the .85 reported, how much is attributable to generic effects of interaction (and nothing to do with CBT) - well it could be .63; and if we assume a minimum of 60% of this may reflect lack of blinding - then what are we left with? An effect size of possibly less than .1 - in other words - (next to) nothing! And patients are not even convinced!

So, their conclusions that the "study provides preliminary evidence that CT is an acceptable and effective treatment for people with psychosis who choose not to take antipsychotic medication" seems unwarranted. Some might say, well its called 'exploratory' - but that is no excuse for a design that leaves the data uninterpretable and may even lead to some people changing their behaviour (withdrawing from their medication). I also understand that an RCT with blind assessment is being conducted - indeed, much of the press around this study has been about the uncompleted study - a bit cart before the horse! But at the moment - no data published exist to show that CBT reduces psychotic symptoms in unmedicated individuals

References

Lynch, D., Laws, K. R., & McKenna, P. J. (2010). Cognitive behavioural therapy for major psychiatric disorder: does it really work? A meta-analytical review of well-controlled trials. Psychological medicine, 40(01), 9-24.

Wykes, T., Steel, C., Everitt, B., & Tarrier, N. (2008). Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophrenia Bulletin, 34(3), 523-537.

- Meta-matic: Meta-analyses Of Cbt For Psychosis
Meta analyses are not a 'ready-to-eat' dish that necessarily satisfy our desire for 'knowledge' - they require as much inspection as any primary data paper and indeed, afford closer inspection...as we have access to all of the data....

- My Bloody Valentine: Cbt For Unmedicated Psychosis
When I critiqued Morrison et als exploratory CBT trial with people who stop taking anti-psychotic medication, I promised to write a post on the final study Well it appeared in the Lancet today and a free copy is here. I am not going to...

- Who Watches The Watchmen? Bias In Studying Bias
The coins are often very old by the time they reach the jeweller With his hands and ashes he will try the best he can He knows that he can only shine them Cannot repair the scratches The Jeweller (by Pearls Before Swine) Publication...

- 'representative' - Its Too "too Too" To Put A Finger On
Prove it... just the facts... the confidentialThis case, this case, this case that I...I've been workin' on so long...Chirpchirp...the birds they're giving you the words The world is just a feeling you undertook. Now the rose, it slowsyou...

- Cbt: You Spin Me Round
Although we failed to show a statistically significant effect of the intervention we cannot rule out a beneficial effect of the cognitive therapy on transition rate (although it could be argued that the sample size required to show such an effect,...

Neuroscience