Neuroscience
Many studies in clinical psychology and psychiatry are making the mistake of using healthy controls who are too healthy. That's according to a thought-provoking opinion piece by Sharon Schwartz and Ezra Susser - experts in the epidemiology of mental health.
Schwartz and Susser invite readers to consider a hypothetical study that samples participants from a wider group made up of people exposed to a virus prenatally and people not exposed to that virus. Imagine that a psychiatric registry is used to identify all the participants from this wider group who are diagnosed with schizophrenia, and they are compared with a slice of healthy participants recruited from the same source. The aim is to see what proportion of the participants with schizophrenia were exposed to the virus and what proportion of the healthy controls were exposed to the virus. If the history of exposure is higher among the schizophrenia participants, then this would suggest there may be an association between the virus and the later development of schizophrenia. In Schwartz and Susser's hypothetical scenario, there is no difference between patients and controls in rates of virus exposure and so the virus seems unassociated with schizophrenia. So far, so good - this is a classic case-controlled study.
The problem identified by Schwartz and Susser is that many such studies apply an exclusion criterion or criteria to the healthy controls that they don't also apply to the patient group. For example, they might rule out healthy controls with an alcohol problem, or depression, or even a physical disorder. The motivation for this is often the fear that these other disorders will obscure the potential link between the cause of interest and the condition of interest (virus exposure and schizophrenia in our ongoing example).
But to apply such exclusion criteria in a one-sided fashion (to the controls but not the patients), creates a serious confound. In our example, imagine that depressed "healthy" controls are excluded and imagine too that there is an underlying association between the virus exposure and depression. Excluding healthy controls with depression in this scenario would distort the results such that the virus appeared wrongly to be associated with schizophrenia (check out the full paper for the data behind this).
"With all the potential sources of bias in a biologic case-control study, why do we focus on the use of well controls?" the researchers asked. "We do so because the use of well controls is a common, and often recommended, method to select controls. Yet it is time-consuming and expensive, can cause considerable bias and does not improve study results."
If researchers include patient participants with other co-morbid diagnoses in their case-controlled studies, Schwartz and Susser went on to explain, then they must also include "healthy" controls who happen to have these other conditions. On the other hand, if researchers want to exclude other conditions, so as to clean up their investigation, then they must exclude both patient participants and controls with these other diagnoses.
_________________________________
Schwartz, S., and Susser, E. (2011). The use of well controls: an unhealthy practice in psychiatric research. Psychological Medicine, 41 (06), 1127-1131 DOI: 10.1017/S0033291710001595
This post was written by Christian Jarrett for the BPS Research Digest.
- Depression Linked With Impaired Spatial Ability
Depression has a detrimental effect on people's ability to find their way around, according to researchers. Husseini Manji and colleagues compared the performance of 30 depressed patients and 19 healthy controls on a virtual reality spatial navigation...
- Body Image - It's 'healthy' People Who Are Deluded
We’re all going to die and there’s nothing we can do about it. Depressing? Well, it’s been argued that depressed people are the sane ones because they see the world for how it really is. Now consider this – a study has found people with eating...
- Cold Comfort For Shy People
"From the Archives", first published in the Digest 15.09.03 How sociable you are affects your chances of catching a cold. That's according to Sheldon Cohen (Carnegie Mellon University, USA) and colleagues (Universities of Pittsburgh and Virginia,...
- Abnormal Social Cognition
To investigate the neuroscience behind socialising, Andreas Meyer-Lindenberg and colleagues have scanned the brains of people with the genetic disorder Williams-Beuren syndrome and compared them with scans of healthy controls. People with Williams syndrome...
- Neuropsychology Abstract Of The Day: Chronic Congestive Heart Failure And Neuropsychological Function
Vogels RL, Oosterman JM, van Harten B, Scheltens P, van der Flier WM, Schroeder-Tanka JM, & Weinstein HC. Profile of Cognitive Impairment in Chronic Heart Failure. Journal of the American Geriatrics Society. 2007 Aug 28; [Epub ahead of print]. Department...
Neuroscience
Beware the "super well" - why the controls in psychology research are often too healthy
Schwartz and Susser invite readers to consider a hypothetical study that samples participants from a wider group made up of people exposed to a virus prenatally and people not exposed to that virus. Imagine that a psychiatric registry is used to identify all the participants from this wider group who are diagnosed with schizophrenia, and they are compared with a slice of healthy participants recruited from the same source. The aim is to see what proportion of the participants with schizophrenia were exposed to the virus and what proportion of the healthy controls were exposed to the virus. If the history of exposure is higher among the schizophrenia participants, then this would suggest there may be an association between the virus and the later development of schizophrenia. In Schwartz and Susser's hypothetical scenario, there is no difference between patients and controls in rates of virus exposure and so the virus seems unassociated with schizophrenia. So far, so good - this is a classic case-controlled study.
The problem identified by Schwartz and Susser is that many such studies apply an exclusion criterion or criteria to the healthy controls that they don't also apply to the patient group. For example, they might rule out healthy controls with an alcohol problem, or depression, or even a physical disorder. The motivation for this is often the fear that these other disorders will obscure the potential link between the cause of interest and the condition of interest (virus exposure and schizophrenia in our ongoing example).
But to apply such exclusion criteria in a one-sided fashion (to the controls but not the patients), creates a serious confound. In our example, imagine that depressed "healthy" controls are excluded and imagine too that there is an underlying association between the virus exposure and depression. Excluding healthy controls with depression in this scenario would distort the results such that the virus appeared wrongly to be associated with schizophrenia (check out the full paper for the data behind this).
"With all the potential sources of bias in a biologic case-control study, why do we focus on the use of well controls?" the researchers asked. "We do so because the use of well controls is a common, and often recommended, method to select controls. Yet it is time-consuming and expensive, can cause considerable bias and does not improve study results."
If researchers include patient participants with other co-morbid diagnoses in their case-controlled studies, Schwartz and Susser went on to explain, then they must also include "healthy" controls who happen to have these other conditions. On the other hand, if researchers want to exclude other conditions, so as to clean up their investigation, then they must exclude both patient participants and controls with these other diagnoses.
_________________________________
Schwartz, S., and Susser, E. (2011). The use of well controls: an unhealthy practice in psychiatric research. Psychological Medicine, 41 (06), 1127-1131 DOI: 10.1017/S0033291710001595This post was written by Christian Jarrett for the BPS Research Digest.
- Depression Linked With Impaired Spatial Ability
Depression has a detrimental effect on people's ability to find their way around, according to researchers. Husseini Manji and colleagues compared the performance of 30 depressed patients and 19 healthy controls on a virtual reality spatial navigation...
- Body Image - It's 'healthy' People Who Are Deluded
We’re all going to die and there’s nothing we can do about it. Depressing? Well, it’s been argued that depressed people are the sane ones because they see the world for how it really is. Now consider this – a study has found people with eating...
- Cold Comfort For Shy People
"From the Archives", first published in the Digest 15.09.03 How sociable you are affects your chances of catching a cold. That's according to Sheldon Cohen (Carnegie Mellon University, USA) and colleagues (Universities of Pittsburgh and Virginia,...
- Abnormal Social Cognition
To investigate the neuroscience behind socialising, Andreas Meyer-Lindenberg and colleagues have scanned the brains of people with the genetic disorder Williams-Beuren syndrome and compared them with scans of healthy controls. People with Williams syndrome...
- Neuropsychology Abstract Of The Day: Chronic Congestive Heart Failure And Neuropsychological Function
Vogels RL, Oosterman JM, van Harten B, Scheltens P, van der Flier WM, Schroeder-Tanka JM, & Weinstein HC. Profile of Cognitive Impairment in Chronic Heart Failure. Journal of the American Geriatrics Society. 2007 Aug 28; [Epub ahead of print]. Department...