Neuroscience
In the spirit of American political debate shows such as Crossfire, The McLaughlin Group, Hannity & Colmes, and the classic Point/Counterpoint (both the 60 Minutes and SNL versions), The Neurocritic is pleased to present an excerpt from a rebuttal to the lively and controversial paper by Vul, Harris, Winkielman, and Pashler (PDF).
Two "anonymous commenters" tipped me off to the preliminary version of a detailed reply by some of the authors on the Vul et al. hit list. The entire document is available for download as a PDF. The abstract, main bullet points, and conclusions are reproduced below.
The paper by Vul et al., entitled "Voodoo correlations in social neuroscience" and accepted for publication by Perspectives on Psychological Science, claims that "a disturbingly large, and quite prominent, segment of social neuroscience research is using seriously defective research methods and producing a profusion of numbers that should not be believed." In all brevity, we here summarise conceptual shortcomings and methodological errors of this paper and explain why their criticisms are invalid. A detailed reply will be submitted to a peer reviewed scientific journal shortly.
1. The authors misunderstand the critical role of multiple comparison corrections and conflate issues pertaining to null hypothesis testing and effect size estimates, respectively.
2. The authors make strong claims on the basis of a questionable upper bound argument.
3. The authors use misleading simulations to support their claims.
4. The authors inappropriately dismiss the existence of non-significant correlations.
5. The authors' understanding of the rationale behind the use and interpretation of correlations in social neuroscience is incomplete.
6. The authors ignore that the same brain-behaviour correlations have been replicated by several independent studies and that major results in social neuroscience are not based on correlations at all.
7. The authors used an ambiguous and incomplete questionnaire.
8. The authors make flawed suggestions for data analysis.
. . .
ConclusionsIn this summary, we have provided a very brief summary that exposes some of the flaws that undermine the criticisms by Vul et al. We have pointed out that brain-behaviour correlations in social neuroscience are valid, provided that one adheres to good statistical practice. It has also been emphasized that many analyses and findings in social neuroscience do not rest on brain-behaviour correlations and have been replicated several times by independent studies conducted by different laboratories. A full analysis of the Vul et al. paper and a detailed reply will be submitted to a peer-reviewed scientific journal shortly.
_____________________________________________________
A rebuttal to the rebuttal, along with commentary by The Neurocritic, all to come in the next exciting episode!
But for now, you can watch Colbert & Colmes discuss Roland Burris, who was appointed to replace Barack Obama as junior senator of Illinois.
- Do You Do Voodoo?
They are beloved by prestigious journals and the popular press, but many recent social neuroscience studies are profoundly flawed, according to a devastating critique - Voodoo Correlations in Social Neuroscience - in press at Perspectives on Psychological...
- Nominees For The 2009 3qd Prize In Science
3 Quarks Daily 2009 Science Prize: Vote HereFor details of the prize you can look at the announcement here, and to read the nominated posts you can go here for a complete list with links. Voting ends at midnight on June 8, 2009. Four posts from The Neurocritic...
- The Paper Formerly Known As "voodoo Correlations In Social Neuroscience"
Voodoo no more! The paper everyone loves (or loves to hate) has a new name.1 Through a number of channels [The Chronicle of Higher Education via @vaughanbell, Ed Vul's website, and Neuroskeptic], The Neurocritic has learned that the "Voodoo Correlations"...
- Voodoo Gurus
Marie Laveau's House of Voodoo [by OZinOH] Does anyone else miss having the Society for Neuroscience conference in New Orleans as much as I do? The 2003 meeting was great fun, but there seems to be no plan to return any time soon. The 2009 SFN meeting...
- Scan Scandal Hits Social Neuroscience
Mind Hacks uncovers a pre-print (PDF) by Vul, Harris, Winkielman, and Pashler entitled "Voodoo Correlations in Social Neuroscience". It's a "bombshell of a paper" that questions the implausibly high correlations observed in some fMRI studies in the...
Neuroscience
Voodoo Counterpoint
In the spirit of American political debate shows such as Crossfire, The McLaughlin Group, Hannity & Colmes, and the classic Point/Counterpoint (both the 60 Minutes and SNL versions), The Neurocritic is pleased to present an excerpt from a rebuttal to the lively and controversial paper by Vul, Harris, Winkielman, and Pashler (PDF).
Two "anonymous commenters" tipped me off to the preliminary version of a detailed reply by some of the authors on the Vul et al. hit list. The entire document is available for download as a PDF. The abstract, main bullet points, and conclusions are reproduced below.Rebuttal of "Voodoo Correlations in Social Neuroscience" by Vul et al. – summary information for the press
Mbemba Jabbi1, Christian Keysers2, Tania Singer3, Klaas Enno Stephan3,
(authors listed in alphabetical order)
1 National Institute of mental Health, Bethesda, Maryland, USA.2 University Medical Center Groningen, Department of Neuroscience, University of Groningen, The Netherlands. www.bcn-nic.nl/socialbrain.html3 Laboratory for Social and Neural Systems Research, University of Zurich, Switzerland. http://www.socialbehavior.uzh.ch/index.html
Mbemba Jabbi1, Christian Keysers2, Tania Singer3, Klaas Enno Stephan3,
(authors listed in alphabetical order)
1 National Institute of mental Health, Bethesda, Maryland, USA.2 University Medical Center Groningen, Department of Neuroscience, University of Groningen, The Netherlands. www.bcn-nic.nl/socialbrain.html3 Laboratory for Social and Neural Systems Research, University of Zurich, Switzerland. http://www.socialbehavior.uzh.ch/index.html
The paper by Vul et al., entitled "Voodoo correlations in social neuroscience" and accepted for publication by Perspectives on Psychological Science, claims that "a disturbingly large, and quite prominent, segment of social neuroscience research is using seriously defective research methods and producing a profusion of numbers that should not be believed." In all brevity, we here summarise conceptual shortcomings and methodological errors of this paper and explain why their criticisms are invalid. A detailed reply will be submitted to a peer reviewed scientific journal shortly.
1. The authors misunderstand the critical role of multiple comparison corrections and conflate issues pertaining to null hypothesis testing and effect size estimates, respectively.
2. The authors make strong claims on the basis of a questionable upper bound argument.
3. The authors use misleading simulations to support their claims.
4. The authors inappropriately dismiss the existence of non-significant correlations.
5. The authors' understanding of the rationale behind the use and interpretation of correlations in social neuroscience is incomplete.
6. The authors ignore that the same brain-behaviour correlations have been replicated by several independent studies and that major results in social neuroscience are not based on correlations at all.
7. The authors used an ambiguous and incomplete questionnaire.
8. The authors make flawed suggestions for data analysis.
. . .
ConclusionsIn this summary, we have provided a very brief summary that exposes some of the flaws that undermine the criticisms by Vul et al. We have pointed out that brain-behaviour correlations in social neuroscience are valid, provided that one adheres to good statistical practice. It has also been emphasized that many analyses and findings in social neuroscience do not rest on brain-behaviour correlations and have been replicated several times by independent studies conducted by different laboratories. A full analysis of the Vul et al. paper and a detailed reply will be submitted to a peer-reviewed scientific journal shortly.
_____________________________________________________
A rebuttal to the rebuttal, along with commentary by The Neurocritic, all to come in the next exciting episode!
But for now, you can watch Colbert & Colmes discuss Roland Burris, who was appointed to replace Barack Obama as junior senator of Illinois.
- Do You Do Voodoo?
They are beloved by prestigious journals and the popular press, but many recent social neuroscience studies are profoundly flawed, according to a devastating critique - Voodoo Correlations in Social Neuroscience - in press at Perspectives on Psychological...
- Nominees For The 2009 3qd Prize In Science
3 Quarks Daily 2009 Science Prize: Vote HereFor details of the prize you can look at the announcement here, and to read the nominated posts you can go here for a complete list with links. Voting ends at midnight on June 8, 2009. Four posts from The Neurocritic...
- The Paper Formerly Known As "voodoo Correlations In Social Neuroscience"
Voodoo no more! The paper everyone loves (or loves to hate) has a new name.1 Through a number of channels [The Chronicle of Higher Education via @vaughanbell, Ed Vul's website, and Neuroskeptic], The Neurocritic has learned that the "Voodoo Correlations"...
- Voodoo Gurus
Marie Laveau's House of Voodoo [by OZinOH] Does anyone else miss having the Society for Neuroscience conference in New Orleans as much as I do? The 2003 meeting was great fun, but there seems to be no plan to return any time soon. The 2009 SFN meeting...
- Scan Scandal Hits Social Neuroscience
Mind Hacks uncovers a pre-print (PDF) by Vul, Harris, Winkielman, and Pashler entitled "Voodoo Correlations in Social Neuroscience". It's a "bombshell of a paper" that questions the implausibly high correlations observed in some fMRI studies in the...