Neuroscience
postcards from nowhere
...or triple entendre, or even more [depending on your source].
Stolen from Neuroanthropology's Wednesday Round Up #62:
Reference
Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI. (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci. 12:535-40.
A neuroscientific experiment typically generates a large amount of data, of which only a small fraction is analyzed in detail and presented in a publication. However, selection among noisy measurements can render circular an otherwise appropriate analysis and invalidate results. Here we argue that systems neuroscience needs to adjust some widespread practices to avoid the circularity that can arise from selection. In particular, 'double dipping', the use of the same dataset for selection and selective analysis, will give distorted descriptive statistics and invalid statistical inference whenever the results statistics are not inherently independent of the selection criteria under the null hypothesis. To demonstrate the problem, we apply widely used analyses to noise data known to not contain the experimental effects in question. Spurious effects can appear in the context of both univariate activation analysis and multivariate pattern-information analysis. We suggest a policy for avoiding circularity.
- Motivating A Cumulative Cognitive Neuroscience
Why are large-scale structured databases and meta-analyses important to advance the field of human brain mapping? One reason is that individual functional magnetic resonance imaging (fMRI) studies can be notoriously unreliable and underpowered (Bennett...
- Scan Scandal Hits Social Neuroscience
Mind Hacks uncovers a pre-print (PDF) by Vul, Harris, Winkielman, and Pashler entitled "Voodoo Correlations in Social Neuroscience". It's a "bombshell of a paper" that questions the implausibly high correlations observed in some fMRI studies in the...
- Research Assistant Positions With The Diversity & Selection Lab!
Research Assistant Positions Diversity & Selection Lab Spring 2016 Description: The Diversity & Selection Lab focuses on diversity issues during the applicant selection process in organizations. The lab is more specifically interested in...
- Internship With College Park's Optimal Solutions Group!
Internship Optimal Solutions Group Health Policy Research Center Optimal Solutions Group has an opening for an intern in its Health Policy Research Center. The intern would assist senior researchers in a variety of research and analysis tasks including:...
- Take Your Statistical Skills To The Next Level With Psyc 489j!
PSYC 489J: Advanced Statistics for Psychology Fall 2012 Tuesday & Thursday 9:30-10:45 Instructor: Kristen Klein Prerequisite: PSYC 300 Registration by permission of instructor If you’re interested in graduate school or a career in research, this...
Neuroscience
double entendre
postcards from nowhere
...or triple entendre, or even more [depending on your source].
Stolen from Neuroanthropology's Wednesday Round Up #62:
Others have already written about the new voodoo-esque paper by Kriegeskorte et al. (2009). Having blogged extensively about the original controversy, I don't have anything to add... other than the links above... and some fun and wholesome pictures.Research Digest, It’s Those Voodoo Correlations Again … Brain Imagers Accused of “Double Dipping”
More methods problems for imaging researchers – using the same data twice, first to find the area and then to show that area is really the one responsible for whatever hypothesis is at stake. For more commentary, see Neuroskeptic, Mind Hacks, and Newsweek.
Reference
Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI. (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci. 12:535-40.
A neuroscientific experiment typically generates a large amount of data, of which only a small fraction is analyzed in detail and presented in a publication. However, selection among noisy measurements can render circular an otherwise appropriate analysis and invalidate results. Here we argue that systems neuroscience needs to adjust some widespread practices to avoid the circularity that can arise from selection. In particular, 'double dipping', the use of the same dataset for selection and selective analysis, will give distorted descriptive statistics and invalid statistical inference whenever the results statistics are not inherently independent of the selection criteria under the null hypothesis. To demonstrate the problem, we apply widely used analyses to noise data known to not contain the experimental effects in question. Spurious effects can appear in the context of both univariate activation analysis and multivariate pattern-information analysis. We suggest a policy for avoiding circularity.
- Motivating A Cumulative Cognitive Neuroscience
Why are large-scale structured databases and meta-analyses important to advance the field of human brain mapping? One reason is that individual functional magnetic resonance imaging (fMRI) studies can be notoriously unreliable and underpowered (Bennett...
- Scan Scandal Hits Social Neuroscience
Mind Hacks uncovers a pre-print (PDF) by Vul, Harris, Winkielman, and Pashler entitled "Voodoo Correlations in Social Neuroscience". It's a "bombshell of a paper" that questions the implausibly high correlations observed in some fMRI studies in the...
- Research Assistant Positions With The Diversity & Selection Lab!
Research Assistant Positions Diversity & Selection Lab Spring 2016 Description: The Diversity & Selection Lab focuses on diversity issues during the applicant selection process in organizations. The lab is more specifically interested in...
- Internship With College Park's Optimal Solutions Group!
Internship Optimal Solutions Group Health Policy Research Center Optimal Solutions Group has an opening for an intern in its Health Policy Research Center. The intern would assist senior researchers in a variety of research and analysis tasks including:...
- Take Your Statistical Skills To The Next Level With Psyc 489j!
PSYC 489J: Advanced Statistics for Psychology Fall 2012 Tuesday & Thursday 9:30-10:45 Instructor: Kristen Klein Prerequisite: PSYC 300 Registration by permission of instructor If you’re interested in graduate school or a career in research, this...