This Week in Neuroblunders: fMRI Edition

My entire body of work has been called into question!

And what a fine week for technical neurogaffes it is. First was the threat that many trendy and important studies of neural circuits may need to be replicated using old-fashioned lesion methods, because of “off-target” effects:

Where do we go from here? Most acute manipulation studies that use optogenetics confirm, and so add valuable support to, existing hypotheses that were established in earlier studies. But for those studies that have proposed new circuit functions, it may be advisable to re-evaluate the conclusions using independent approaches.¹

Up next we have....

fMRI Neuroblunders in Brief
 
The most notable one of late is a new paper by Eklund et al. (2015), which demonstrated that common statistical tests used to analyze fMRI data can give wildly inflated false positive rates of up to 60%, as illustrated in the top figure.

“What they found is shocking”!

While voxel-wise error rates were valid, nearly all cluster-based parametric methods (except for FSL’s FLAME 1) have greatly inflated familywise Type I error rates. This inflation was worst for analyses using lower cluster-forming thresholds (e.g. p=0.01) compared to higher thresholds, but even with higher thresholds there was serious inflation. This should be a sobering wake-up call for fMRI researchers, as it suggests that the methods used in a large number of previous publications suffer from exceedingly high false positive rates (sometimes greater than 50%).

The problems (and recommended solutions) were expertly discussed already by Russ Poldrack, who is quoted above (see Big problems for common fMRI thresholding methods), and by Neuroskeptic (False Positive fMRI Revisited). I needn't belabor the issues any further.


Next question:

Is the ubiquitously activated dorsal anterior cingulate cortex (dACC) selective for pain (as opposed to conflict or cognitive control or salience)? That was the contention of a new paper by Lieberman and Eisenberger (2015) that made use of the Neurosynth meta-analytic framework developed by Tal Yarkoni.

It Depends on What “Selective” Means ²

A 15,000 word debate between Yarkoni (No, the dorsal anterior cingulate is not selective for pain) and Lieberman (Comparing Pain, Cognitive, and Salience Accounts of dACC) ensued, with no end in sight.

It Also Depends on What “Pain” Means

Social Pain and Physical Pain Are Not Interchangeable. This may sound obvious to you, but Eisenberger and Lieberman have argued otherwise, with their neural alarm view of dACC function. Neurosynth uses text mining and machine learning to build maps based on terms that appear in published papers, along with activation coordinates. So the map above doesn't distinguish between different types of experimentally-induced physical pain (heat, cold, pressure, etc.) vs. emotional pain or social exclusion in a video game.

This might be one of L&E's major points, but pain researchers aren't on board; many don't even think the dorsal posterior insula is a pain-specific region.


Footnotes

¹ If anyone can parse the bold red sentence that appears in the Nature commentary (immediately after the first quoted passage in the post), please let me know.

In the future, it might be helpful always to correlate acute and chronic manipulations of specific neurons. If results from acute and chronic manipulations are discrepant, analyses of circuits that act in parallel to the manipulated circuit, or of similar neurons that are activated by different stimuli, might be more likely to provide an explanation for the discrepancy than examination of chains of hierarchically connected neurons, because off-target effects probably propagate throughout neural circuits by spilling over into adjacent, connected circuits.

² Sam Schwarzkopf addressed this in his post, What is selectivity?