Voodoo Schadenfreude

Voodoo doll, by Sickboy

Most hip researchers in cognitive neuroscience and human brain imaging have already heard about the critical new journal article with the incendiary title: "Voodoo Correlations in Social Neuroscience" (Vul et al., in press - PDF). If you haven't, you can read a comprehensive summary here and a micro version here.

Avenging Voodoo Schadenfreude

Nature News ran a piece on the debate and the burgeoning backlash from an angry mob of researchers whose methods were derided as fatally flawed. Some of these authors (and perhaps some Nature editors) were miffed that bloggers wrote about the preprint when it was first made available to the public, as if that somehow violates the scientific method:

The swift rebuttal was prompted by scientists' alarm at the speed with which the accusations have spread through the community. The provocative title — 'Voodoo correlations in social neuroscience' — and iconoclastic tone have attracted coverage on many blogs, including that of Newsweek. Those attacked say they have not had the chance to argue their case in the normal academic channels.

"I first heard about this when I got a call from a journalist," comments neuroscientist Tania Singer of the University of Zurich, Switzerland, whose papers on empathy are listed as examples of bad analytical practice. "I was shocked — this is not the way that scientific discourse should take place." Singer says she asked for a discussion with the authors when she received the questionnaire, to clarify the type of information needed, but got no reply.

Based on the statements above, it would seem that Dr. Singer and her colleagues (Jabbi, Keysers, and Stephan) are not keeping up with the way that scientific discourse is evolving. [See Mind Hacks on this point as well.] Citing "in press" articles in the normal academic channels is a frequent event; why should bloggers, some of whom are read more widely than the authors' original papers, refrain from such a practice? Is it the "read more widely" part? To their credit, however, they commented in blogs and publicized the link to a preliminary version of their detailed reply.....although calling it "summary information for the press" assumes that "the press" is extremely knowledgeable about neuroimaging methodology and statistical analysis.

To learn more about the evolution of scientific discourse, let me briefly introduce you to the world of social media (e.g., FriendFeed, Facebook, and even Twitter). You can join the discussion at FriendFeed's Science 2.0 Room, which is "For people interested in Science 2.0 and Open Science, especially the use of online tools to do science in new ways." Although one needs a Facebook account to view these, Facebook groups include Neuroscience ROCKS (4,006 members), Neuroscience and Brain Studies (3,194 members), and Cognitive Neuroscience Society (2,147 members). Some social neuroscientists are, well, social enough to get it, because Columbia University Social Cognitive Affective Neuroscience Lab has a posse (79 fans, albeit inactive ones). And believe it or not, NIH now has Twitter feeds for press releases and funding announcements. As for individuals who are powerhouse resources on the future of scientific communication, I would recommend reading BoraZ (blog, Twitter), who is organizing the Jan. 16-18 ScienceOnline’09 conference, and Björn Brembs on scientific publishing and misuse of the Impact Factor.

All is not puppies and flowers in the world of science social media, however. Proponents rarely acknowledge that many companies and institutions block access to these sites, so at present their usefulness is limited for many in the scientific community. A more obvious issue is that these sites can turn into an enormous time sink.

Now back to Nature News and the voodoo backlash. In an ironic twist, one of the 'red listed' papers (Singer et al., 2006), published in Nature, was publicized as a study on Schadenfreude. Here's the Editor's Summary [which was covered by The Neurocritic three years ago]:

I feel your pain

Humans have the capacity to empathize with the pain of others, but we don't empathize in all circumstances. An experiment on human volunteers playing an economic game looked at the conditional nature of our sympathy, and the results show that fairness of social interactions is key to the empathic neural response. Both men and women empathized with the pain of cooperative people. But if people are selfish, empathic responses were absent, at least in men. And it seems that physical harm might even be considered a good outcome — perhaps the first neuroscientific evidence for schadenfreude.

Nature and Science have a long history of issuing overblown press releases that extrapolate the findings of a single, quite flawed [if you side with Vul et al.] neuroimaging paper to yield the revelation of deep truths about human social interactions (among other things). The Nature News piece, Brain imaging studies under fire (Abbott, 2009), continues:

The article is scheduled for publication in September, alongside one or more replies. But the accused scientists are concerned that the impression now being established through media reports will be hard to shake after the nine-month delay. "We are not worried about our close colleagues, who will understand the arguments. We are worried that the whole enterprise of social neuroscience falls into disrepute," says neuroscientist Chris Frith of University College London, whose Nature paper [Singer et al., 2006] on response to perceived fairness was called into question.

So media reports heavily promoted the field, and media reports will unduly tarnish the field.¹

NewScientist provides a clear instance of this, in what is surely a textbook exemplar of a pot-kettle moment.

Doubts raised over brain scan findings

14 January by Jim Giles

SOME of the hottest results in the nascent field of social neuroscience, in which emotions and behavioural traits are linked to activity in a particular region of the brain, may be inflated and in some cases entirely spurious.

But one doesn't have to look very far to find NewScientist headlines like these (I just searched the archives of this blog):

Watching the brain 'switch off' self-awareness

Do games prime brain for violence?

Starving is like ecstasy use for anorexia sufferers

Mirror neurons control erection response to porn

Source of ‘optimism’ found in the brain

So the NS editorial below comes across as a wee bit hypocritical, even though it eventually acknowledges their own role in promoting "sexy-sounding" brain scan results.

Editorial: What were the neuroscientists thinking?

14 January 2009

IT IS two centuries since the birth of Charles Darwin, but even now his advice can be spot on. The great man attempted a little neuroscience in The Expressions of the Emotions in Man and Animals, published in 1872, in which he discussed the link between facial expressions and the brain. "Our present subject is very obscure," Darwin warned in his book, "and it is always advisable to perceive clearly our ignorance."

Modern-day neuroscience might benefit from adopting a similar stance. The field has produced some wonderful science, including endless technicolor images of the brain at work and headline-grabbing papers about the areas that "light up" when registering emotions. Researchers charted those sad spots that winked on in women mourning the end of a relationship, the areas that got fired up when thinking about infidelity, or those that surged in arachnophobes when they thought they were about to see a spider. The subjective subject of feelings seemed at last to be becoming objective.

Now it seems that a good chunk of the papers in this field contain exaggerated claims, according to an analysis which suggests that "voodoo correlations" often inflate the link between brain areas and particular behaviours.

Some of the resulting headlines appeared in New Scientist, so we have to eat a little humble pie and resolve that next time a sexy-sounding brain scan result appears we will strive to apply a little more scepticism to our coverage.

Um, no joke guys.

On the other hand, Sharon Begley at Newsweek is one science writer who hasn't been entirely convinced by the colorful brain images. On March 10, 2008, she wrote:

Brain-imaging studies have proliferated so mindlessly (no pun intended) that neuroscientists should have to wear a badge pleading, “stop me before I scan again.” I mean, does it really add to the sum total of human knowledge to learn that the brain’s emotion regions become active when people listen to candidates for president? Or that the reward circuitry in the brains of drug addicts become active when they see drug paraphernalia?

Therefore, her recent commentary on the brouhaha does not come across as an opinion that was invented yesterday:

The 'Voodoo' Science of Brain Imaging

If you are a fan of science news, then odds are you are also intrigued by brain imaging, the technique that produces those colorful pictures of brains “lit up” with activity, showing which regions are behind which behaviors, thoughts and emotions. So maybe you remember these recent hits... [gives many examples here] . . . the list goes on and on and on. And now a bombshell has fallen on dozens of such studies: according to a team of well-respected scientists, they amount to little more than voodoo science.

The neuroscience blogosphere is crackling with—so far—glee over the upcoming paper, which rips apart an entire field: the use of brain imaging in social neuroscience.....

Before concluding, I will state that I am not a complete neuroimaging nihilist. For examples of this view, see Coltheart, 2006 and especially van Orden and Paap, 1997 (as quoted by Coltheart):

What has functional neuroimaging told us about the mind so far? Nothing, and it never will: the nature of cognition is such that this technique in principle cannot provide evidence about the nature of cognition.

So no, I am not a Jerry Fodor Functionalist. I do believe that learning about human brain function is essential to learing about "the mind," that the latter can be reduced to the former, that fMRI can have something useful to say, and (more broadly, in case any anti-psychiatry types are listening) that psychiatric disorders are indeed caused by faulty brain function. But there's still a lot about fMRI as a technique that we don't really know. The best-practice statistical procedures for analyzing functional images is obviously a contentious issue; there is no consensus at this point. Our knowledge of what the BOLD signal is measuring, exactly, is not very clear either [see the recent announcement in J. Neurosci. that "BOLD Signals Do Not Always Reflect Neural Activity."] The critics among us² are not trying to trash the entire field of social neuroscience (or neuroimaging in general). Some of us are taking concrete steps to open a dialogue and improve its methodology, while others are trying to rein in runaway interpretations.


ADDENDUM: via Pieces of Me, I've just discovered the link to PsyBlog's detailed discussion of the Coltheart paper: Can Cognitive Neuroscience Tell Us Anything About the Mind?

Footnote

¹ It isn't even necessary to quote the appropriate metaphorical expression here.

² By "us" I mean scientists: people who are students and post-docs and colleagues of esteemed investigators like Dr. Frith.

References

Abbott A (2009). News: Brain imaging studies under fire. Social neuroscientists criticized for exaggerating links between brain activity and emotions. Nature 457:245.

Jabbi M, Keysers C, Singer T, Stephan KE. (in preparation). Rebuttal of "Voodoo Correlations in Social Neuroscience" by Vul et al. – summary information for the press. PDF

Singer T, Seymour B, O'doherty JP, Stephan KE, Dolan RJ, Frith CD. (2006) Empathic neural responses are modulated by the perceived fairness of others. Nature 439:466-9.

Vul E, Harris C, Winkielman P, Pashler H (2009). Voodoo Correlations in Social Neuroscience. Perspectives on Psychological Science, in press. PDF