Towards a Focal Consensus in Cognitive Neuroscience: Databases and Meta-Analyses

Moving right along with our timely, fast-paced, cutting edge blog coverage from the CNS 2010 Annual Meeting [held last month], the first symposium urged the field to advance beyond the current piecemeal single-study approach to neuroimaging by moving Towards a cumulative science of human brain function.¹ Building comprehensive, structured, and searchable databases (Van Essen, 2009) and using meta-analytic tools (Wager et al., 2009) were proposed to be key methods aimed at achieving this goal.

In his talk, prolific neuroanatomist Dr. David Van Essen (of primate visual cortex wiring diagram fame)² discussed SumsDB (Surface Management System Database), "a repository of brain-mapping data (surfaces & volumes; structural & functional data) from many laboratories."

---------------
Talk 2: Lost in Localization – But Found with Foci!

David Van Essen; Washington University in St. Louis

More than 50,000 studies related to functional imaging of the human brain have been published in recent decades. Of these, more than 10,000 report key experimental data (centers of fMRI activation foci, etc.) in tables of stereotaxic coordinates (‘foci’) in one or another standardized atlas space. To aid in mining this extensive literature, we developed the SumsDB database, which supports storage, visualization, and searching of many types of neuroimaging data. SumsDB includes a Foci Library that currently contains >40,000 foci from ~1,400 published studies. This includes comprehensive coverage of five major journals and almost 15% of the relevant literature. Foci searches can be based on many criteria (e.g., cortical area or region, spatial coordinates, functional criteria, or disease condition). Search results can be viewed online (WebCaret) or downloaded for offline visualization and analysis using Caret sofware. As the Foci Library continues to expand, through contributions from curators and volunteers alike, it will become increasingly valuable as a way to efficiently access the burgeoning neuroimaging literature.
---------------

Van Essen emphasized the importance of maintaining a central repository of neuroimaging foci, the 3D localization of peak activations in x, y, z coordinates (Fox et al., 1985). He and his lab have established SumsDB, which currently contains 50,000 foci from 1,700 studies, a relatively small fraction of the literature (see figure below).


Fig. 1 (Derrfuss & Mar, 2009). Total number of published fMRI studies reporting coordinates by year and number of studies included in current coordinate databases.

One impediment to having more complete coverage of the literature is how labor intensive it is to add new articles to the database (30-60 min per paper after 5-10 hr training), as lamented by Derrfuss and Mar (2009) in their Comments and Controversies article in NeuroImage. Because of the effort involved, many in the neuroimaging community haven't been particularly motivated to participate in the project. In in his reply (2009) to Derrfuss and Mar, Van Essen listed the benefits of voluntary data entry duties:

• submitting foci from publications of your own lab will increase their visibility, through data mining initiated in SumsDB or NIF [Neuroscience Information Framework];

• submitting relevant studies from your research subfield will facilitate cross-study comparisons and promote broader awareness of research in that area;

• individual contributors are recognized by ‘provenance’ assignments for each study (or version) entered into SumsDB.

• SumsDB libraries can also be used to store foci and study collections for ongoing projects that are not yet published. (Data in these libraries are not made public until requested by the submitter and then vetted by a curator in the Van Essen lab to insure conformance to basic metadata description standards.)

And what a great classroom project for graduate students and highly motivated undergraduates! (suggested Van Essen, 2009). Greater participation is essential, however. But who wants to do all that work for free?

An attractive and feasible model is for one or two individuals (students, postdocs, or knowledgeable technicians) from each of many laboratories to enter data published by their own laboratory plus selected topics related to that lab's research interests. For example, if 50 volunteers each added ~20 studies per year (15–30 h per volunteer, including training), the current rate of submission would approximately double, and about half of the relevant literature would be covered in ~5 years.

But even more appealing, a semi-automated data entry system for SumsDB is under development...

What does one do with all that data? In the next talk, Dr. Tor Wager discussed meta-analysis, a statistical technique for summarizing quantitative research. Why is it important to combine results across multiple studies (Wager et al. 2007)?

Meta-analysis is an increasingly popular and valuable tool for summarizing results across many neuroimaging studies. It can be used to establish consensus on the locations of functional regions, test hypotheses developed from patient and animal studies and develop new hypotheses on structure–function correspondence. It is particularly valuable in neuroimaging because most studies do not adequately correct for multiple comparisons; based on statistical thresholds used, we estimate that roughly 10–20% of reported activations in published studies are false positives.

The 'Quick-Search' function in SumsDB can be used to retrieve foci of interest from all papers in the database. For example, one can search by anatomical area (Fig. 1B) or by research topic (Fig. 1C). Searches can also be performed by task, function, disorder, etc.


Adapted from Fig. 1 (Van Essen, 2009). (B) 493 foci localized to area MT+, displayed on the PALS-B12 right hemisphere [Population-Average, Landmark- and Surface-based atlas]. (C) 627 foci associated with studies containing the term ‘music’ in the abstract, keywords, or other metadata.

BrainMap is another major online database. It was developed at the Research Imaging Institute of the University of Texas Health Science Center San Antonio (Fox et al., 2005) and currently contains 1,928 papers (and a more user-friendly interface than SumsDB). The GingerALE program can perform coordinate-based meta-analyses of the neuroimaging literature (on chosen topics or brain areas) using the activation likelihood estimation (ALE) method (see this PDF).

The example below is a figure I generated from entering the x, y, z coordinates from a specific right insular focus [related to low $ offers in a Trust Game by King-Casas et al., 2008] into the Sleuth program (available at brainmap.org), which searched the available database of papers for matches [see Who Can You Trust?]. The resulting list of coordinates and experiments was then imported into the GingerALE program, which performed the ALE meta-analysis. The figure illustrates that this exact same region of the right insula was activated during tasks that assessed speech, language, explicit memory, working memory, reasoning, pain, and listening to emotional music (i.e., not only during social norm violations in the Trust Game).



Other Neurocritic-generated examples include ALE meta-analyses for physical pain (Hypnosis and Pain Control), the left head of the caudate nucleus (Sie Ist Gefundene Steuerung Über Two Languages: Part 2), the right dorsolateral prefrontal cortex (Crime, Punishment, and Jerry Springer), and an update on physical pain (Bleed It Out).

Tor Wager has developed his own meta-analysis software (Multilevel Kernel Density Analysis), a set of Matlab scripts and functions freely available from his website. In addition, he has a nifty collection of meta-analysis data and image files that can be downloaded as well. The documentation for MKDA (PDF) notes some problems with other meta-analysis procedures:

Prior meta-analyses have divided the brain into voxels and plotted peak coordinates. Then count how many peaks within each voxel (the observed frequency count). Compare this to the number expected by chance if peaks were distributed randomly throughout the brain (the expected frequency count). Have to establish threshold (using Monte Carlo method).

Problems with this method: This is a fixed effect procedure which ignores the fact that points are not independent of one another (as they are nested within contrasts within studies). An important consequence is that any single study that has a large number of peaks (due to differences in reporting, voxel size, thresholding) can overly influence the analyses.

MKDA offers a number of different options for dealing with these issues (Wager et al., 2009).

In his talk, Wager discussed four major uses of meta-analysis:

1. Formulating a priori hypotheses
2. Testing existing theories
3. Developing new theories
4. Making inferences about psychological states

---------------
Talk 3: Consensus-building and brain-based taxonomies using meta-analysis

Tor Wager; University of Colorado at Boulder

Much of cognitive and affective neuroscience has centered on identifying particular brain regions or circuits with categories of psychological processes. Properly synthesized, the accumulation of knowledge can aid in this goal. However, without synthesis, more information is not necessarily better, as it may be difficult to separate truly activated brain regions from spurious or idiosyncratic findings. Meta-analysis of neuroimaging data provides a potential solution to this problem. I will present methods and findings from meta-analyses of both cognitive control and emotion. Meta-analyses of cognitive control reveal a consensus on the prefrontal cortical networks involved in cognitive control processes, and a complexity-dependent posterior-anterior hierarchy of prefrontal activity that complements recent findings on prefrontal organization. Conversely, meta-analyses of emotion argue against several long-standing principles of organization of the emotional brain, paving the way for the development of new models. These findings illustrate the utility of meta-analysis in developing taxonomies of psychological processes based on the patterns of brain activity they elicit rather than folk psychological categorization schemes. I will close the talk by presenting a brain-based classification of psychological tasks that suggests that different quadrants of the brain respect different organizational schemes.
---------------

The talk used some examples from fMRI studies of emotion. For instance, "The insula is a marker of disgust." In a test of this existing theory, meta-analytic results demonstrated there isn't much specificity in the insula for disgust (as also shown above). Test of another theory ("the amygdala is specific for fear") revealed results consistent with that notion, but the amygdala was just as active for disgust as for fear. Another region important for experiencing emotions emerged from the meta-analysis: the periaqueductal gray (PAG), better known for its involvement in pain and defensive behavior.

A final note is the importance of identifying networks of brain regions involved in a particular function. This can be accomplished by identifying consistent patterns of co-activation in a given meta-analysis (Wager et al., 2009). It's time to stop the madness of the “one tiny brain area controls complex behavior” approach!

Footnotes

¹ For more information about the symposium and the first talk in the series, see Coordinates, Consensus, Cooperation: Towards a Cumulative Science of Human Brain Function and Motivating a Cumulative Cognitive Neuroscience, plus this helpful list of links and slides compiled by the symposium organizer.

² Classic 1991 Felleman and Van Essen paper ("Distributed hierarchical processing in the primate cerebral cortex") is freely available as a 47 page PDF.

References

Derrfuss J, Mar RA. (2009). Lost in localization: the need for a universal coordinate database. Neuroimage 48:1-7.

Fox PT, Laird AR, Fox SP, Fox PM, Uecker AM, Crank M, Koenig SF, Lancaster JL. (2005). BrainMap taxonomy of experimental design: description and evaluation. Hum Brain Mapp. 25:185-98.

Fox PT, Perlmutter JS, Raichle ME. (1985). A stereotactic method of anatomical localization for positron emission tomography. J Comput Assist Tomogr. 9:141-53.

Van Essen, D. (2009). Lost in localization — But found with foci?! NeuroImage, 48 (1), 14-17. DOI: 10.1016/j.neuroimage.2009.05.050

Wager TD, Lindquist M, Kaplan L. (2007). Meta-analysis of functional neuroimaging data: current and future directions. Soc Cogn Affect Neurosci. 2:150-8.

Wager, T., Lindquist, M., Nichols, T., Kober, H., & Van Snellenberg, J. (2009). Evaluating the consistency and specificity of neuroimaging data using meta-analysis. NeuroImage, 45 (1) DOI: 10.1016/j.neuroimage.2008.10.061

Figure 4 (Felleman & Van Essen, 1991). Hierarchy of visual areas. This hierarchy shows 32 visual cortical areas. ... These areas are connected by 187 linkages, most of which have been demonstrated to be reciprocal pathways.