Reliability in Cognitive Neuroscience: A Meta-Meta-Analysis

Cognitive neuroscientists increasingly claim that brain images generated by new brain imaging technologies reflect, correlate, or represent cognitive processes. In this book, William Uttal warns against these claims, arguing that, despite its utility in anatomic and physiological applications, brain imaging research has not provided consistent evidence for correlation with cognition.

Uttal bases his argument on an extensive review of the empirical literature, pointing to variability in data not only among subjects within individual experiments but also in the new meta-analytical approach that pools data from different experiments. This inconsistency of results, he argues, has profound implications for the field, suggesting that cognitive neuroscientists have not yet proven their interpretations of the relation between brain activity captured by macroscopic imaging techniques and cognitive processes; what may have appeared to be correlations may have only been illusions of association. He supports the view that the true correlates are located at a much more microscopic level of analysis: the networks of neurons that make up the brain.

Uttal carries out comparisons of the empirical data at several levels of data pooling, including the meta-analytical. He argues that although the idea seems straightforward, the task of pooling data from different experiments is extremely complex, leading to uncertain results, and that little is gained by it. Uttal's investigation suggests a need for cognitive neuroscience to reevaluate the entire enterprise of brain imaging-cognition correlational studies.