Excess Significance Bias in the Literature on Brain Volume Abnormalities

ContextMany studies report volume abnormalities in diverse brain structures in patients with various mental health conditions.ObjectiveTo evaluate whether there is evidence for an excess number of statistically significant results in studies of brain volume abnormalities that suggest the presence of bias in the literature.Data sourcesPubMed (articles published from January 2006 to December 2009).Study selectionRecent meta-analyses of brain volume abnormalities in participants with various mental health conditions vs control participants with 6 or more data sets included, excluding voxel-based morphometry.Data extractionStandardized effect sizes were extracted in each data set, and it was noted whether the results were "positive" (P Data synthesisFrom 8 articles, 41 meta-analyses with 461 data sets were evaluated (median, 10 data sets per meta-analysis) pertaining to 7 conditions. Twenty-one of the 41 meta-analyses had found statistically significant associations, and 142 of 461 (31%) data sets had positive results. Even if the summary effect sizes of the meta-analyses were unbiased, the expected number of positive results would have been only 78.5 compared with the observed number of 142 (P ConclusionThere are too many studies with statistically significant results in the literature on brain volume abnormalities. This pattern suggests strong biases in the literature, with selective outcome reporting and selective analyses reporting being possible explanations.