Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies

ContextThere is substantial debate about whether the results of nonrandomized studies are consistent with the results of randomized controlled trials on the same topic.ObjectivesTo compare results of randomized and nonrandomized studies that evaluated medical interventions and to examine characteristics that may explain discrepancies between randomized and nonrandomized studies.Data sourcesMEDLINE (1966-March 2000), the Cochrane Library (Issue 3, 2000), and major journals were searched.Study selectionForty-five diverse topics were identified for which both randomized trials (n = 240) and nonrandomized studies (n = 168) had been performed and had been considered in meta-analyses of binary outcomes.Data extractionData on events per patient in each study arm and design and characteristics of each study considered in each meta-analysis were extracted and synthesized separately for randomized and nonrandomized studies.Data synthesisVery good correlation was observed between the summary odds ratios of randomized and nonrandomized studies (r = 0.75; PConclusionsDespite good correlation between randomized trials and nonrandomized studies-in particular, prospective studies-discrepancies beyond chance do occur and differences in estimated magnitude of treatment effect are very common.