Generalizability of findings from a clinical sample to a community-based sample: A comparison of ADNI and ARIC

Introduction Clinic-based study samples, including the Alzheimer's Disease Neuroimaging Initiative (ADNI), offer rich data, but findings may not generalize to community-based settings. We compared associations in ADNI to those in the Atherosclerosis Risk in Communities (ARIC) study to assess generalizability across the two settings. Methods We estimated cohort-specific associations among risk factors, cognitive test scores, and neuroimaging outcomes to identify and quantify the extent of significant and substantively meaningful differences in associations between cohorts. We explored whether using more homogenous samples improved comparability in effect estimates. Results The proportion of associations that differed significantly between cohorts ranged from 27% to 34% across sample subsets. Many differences were substantively meaningful (e.g., odds ratios [OR] for apolipoprotein E e4 on amyloid positivity in ARIC: OR = 2.8, in ADNI: OR = 8.6). Discussion A higher proportion of associations differed significantly and substantively than would be expected by chance. Findings in clinical samples should be confirmed in more representative samples.