Search

Decoding, i.e. prediction from brain images or signals, calls for empirical evaluation of its predictive power. Such evaluation is achieved via cross-validation, a method also used to tune decoders' hyper-parameters. This paper is a review on cross-validation procedures for decoding in neuroimaging. It includes a didactic overview of the relevant theoretical considerations. Practical aspects are highlighted with an extensive empirical study of the common decoders in within- and across-subject predictions, on multiple datasets -anatomical and functional MRI and MEG- and simulations. Theory and experiments outline that the popular "leave-one-out" strategy leads to unstable and biased estimates, and a repeated random splits method should be preferred. Experiments outline the large error bars of cross-validation in neuroimaging settings: typical confidence intervals of 10%. Nested cross-validation can tune decoders' parameters while avoiding circularity bias. However we find that it can be favorable to use sane defaults, in particular for non-sparse decoders., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Amnestic mild cognitive impairment (aMCI) is a syndrome widely considered to be prodromal Alzheimer's disease. Accurate diagnosis of aMCI would enable earlier treatment, and could thus help minimize the prevalence of Alzheimer's disease. The aim of the present study was to evaluate a magnetic resonance imaging-based automated classification schema for identifying aMCI. This was carried out in a sample of community-dwelling adults aged 70-90 years old: 79 with a clinical diagnosis of aMCI and 204 who were cognitively normal. Our schema was novel in using measures of both spatial atrophy, derived from T1-weighted images, and white matter alterations, assessed with diffusion tensor imaging (DTI) tract-based spatial statistics (TBSS). Subcortical volumetric features were extracted using a FreeSurfer-initialized Large Deformation Diffeomorphic Metric Mapping (FS+LDDMM) segmentation approach, and fractional anisotropy (FA) values obtained for white matter regions of interest. Features were ranked by their ability to discriminate between aMCI and normal cognition, and a support vector machine (SVM) selected an optimal feature subset that was used to train SVM classifiers. As evaluated via 10-fold cross-validation, the classification performance characteristics achieved by our schema were: accuracy, 71.09%; sensitivity, 51.96%; specificity, 78.40%; and area under the curve, 0.7003. Additionally, we identified numerous socio-demographic, lifestyle, health and other factors potentially implicated in the misclassification of individuals by our schema and those previously used by others. Given its high level of performance, our classification schema could facilitate the early detection of aMCI in community-dwelling elderly adults., (Copyright © 2011 Elsevier Inc. All rights reserved.)