Search

Margin maximization in the hard-margin sense, proposed as feature elimination criterion by the MFE-LO method, is combined here with data radius utilization to further aim to lower generalization error, as several published bounds and bound-related formulations pertaining to lowering misclassification risk (or error) pertain to radius e.g. product of squared radius and weight vector squared norm. Additionally, we propose additional novel feature elimination criteria that, while instead being in the soft-margin sense, too can utilize data radius, utilizing previously published bound-related formulations for approaching radius for the soft-margin sense, whereby e.g. a focus was on the principle stated therein as "finding a bound whose minima are in a region with small leave-one-out values may be more important than its tightness". These additional criteria we propose combine radius utilization with a novel and computationally low-cost soft-margin light classifier retraining approach we devise named QP1; QP1 is the soft-margin alternative to the hard-margin LO. We correct an error in the MFE-LO description, find MFE-LO achieves the highest generalization accuracy among the previously published margin-based feature elimination (MFE) methods, discuss some limitations of MFE-LO, and find our novel methods herein outperform MFE-LO, attain lower test set classification error rate. On several datasets that each both have a large number of features and fall into the `large features few samples' dataset category, and on datasets with lower (low-to-intermediate) number of features, our novel methods give promising results. Especially, among our methods the tunable ones, that do not employ (the non-tunable) LO approach, can be tuned more aggressively in the future than herein, to aim to demonstrate for them even higher performance than herein., Comment: Incomplete but good, again. To Apr 28 version, made few misc text and notation improvements including typo corrections, probably mostly in Appendix, but probably best to read in whole again. New results for one of the datasets (Leukemia gene dataset)

Alzheimer's disease (AD) and mild cognitive impairment (MCI), continue to be widely studied. While there is no consensus on whether MCIs actually "convert" to AD, the more important question is not whether MCIs convert, but what is the best such definition. We focus on automatic prognostication, nominally using only a baseline image brain scan, of whether an MCI individual will convert to AD within a multi-year period following the initial clinical visit. This is in fact not a traditional supervised learning problem since, in ADNI, there are no definitive labeled examples of MCI conversion. Prior works have defined MCI subclasses based on whether or not clinical/cognitive scores such as CDR significantly change from baseline. There are concerns with these definitions, however, since e.g. most MCIs (and ADs) do not change from a baseline CDR=0.5, even while physiological changes may be occurring. These works ignore rich phenotypical information in an MCI patient's brain scan and labeled AD and Control examples, in defining conversion. We propose an innovative conversion definition, wherein an MCI patient is declared to be a converter if any of the patient's brain scans (at follow-up visits) are classified "AD" by an (accurately-designed) Control-AD classifier. This novel definition bootstraps the design of a second classifier, specifically trained to predict whether or not MCIs will convert. This second classifier thus predicts whether an AD-Control classifier will predict that a patient has AD. Our results demonstrate this new definition leads not only to much higher prognostic accuracy than by-CDR conversion, but also to subpopulations much more consistent with known AD brain region biomarkers. We also identify key prognostic region biomarkers, essential for accurately discriminating the converter and nonconverter groups.