Dimension reduction for regression : theoretical and methodological developments

This thesis has two themes: (1) the predictive potential of principal components in regression, and (2) methodological developments in sufficient dimension reduction. For the first theme, several research papers have established a number of results showing that, under some uniformity assumptions, higher-ranking principal components of a predictor vector tend, across a range of datasets, to have greater squared correlation with a response variable than lower-ranking ones. This is despite the procedure being unsupervised. This thesis reviews these results and greatly extends them by showing that analogues hold in the setting where nonlinear principal component analysis with general predictors is applied. For the second theme, research in the past 10 years has led to a measure-theoretic framework for sufficient dimension reduction, inspired by the measure-theoretic formulation of sufficient statistics, which permits nonlinear reductions. This thesis extends this framework to allow for some of the predictors to be categorical. A new estimator, partial generalised sliced inverse regression, is proposed and its properties and effectiveness are explored.