Preprocessing of analytical profiles in the presence of homoscedastic or heteroscedastic noise

Analytical profiles are commonly normalized to the most intense peak or to constant sum prior to library searches or multivariate analysis. This work examines normalization procedures from a theoretical point of view and their effects on simulated and real data. It is found that while normalization of data with homoscedastic noise to constant sum only introduces the well-known bias toward negative peak correlations, normalization in the presence of heteroscedastic noise has a profound impact on the correlation structure. It follows that heteroscedastic noise in analytical signals should be transformed to homoscedasticity prior to the use of any normalization procedure. For cases where the standard deviation of the noise is proportional to the peak intensity, the log-transform followed by row centering is shown to be an effective cure against the spurious correlations induced by constant-sum normalization. For the case of heteroscedastic noise where the relative standard deviation of the noise is decreasing with increasing peak intensity, a modified Box-Cox power transform is developed for removing heteroscedasticity in data prior to normalization to constant sum. The power transform has several attractive features: (i) it can be tailored to cure different patterns of heteroscedastic noise, (ii) it enhances the information in small peaks compared to large ones, and, (iii) it conserves strong linear correlations between signals.