On associating significance levels with temporal changes in empirical orthogonal function analysis: a case study for ENSO

The availability of a large ensemble enables one to evaluate empirical orthogonal functions (EOFs)with respect to the ensemble without relying on temporal variability at all. Variability across theensemble at any given time is supposed to represent the most relevant probability distribution forclimate-related studies, and this distribution is presumably subject to temporal changes in thepresence of time-dependent forcing. Such changes may be observable in spatial patterns ofensemble-based EOFs and associated eigenvalues. Unfortunately, estimates of these changescome with a considerable error due to the finite size of the ensemble, so that associating asignificance level with the presence of a change (with respect to a null hypothesis about theabsence of any change) should be the first step of analyzing the time evolution.It turns out, however, that the conditions for the applicability of usual hypothesis tests aboutstationarity are not satisfied for the above-mentioned quantities. What proves to be feasible is toestimate an upper bound on the significance level for nonstationarity. This means that the truesignificance level would ideally be lower or equal to what is estimated, which would preventunjustified confidence in the detection of nonstationarity (i.e., falsely rejecting the null hypothesiscould not become more probable than claimed). Most importantly, one would avoid seriouslyoverconfident conclusions about the sign of the change in this way. Notwithstanding, the estimatefor the upper bound on the significance level is also affected by the finite number of the ensemblemembers. It nevertheless becomes more and more precise for increasing ensemble size and mayserve as a first guidance for currently available ensemble sizes.The details of the estimation are presented in the example of the EOF-based analysis of the ElNiño–Southern Oscillation (ENSO) as it appears in the historical and RCP8.5 simulations of the MaxPlanck Institute Grand Ensemble. A comparison bet