Convergence of forecast distributions in a 100,000‐member idealised convective‐scale ensemble

Many operational weather services use ensembles of forecasts to generate probabilistic predictions. Computational costs generally limit the size of the ensemble to fewer than 100 members, although the large number of degrees of freedom in the forecast model would suggest that a vastly larger ensemble would be required to represent the forecast probability distribution accurately. In this study, we use a computationally efficient idealised model that replicates key properties of the dynamics and statistics of cumulus convection to identify how the sampling uncertainty of statistical quantities converges with ensemble size. Convergence is quantified by computing the width of the 95% confidence interval of the sampling distribution of random variables, using bootstrapping on the ensemble distributions at individual time and grid points. Using ensemble sizes of up to 100,000 members, it was found that for all computed distribution properties, including mean, variance, skew, kurtosis, and several quantiles, the sampling uncertainty scaled as n−1/2 for sufficiently large ensemble size n. This behaviour is expected from the Central Limit Theorem, which further predicts that the magnitude of the uncertainty depends on the distribution shape, with a large uncertainty for statistics that depend on rare events. This prediction was also confirmed, with the additional observation that such statistics also required larger ensemble sizes before entering the asymptotic regime. By considering two methods for evaluating asymptotic behaviour in small ensembles, we show that the large‐n theory can be applied usefully for some forecast quantities even for the ensemble sizes in operational use today.
An idealised ensemble that replicates key properties of the dynamics and statistics of cumulus convection is used to identify how sampling uncertainty of statistical quantities converges with ensemble size. A universal asymptotic scaling for this convergence was found, which was dependent on the statistic and the distribution shape, with largest uncertainty for statistics that depend on rare events. This is demonstrated in the figure below for a Gaussian distributed model variable, where the sampling uncertainty (y‐axis) for 5 quantiles (red lines) indicates that after a certain ensemble size, it begins converging asymptotically (grey lines), and the more extreme the quantile, the more members it requires for this to be the case.
Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659
Klaus Tschira Stiftung http://dx.doi.org/10.13039/501100007316