A framework for comparative cluster analysis of ensemble weather prediction data

Ensemble forecasting has become a standard means to obtain information about forecast uncertainties in meteorological centres across the world. The large datasets generated by ensemble prediction systems carry much information that is difficult to analyse manually – here, techniques from the field of artificial intelligence can be beneficial to aid the analysis. Cluster analysis is one commonly used (unsupervised machine learning) approach to automatically determine distinct scenarios in numerical weather forecasting ensembles, both in atmospheric research and operational forecasting. Typically, a cluster analysis focusses on a selected meteorological forecast variable, a specific region, and time (or a time window). The dimensionality of the data is reduced by techniques like principal component analysis, and a clustering algorithm – typically k-means – is applied to the reduced data set. Challenges with such an approach arise through the determined clusters often being sensitive to factors including the selected region, forecast variable, and algorithm parameters, and also through the employed algorithms often appearing as a “black box” to the user. In our work, we attempt to make the clustering process more transparent by providing a visual analysis framework to analyse the sensitivity of generated clusters with respect to various factors. The presented framework is coupled to the open-source meteorological ensemble visualization software Met.3D, allowing for interactive specification of clustering parameters and for interactive visual analysis, including 3-D elements. A case study using ensemble prediction data of sudden stratospheric warmings (SSWs) is presented, demonstrating how visualizing similarity between clusterings with different parameters can aid the interpretation of the data.