Search

Forecast error growth as a function of lead time of atmospheric phenomena is caused by initial and model errors. When studying the initial error growth, it may turn out that small-scale phenomena, which contribute little to the forecast product, significantly affect the ability to predict this product. The question under investigation is whether omitting these atmospheric phenomena will improve the predictability of the resulting value. The topic is studied in the extended Lorenz (2005) system. This system shows that omitting small spatiotemporal scales that significantly affect prediction ability will reduce predictability more than modeling it. In other words, a system with model error (omitting phenomena) will not improve predictability. A hypothesis explaining and describing this behavior is developed, with the difference between systems (model error) produced at each time step seen as the error of the initial conditions. The resulting model error is then defined as the sum of the increments of the time evolution of the initial conditions so defined. The hypothesis is compared to the fit parameters that define the model error in certain approximations of the average forecast error growth. Parameters are interpreted in this context, and the approximations are used to estimate the errors described in the hypothesis. A method is proposed to distinguish increments of prediction error growth from small-spatiotemporal-scale phenomena and model errors. Results are presented for the error growth of the ECMWF system, where a 40 % reduction in model error between 1987 and 2011 is calculated based on the developed hypothesis, while over the same period the instability (error growth rate) of the system with respect to initial condition errors has grown. [ABSTRACT FROM AUTHOR]

Estimates of PM2.5 levels are crucial for monitoring air quality and studying the epidemiological impact of air quality on the population. Currently, the most precise measurements of PM2.5 are obtained from ground stations, resulting in limited spatial coverage. In this study, we consider satellite-based PM2.5 retrieval, which involves conversion of high-resolution satellite retrieval of Aerosol Optical Depth (AOD) into high-resolution PM2.5 retrieval. To improve the accuracy of the AOD to PM2.5 conversion, we employ the machine learning based post-process correction to correct the AOD-to-PM conversion ratio derived from Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) reanalysis model data. The post-process correction approach utilizes a fusion and downscaling of satellite observation and retrieval data, MERRA-2 reanalysis data, various high resolution geographical indicators, meteorological data and ground station observations for learning a predictor for the approximation error in the AOD to PM2.5 conversion ratio. The corrected conversion ratio is then applied to estimate PM2.5 levels given the high-resolution satellite AOD retrieval data derived from Sentinel-3 observations. Our model produces PM2.5 estimates with a spatial resolution of 100 meters at satellite overpass times. Additionally, we have incorporated an ensemble of neural networks to provide error envelopes for machine learning related uncertainty in the PM2.5 estimates. [ABSTRACT FROM AUTHOR]

This paper proposes a new method that combines checkpointing methods with error-controlled lossy compression for large-scale high-performance full-waveform inversion (FWI), an inverse problem commonly used in geophysical exploration. This combination can significantly reduce data movement, allowing a reduction in run time as well as peak memory. In the exascale computing era, frequent data transfer (e.g., memory bandwidth, PCIe bandwidth for GPUs, or network) is the performance bottleneck rather than the peak FLOPS of the processing unit. Like many other adjoint-based optimization problems, FWI is costly in terms of the number of floating-point operations, large memory footprint during backpropagation, and data transfer overheads. Past work for adjoint methods has developed checkpointing methods that reduce the peak memory requirements during backpropagation at the cost of additional floating-point computations. Combining this traditional checkpointing with error-controlled lossy compression, we explore the three-way tradeoff between memory, precision, and time to solution. We investigate how approximation errors introduced by lossy compression of the forward solution impact the objective function gradient and final inverted solution. Empirical results from these numerical experiments indicate that high lossy-compression rates (compression factors ranging up to 100) have a relatively minor impact on convergence rates and the quality of the final solution. [ABSTRACT FROM AUTHOR]

Satellite-based aerosol retrievals provide global spatially distributed estimates of atmospheric aerosol parameters that are commonly needed in applications such as estimation of atmospherically corrected satellite data products, climate modelling and air quality monitoring. However, a common feature of the conventional satellite aerosol retrievals is that they have reasonably low spatial resolution and poor accuracy caused by uncertainty in auxiliary model parameters, such as fixed aerosol model parameters, and the approximate forward radiative transfer models utilized to keep the computational complexity feasible. As a result, the improvement and reprocessing of the operational satellite data retrieval algorithms would become a tedious and computationally excessive problem. To overcome these problems, we have developed a machine-learning-based post-process correction approach to correct the existing operational satellite aerosol data products. Our approach combines the existing satellite retrieval data and a post-processing step where a machine learning algorithm is utilized to predict the approximation error in the conventional retrieval. With approximation error, we refer to the discrepancy between the true aerosol parameters and the ones retrieved using the satellite data. Our hypothesis is that the prediction of the approximation error with a finite training dataset is a less complex and easier task than the direct, fully learned machine-learning-based prediction in which the aerosol parameters are directly predicted given the satellite observations and measurement geometry. Our approach does not require reprocessing of the satellite retrieval products; it requires only a computationally fast machine-learning-based post-processing step of the existing retrieval product. Our approach is based on neural networks trained based on collocated satellite data and accurate ground-based Aerosol Robotic Network (AERONET) aerosol data. Based on our post-processing approach, we propose a post-process-corrected high-resolution Sentinel-3 Synergy aerosol product, which gives a spectral estimate of the aerosol optical depth at five different wavelengths with a high spatial resolution equivalent to the native resolution of the Sentinel-3 Level-1 data (300 m at nadir). With aerosol data from Sentinel-3A and 3B satellites, we demonstrate that our approach produces high-resolution aerosol data with clearly better accuracy than the operational Sentinel-3 Level-2 Synergy aerosol product, and it also results in slightly better accuracy than the conventional fully learned machine learning approach. We also demonstrate better generalization capabilities of the post-process correction approach over the fully learned approach. [ABSTRACT FROM AUTHOR]

Satellite-based aerosol retrievals provide global spatially distributed estimates of atmospheric aerosol parameters that are commonly needed in applications such as estimation of atmospherically corrected satellite data products, climate modeling and air quality monitoring. However, a common feature of the conventional satellite aerosol retrievals is that they have reasonably low spatial resolution and poor accuracy caused by uncertainty in auxiliary model parameters, such as fixed aerosol model parameters, and the approximate forward radiative transfer models utilized to keep the computational complexity feasible. As a result, the improvement and re-processing of the operational satellite data retrieval algorithms would become a tedious and computationally excessive problem. To overcome these problems, we have developed a machine learning-based post-process correction approach to correct the existing operational satellite aerosol data products. Our approach combines the existing satellite retrieval data and a post-processing step where a machine learning algorithm is utilized to predict the approximation error in the conventional retrieval. With approximation error we refer to the discrepancy between the true aerosol parameters and the ones retrieved using the satellite data. Our hypothesis is that the prediction of the approximation error with a finite training data set is a less complex and easier task than the direct fully learned machine learning based prediction in which the aerosol parameters are directly predicted given the satellite observations and measurement geometry. With our approach, there is no need to re-run the existing retrieval algorithms and only a computationally feasible post-processing step is needed. Our approach is based on neural networks trained based on collocated satellite data and accurate ground based AERONET aerosol data. Based on our post-processing approach, we propose a post-process corrected high resolution Sentinel-3 Synergy aerosol product, which gives a spectral estimate of the aerosol optical depth at five different wavelengths with a high spatial resolution equivalent to the native resolution of the Sentinel-3 level-1 data (300 meters at nadir). With aerosol data from Sentinel-3A and 3B satellites, we demonstrate that our approach produces high-resolution aerosol data with better accuracy than the operational Sentinel-3 level-2 Synergy aerosol product or a conventional fully learned machine learning approach. [ABSTRACT FROM AUTHOR]