Search

Estimates of PM2.5 levels are crucial for monitoring air quality and studying the epidemiological impact of air quality on the population. Currently, the most precise measurements of PM2.5 are obtained from ground stations, resulting in limited spatial coverage. In this study, we consider satellite-based PM2.5 retrieval, which involves conversion of high-resolution satellite retrieval of Aerosol Optical Depth (AOD) into high-resolution PM2.5 retrieval. To improve the accuracy of the AOD to PM2.5 conversion, we employ the machine learning based post-process correction to correct the AOD-to-PM conversion ratio derived from Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) reanalysis model data. The post-process correction approach utilizes a fusion and downscaling of satellite observation and retrieval data, MERRA-2 reanalysis data, various high resolution geographical indicators, meteorological data and ground station observations for learning a predictor for the approximation error in the AOD to PM2.5 conversion ratio. The corrected conversion ratio is then applied to estimate PM2.5 levels given the high-resolution satellite AOD retrieval data derived from Sentinel-3 observations. Our model produces PM2.5 estimates with a spatial resolution of 100 meters at satellite overpass times. Additionally, we have incorporated an ensemble of neural networks to provide error envelopes for machine learning related uncertainty in the PM2.5 estimates. [ABSTRACT FROM AUTHOR]

This paper proposes a new method that combines checkpointing methods with error-controlled lossy compression for large-scale high-performance full-waveform inversion (FWI), an inverse problem commonly used in geophysical exploration. This combination can significantly reduce data movement, allowing a reduction in run time as well as peak memory. In the exascale computing era, frequent data transfer (e.g., memory bandwidth, PCIe bandwidth for GPUs, or network) is the performance bottleneck rather than the peak FLOPS of the processing unit. Like many other adjoint-based optimization problems, FWI is costly in terms of the number of floating-point operations, large memory footprint during backpropagation, and data transfer overheads. Past work for adjoint methods has developed checkpointing methods that reduce the peak memory requirements during backpropagation at the cost of additional floating-point computations. Combining this traditional checkpointing with error-controlled lossy compression, we explore the three-way tradeoff between memory, precision, and time to solution. We investigate how approximation errors introduced by lossy compression of the forward solution impact the objective function gradient and final inverted solution. Empirical results from these numerical experiments indicate that high lossy-compression rates (compression factors ranging up to 100) have a relatively minor impact on convergence rates and the quality of the final solution. [ABSTRACT FROM AUTHOR]

Satellite-based aerosol retrievals provide global spatially distributed estimates of atmospheric aerosol parameters that are commonly needed in applications such as estimation of atmospherically corrected satellite data products, climate modelling and air quality monitoring. However, a common feature of the conventional satellite aerosol retrievals is that they have reasonably low spatial resolution and poor accuracy caused by uncertainty in auxiliary model parameters, such as fixed aerosol model parameters, and the approximate forward radiative transfer models utilized to keep the computational complexity feasible. As a result, the improvement and reprocessing of the operational satellite data retrieval algorithms would become a tedious and computationally excessive problem. To overcome these problems, we have developed a machine-learning-based post-process correction approach to correct the existing operational satellite aerosol data products. Our approach combines the existing satellite retrieval data and a post-processing step where a machine learning algorithm is utilized to predict the approximation error in the conventional retrieval. With approximation error, we refer to the discrepancy between the true aerosol parameters and the ones retrieved using the satellite data. Our hypothesis is that the prediction of the approximation error with a finite training dataset is a less complex and easier task than the direct, fully learned machine-learning-based prediction in which the aerosol parameters are directly predicted given the satellite observations and measurement geometry. Our approach does not require reprocessing of the satellite retrieval products; it requires only a computationally fast machine-learning-based post-processing step of the existing retrieval product. Our approach is based on neural networks trained based on collocated satellite data and accurate ground-based Aerosol Robotic Network (AERONET) aerosol data. Based on our post-processing approach, we propose a post-process-corrected high-resolution Sentinel-3 Synergy aerosol product, which gives a spectral estimate of the aerosol optical depth at five different wavelengths with a high spatial resolution equivalent to the native resolution of the Sentinel-3 Level-1 data (300 m at nadir). With aerosol data from Sentinel-3A and 3B satellites, we demonstrate that our approach produces high-resolution aerosol data with clearly better accuracy than the operational Sentinel-3 Level-2 Synergy aerosol product, and it also results in slightly better accuracy than the conventional fully learned machine learning approach. We also demonstrate better generalization capabilities of the post-process correction approach over the fully learned approach. [ABSTRACT FROM AUTHOR]

Satellite-based aerosol retrievals provide global spatially distributed estimates of atmospheric aerosol parameters that are commonly needed in applications such as estimation of atmospherically corrected satellite data products, climate modeling and air quality monitoring. However, a common feature of the conventional satellite aerosol retrievals is that they have reasonably low spatial resolution and poor accuracy caused by uncertainty in auxiliary model parameters, such as fixed aerosol model parameters, and the approximate forward radiative transfer models utilized to keep the computational complexity feasible. As a result, the improvement and re-processing of the operational satellite data retrieval algorithms would become a tedious and computationally excessive problem. To overcome these problems, we have developed a machine learning-based post-process correction approach to correct the existing operational satellite aerosol data products. Our approach combines the existing satellite retrieval data and a post-processing step where a machine learning algorithm is utilized to predict the approximation error in the conventional retrieval. With approximation error we refer to the discrepancy between the true aerosol parameters and the ones retrieved using the satellite data. Our hypothesis is that the prediction of the approximation error with a finite training data set is a less complex and easier task than the direct fully learned machine learning based prediction in which the aerosol parameters are directly predicted given the satellite observations and measurement geometry. With our approach, there is no need to re-run the existing retrieval algorithms and only a computationally feasible post-processing step is needed. Our approach is based on neural networks trained based on collocated satellite data and accurate ground based AERONET aerosol data. Based on our post-processing approach, we propose a post-process corrected high resolution Sentinel-3 Synergy aerosol product, which gives a spectral estimate of the aerosol optical depth at five different wavelengths with a high spatial resolution equivalent to the native resolution of the Sentinel-3 level-1 data (300 meters at nadir). With aerosol data from Sentinel-3A and 3B satellites, we demonstrate that our approach produces high-resolution aerosol data with better accuracy than the operational Sentinel-3 level-2 Synergy aerosol product or a conventional fully learned machine learning approach. [ABSTRACT FROM AUTHOR]

Satellite-based aerosol retrievals provide a timely view of atmospheric aerosol properties, having a crucial role in the subsequent estimation of air quality indicators, atmospherically corrected satellite data products, and climate applications. However, current aerosol data products based on satellite data often have relatively large biases compared to accurate ground-based measurements and distinct uncertainty levels associated with them. These biases and uncertainties are often caused by oversimplified assumptions and approximations used in the retrieval algorithms due to unknown surface reflectance or fixed aerosol models. Moreover, the retrieval algorithms do not usually take advantage of all the possible observational data collected by the satellite instruments and may, for example, leave some spectral bands unused. The improvement and the re-processing of the past and current operational satellite data retrieval algorithms would become tedious and computationally expensive. To overcome this burden, we have developed a model-enforced post-process correction approach to correct the existing operational satellite aerosol data products. Our approach combines the existing satellite aerosol retrievals and a post-processing step carried out with a machine-learning-based correction model for the approximation error in the retrieval. The developed approach allows for the utilization of auxiliary data sources, such as meteorological information, or additional observations such as spectral bands unused by the original retrieval algorithm. The post-process correction model can learn to correct for the biases and uncertainties in the original retrieval algorithms. As the correction is carried out as a post-processing step, it allows for computationally efficient re-processing of existing satellite aerosol datasets without fully re-processing the much larger original radiance data. We demonstrate with over-land aerosol optical depth (AOD) and Ångström exponent (AE) data from the Moderate Imaging Spectroradiometer (MODIS) of the Aqua satellite that our approach can significantly improve the accuracy of the satellite aerosol data products and reduce the associated uncertainties. For instance, in our evaluation, the number of AOD samples within the MODIS Dark Target expected error envelope increased from 63 % to 85 % when the post-process correction was applied. In addition to method description and accuracy results, we also give recommendations for validating machine-learning-based satellite data products. [ABSTRACT FROM AUTHOR]

In this work, we apply a principal component analysis (PCA)-based approach combined with lookup tables (LUTs) of corrections to accelerate the Vector Linearized Discrete Ordinate Radiative Transfer (VLIDORT) model used in the retrieval of ozone profiles from backscattered ultraviolet (UV) measurements by the Ozone Monitoring Instrument (OMI). The spectral binning scheme, which determines the accuracy and efficiency of the PCA-RT performance, is thoroughly optimized over the spectral range 265 to 360 nm with the assumption of a Rayleigh-scattering atmosphere above a Lambertian surface. The high level of accuracy (∼ 0.03 %) is achieved from fast-PCA calculations of full radiances. In this approach, computationally expensive full multiple scattering (MS) calculations are limited to a small set of PCA-derived optical states, while fast single scattering and two-stream MS calculations are performed, for every spectral point. The number of calls to the full MS model is only 51 in the application to OMI ozone profile retrievals with the fitting window of 270–330 nm where the RT model should be called at fine intervals (∼ 0.03 nm with ∼ 2000 wavelengths) to simulate OMI measurements (spectral resolution: 0.4–0.6 nm). LUT corrections are implemented to accelerate the online RT model due to the reduction of the number of streams (discrete ordinates) from 8 to 4, while improving the accuracy at the level attainable from simulations using a vector model with 12 streams and 72 layers. Overall, we speed up our OMI retrieval by a factor of 3.3 over the previous version, which has already been significantly sped up over line-by-line calculations due to various RT approximations. Improved treatments for RT approximation errors using LUT corrections improve spectral fitting (2 %–5 %) and hence retrieval errors, especially for tropospheric ozone by up to ∼ 10 %; the remaining errors due to the forward model errors are within 5 % in the troposphere and 3 % in the stratosphere. [ABSTRACT FROM AUTHOR]

We consider the problem of inferring the basal sliding coefficient field for an uncertain Stokes ice sheet forward model from synthetic surface velocity measurements. The uncertainty in the forward model stems from unknown (or uncertain) auxiliary parameters (e.g., rheology parameters). This inverse problem is posed within the Bayesian framework, which provides a systematic means of quantifying uncertainty in the solution. To account for the associated model uncertainty (error), we employ the Bayesian approximation error (BAE) approach to approximately premarginalize simultaneously over both the noise in measurements and uncertainty in the forward model. We also carry out approximative posterior uncertainty quantification based on a linearization of the parameter-to-observable map centered at the maximum a posteriori (MAP) basal sliding coefficient estimate, i.e., by taking the Laplace approximation. The MAP estimate is found by minimizing the negative log posterior using an inexact Newton conjugate gradient method. The gradient and Hessian actions to vectors are efficiently computed using adjoints. Sampling from the approximate covariance is made tractable by invoking a low-rank approximation of the data misfit component of the Hessian. We study the performance of the BAE approach in the context of three numerical examples in two and three dimensions. For each example, the basal sliding coefficient field is the parameter of primary interest which we seek to infer, and the rheology parameters (e.g., the flow rate factor or the Glen's flow law exponent coefficient field) represent so-called nuisance (secondary uncertain) parameters. Our results indicate that accounting for model uncertainty stemming from the presence of nuisance parameters is crucial. Namely our findings suggest that using nominal values for these parameters, as is often done in practice, without taking into account the resulting modeling error, can lead to overconfident and heavily biased results. We also show that the BAE approach can be used to account for the additional model uncertainty at no additional cost at the online stage. [ABSTRACT FROM AUTHOR]

This paper proposes a new method that combines check -pointing methods with error-controlled lossy compression for large-scale high-performance Full-Waveform Inversion (FWI), an inverse problem commonly used in geophysical exploration. This combination can signif- icantly reduce data movement, allowing a reduction in run time as well as peak memory. In the Exascale computing era, frequent data transfer (e.g., memory bandwidth, PCIe bandwidth for GPUs, or network) is the performance bottleneck rather than the peak FLOPS of the processing unit. Like many other adjoint-based optimization problems, FWI is costly in terms of the number of floating-point operations, large memory foot- print during backpropagation, and data transfer overheads. Past work for adjoint methods has developed checkpointing methods that reduce the peak memory requirements during backpropagation at the cost of additional floating-point computations. Combining this traditional checkpointing with error-controlled lossy compression, we explore the three-way tradeoff between memory, precision, and time to solution. We investigate how approximation errors introduced by lossy compression of the forward solution impact the objective function gradient and final inverted solution. Empirical results from these numerical experiments indicate that high lossy-compression rates (compression factors ranging up to 100) have a relatively minor impact on convergence rates and the quality of the final solution. [ABSTRACT FROM AUTHOR]

Satellite-based aerosol retrievals provide a timely global view of atmospheric aerosol properties for air quality, atmospheric characterization, and correction of satellite data products and climate applications. Current aerosol data products based on satellite data, however, often have relatively large biases relative to accurate ground-based measurements and distinct levels of uncertainty associated with them. These biases and uncertainties are often caused by oversimplified assumptions and approximations used in the retrieval algorithms due to unknown surface reflectance or fixed aerosol models. Moreover, the retrieval algorithms do not usually take advantage of all the possible observational data collected by the satellite instruments and may, for example, leave some spectral bands unused. The improvement and the re-processing of the past and current operational satellite data retrieval algorithms would become a tedious and computationally expensive task. To overcome this burden, we have developed a model enforced post-process correction approach that can be used to correct the existing and operational satellite aerosol data products. Our approach combines the existing satellite aerosol retrievals and a post-processing step carried out with a machine learning based correction model for the approximation error in the retrieval. The developed approach allows for the utilization of auxiliary data sources, such as meteorological information, or additional observations such as spectral bands unused by the original retrieval algorithm. The post-process correction model can learn to correct for the biases and uncertainties in the original retrieval algorithms. As the correction is carried out as a post-processing step, it allows for computationally efficient re-processing of existing satellite aerosol datasets with no need to fully reprocess the much larger original radiance data. We demonstrate with over land aerosol optical depth (AOD) and Angstrom exponent (AE) data from the Moderate Imaging Spectroradiometer (MODIS) of Aqua satellite that our approach can significantly improve the accuracy of the satellite aerosol data products and reduce the associated uncertainties. We also give recommendations for the validation of satellite data products that are constructed using machine learning based models. [ABSTRACT FROM AUTHOR]

In this work, we apply a principal component analysis (PCA)-based approach combined with look-up tables (LUTs) of corrections to accelerate the VLIDORT radiative transfer (RT) model used in the retrieval of ozone profiles from backscattered ultraviolet (UV) measurements by the Ozone Monitoring Instrument (OMI). The spectral binning scheme, which determines the accuracy and efficiency of the PCA–RT performance, is thoroughly optimized over the spectral range 265 to 360 nm with the assumption of a Rayleigh-scattering atmosphere above a Lambertian surface. The high level of accuracy (~ 0.03 %) is achieved from fast-PCA calculations of full radiances. In this approach, computationally expensive full multiple scattering (MS) calculations are limited to a small set of PCA-derived optical states, while fast single scattering and 2-stream multiple scattering calculations are performed, for every spectral point. The number of calls to the full MS model is only 51 in the application to OMI ozone profile retrievals with the fitting window of 270-330 nm where the RT model should be called at fine intervals (~ 0.03 nm with ~ 2000 wavelengths) to simulate OMI native measurements at 229 wavelengths (spectral resolution: 0.4-0.6 nm). We also developed a Look Up Table (LUT) to correct RT approximations performed using a scalar RT model with 4 streams (discrete ordinates) and 24 layers, thereby achieving the accuracy at the level attainable from simulations with a vector model with 12 streams and 72 layers; this speeds up the RT calculations by more than 2 orders of magnitude when ignoring other overhead. Overall, we speed up our OMI retrieval by a factor of 3.3 over the previous version, which has already been significantly sped up over line-by-line calculations due to various RT approximations. Improved treatments for RT approximation errors using LUT corrections improve spectral fitting (2-5 %) and hence retrieval errors, especially for tropospheric ozone by up to ~ 10 %; the remaining errors due to the forward model errors are within 5 % in the troposphere and 3 % in the stratosphere. [ABSTRACT FROM AUTHOR]

Fibre optic distributed temperature sensing (DTS) is widely applied in Earth sciences. Many applications require a spatial resolution higher than that provided by the DTS instrument. Measurements at these higher resolutions can be achieved with a fibre optic cable helically wrapped on a cylinder. The effect of the probe construction, such as its material, shape, and diameter, on the performance has been poorly understood. In this article, we study data sets obtained from a laboratory experiment using different cable and construction diameters, and three field experiments using different construction characteristics. This study shows that the construction material, shape, diameter, and cable attachment method can have a significant influence on DTS temperature measurements. We present a qualitative and quantitative approximation of errors introduced through the choice of auxiliary construction, influence of solar radiation, coil diameter, and cable attachment method. Our results provide insight into factors that influence DTS measurements, and we present a number of solutions to minimize these errors. These practical considerations allow designers of future DTS measurement set-ups to improve their environmental temperature measurements. [ABSTRACT FROM AUTHOR]