Search

Due to their fast computational performance, neural networks (NNs) are nowadays commonly used in the context of remote sensing. The issue of performance is especially important in the context of big data and operational processing. Classical retrieval algorithms often use a radiative transfer model (RTM) as forward model with which an optimization algorithm can then solve the inverse problem of inferring the quantities of interest from the measured spectra. However, these RTMs are usually computationally very expensive and therefore replacing them by a NN is desirable to increase performance. But the application of NNs is not straightforward and there are at least two main approaches:1. NNs used as forward model, where a NN approximates the radiative transfer model and can thus replace it in the inversion algorithm2. NNs for solving the inverse problem, where a NN is trained to infer the atmospheric parameters from the measurement directlyThe first approach is more straightforward to apply. However, the inversion algorithm still faces many challenges, as the spectral fitting problem is generally ill-posed. Therefore, local minima are possible and the results often depend on the selection of the a-priori values for the retrieval parameters.For the second case, some of these issues can be avoided: no a-priori values are necessary, and as the training of the NN is performed globally, i.e. for many training samples at once, this approach is potentially less affected by local minima. However, due to the black-box nature of a NN, no indication about the quality of the results is available. In order to address this issue, novel methods like Bayesian neural networks (BNNs) or invertible neural networks (INNs) have been presented in recent years. This allows the characterization of the retrieved values by an estimate of uncertainty describing a range of values that are probable to produce the observed measurement. We apply and evaluate these new BNN and INN methods for the retrieval of cloud properties from TROPOMI in order to demonstrate their potential as operational algorithms for current (Sentinel-5P) and future (Sentinel-4 and Sentinel-5) Copernicus atmospheric composition missions.

The retrieval of atmospheric composition from space-based measurements, by e.g., Sentinel-5p TROPOMI, is strongly affected by radiative interferences with clouds. Dedicated cloud data products, typically retrieved from measurements by the same sounder, are therefore essential. Cloud information is used to filter data and as input to the modelling of atmospheric radiative transfer and the conversion of slant column densities into vertical column densities. The three main TROPOMI cloud retrieval algorithms are: (i) L2_CLOUD OCRA/ROCINN CAL (Optical Cloud Recognition Algorithm/Retrieval of Cloud Information using Neural Networks; Clouds-As-Layers), (ii) L2_CLOUD OCRA/ROCINN CRB (Clouds-as Reflecting Boundaries), and (iii) the S5P support product FRESCO-S (Fast Retrieval Scheme for Clouds from Oxygen absorption bands for Sentinel). The cloud variables provided by these products (radiometric cloud fraction, cloud (top) height, and cloud albedo/cloud optical thickness) are subsequently used in the retrieval of the TROPOMI trace gas products. The quality of cloud products and trace gas products is routinely assessed by the ESA/Copernicus Atmospheric Mission Performance Cluster (ATM-MPC) validation service, with ad hoc support from Sentinel-5p Validation Team (S5PVT) AO projects. Version upgrades have had a significant impact on the characteristics of S5P cloud data. The change of the wavelength window in the FRESCO product since version 1.4 (‘FRESCO-wide’) leads to a clear increase in the height of low clouds with a large impact on the tropospheric NO2 retrieval (van Geffen, 2022), and improving the validation results regarding the tropospheric and total NO2 column. The first upgrades of the ROCINN products (from v1 to v2.1-v2.3) led to an increase in correlation with CLOUDNET cloud height, but to a more negative bias for the low clouds, with ROCINN CRB cloud height even dropping below the CLOUDNET cloud base height on average. However, this effect seems alleviated with the latest upgrade to v2.4. The impact on the HCHO validation results is investigated but is less clear compared to the NO2 case. To resolve the discontinuities due to the processor version jumps, a full mission reprocessing is currently ongoing and largely carried out for the L2_CLOUD and FRESCO-S products. The reprocessed ROCINN data have a lower dispersion and higher correlation with respect to the CLOUDNET cloud heights. The bias of the L2_CLOUD OCRA/ROCINN CAL CTH becomes more negative, but that of L2_CLOUD OCRA/ROCINN CRB CH bias improves. Finally, we also discuss the impact of the FRESCO-S reprocessing on the validation results.

Clouds have a strong impact on satellite measurements of tropospheric trace gases in the ultraviolet, visible, and near-infrared spectral ranges from space. Therefore, trace gas retrievals rely on information on cloud fraction, cloud albedo, and cloud height from cloud products. In this study, the cloud parameters from different cloud retrieval algorithms for the Sentinel-5 Precursor (S5P) TROPOspheric Monitoring Instrument (TROPOMI) are compared: the Optical Cloud Recognition Algorithm (OCRA) a priori cloud fraction, the Retrieval Of Cloud Information using Neural Networks (ROCINN) CAL (Clouds-As-Layers) cloud fraction and cloud top and base height, the ROCINN CRB (Clouds-as-Reflecting-Boundaries) cloud fraction and cloud height, the Fast Retrieval Scheme for Clouds from the Oxygen A-band (FRESCO) cloud fraction, the interpolated FRESCO cloud height from the TROPOMI NO2 product, the cloud fraction from the NO2 fitting window, the O2–O2 cloud fraction and cloud height, the Mainz Iterative Cloud Retrieval Utilities (MICRU) cloud fraction, and the Visible Infrared Imaging Radiometer Suite (VIIRS) cloud fraction. Two different versions of the TROPOMI cloud products OCRA/ROCINN, FRESCO, and the TROPOMI NO2 product are included in the comparisons (processor version 1.x and 2.x). Overall, the cloud parameters retrieved by the different algorithms show qualitative consistency in version 1.x and good agreement in version 2.x with the exception of the VIIRS cloud fraction, which cannot be directly compared to the other data. Differences between the cloud retrievals are found especially for small cloud heights with a cloud fraction threshold of 0.2, i.e. clouds that are particularly relevant for tropospheric trace gas retrievals. The cloud fractions of the different version 2 cloud products primarily differ over snow- and ice-covered pixels and scenes with sun glint, for which only MICRU includes an explicit treatment. All cloud parameters show some systematic problems related to the across-track dependence, where larger values are found at the edges of the satellite view. The consistency between the cloud parameters from different algorithms depends strongly on how the data are filtered for the comparison, for example, what quality value is used or whether snow- and ice-covered pixels are excluded from the analysis. In summary, clear differences were found between the results of various algorithms, but these differences are reduced in the most recent versions of the cloud data.