Use of the Cost Function of the Data Fusion Process to Estimate the Inconsistency Covariance Matrices

Vertical profiles of atmospheric variables are often obtained with the inversion of remote sensing observations performed by instruments operating on space-borne and airborne platforms, as well as from ground-based stations. When the same portion (or nearby portions) of atmosphere is observed more times by the same instrument or by different instruments the measurements can be combined in order to obtain a single vertical profile of improved quality with respect to that of the profiles retrieved from the single observation. The simultaneous retrieval from several observations is considered the most comprehensive way to combine different measurements of the same quantity, however, recently a new method, referred to as Complete Data Fusion, was proposed that, with simple implementation requirements, provides products of quality equivalent to that of the simultaneous retrieval products. The adoption of the Complete Data Fusion has highlighted a problem that we believe to be common to simultaneous retrieval and data fusion. The measurements that we wish to fuse often present some inconsistencies, due to representations on different vertical grids, not perfect time and space coincidence and different forward model errors, which may spoil the quality of the fused profile. In order to apply the Complete Data Fusion method to inconsistent measurements without a degradation of the product, it is necessary to add to the measurement covariance matrices of each fusing profile a covariance matrix that takes into account these inconsistencies. Therefore, the main problem in the fusion of inconsistent measurements is the realistic estimate of these inconsistency covariance matrices. We propose to use the value of the cost function that is minimized in the Complete Data Fusion to constrain the values of the inconsistency covariance matrices. To this purpose, we have analytically calculated the expected value of the cost function and its variance: the result is presented and is used in the proposed procedure. Modelling the inconsistency covariance matrices with one parameter, we tune the value of this parameter using the constraint provided by the requirement of the value of the cost function being equal to the expected value and using the variance to assign an error to the obtained value of the parameter. The use of Complete Data Fusion will be particularly useful for the analysis of the future atmospheric Sentinel missions of the Copernicus program. Indeed, the amount of data that will be available from these missions will pose technical challenges to many applications and the Complete Data Fusion can be used to reduce the number of products while maintaining the information content of the original datasets. For this reason, we test the proposed method on simulated measurements of ozone profiles obtained in the thermal infrared in the framework of the Sentinel 4 mission. In this context, the method is used to estimate the coincidence covariance matrices that take into account the variability of ozone when the fusing profiles refer to different times and space locations. The merits of this new procedure in the case of this specific data set are presented and discussed.