Search

International audience; Uncertainty quantification (UQ) plays a crucial role in data assimilation (DA) since it impacts both the quality of the reconstruction and near-future forecast. However, traditional UQ approaches are often limited in their ability to handle complex datasets and may have a large computational cost. In this paper, we present a new ensemble-based approach to extend the 4DVarNet framework, an end-to-end deep learning scheme backboned on variational DA used to estimate the mean of the state along a given DA window. We use conditional 4DVarNet simulations compliant with the available observations to estimate the 4DVarNet probability density function. Our approach enables to combine both the efficiency of 4DVarNet in terms of computational cost and validation performance with a fast and memorysaving Monte-Carlo based post-processing of the reconstruction, leading to the so-called En4DVarNet estimation of the state pdf. We demonstrate our approach in a case study involving the sea surface height: 4DVarNet is pretrained on an idealized Observation System Simulation Experiment (OSSE), then used on real-world dataset (OSE). The sampling of independent realizations of the state is made among the catalogue of model-based data used during training. To illustrate our approach, we use a nadir altimeter constellation in January 2017 and show how the uncertainties retrieved by combining 4DVarNet with the statistical properties of the training dataset lead to a relevant information providing in most cases a confidence interval compliant with the Cryosat-2 nadir alongtrack dataset kept for validation. Impact Statement This research paper discusses the extension of the 4DVarNet framework to ensemble-based approaches. It paves the way to uncertainty quantification and enables to initiate the discussion of how to retrieve posterior pdf from such deep learning framework. This paper also involves pieces of previous work initiated inside the 4DVarNet team of developers in the framework of AI Chair Oceanix. It is meant to help the readers in understanding this framework from a broader point of view while diving into some new and ongoing developments.

This report includes recommendations for the planification of in situ experiments aimed to reconstruct fine-scale ocean currents (~20 km), such as those that will be conducted to validate SWOT satellite observations.

Algorithms using trainable components, data assimilation and other inverse techniques are becoming unavoidable tools for designing high-level data products leveraging oceanic observations for a wide range of applications. These fast and promising advances are speeding up the pace of technological and scientific progress in our field. However, because of the variety of approaches, sometimes emanating from different communities and scattered research groups, keeping track of progress is becoming more difficult. This is why fostering community-driven, intercomparison frameworks would be of great benefit for creating the next-generation high level ocean data products and to the oceanographic community in general. Our project BOOST-SWOT ( “Building Of Ocean Surface Topography maps with SWOT”) has led to a number of methodological developments for sea surface height cartography and for the preprocessing of the SWOT ocean data. This includes: inverse techniques for filtering SWOT observational noise, algorithms for mapping SSH with data assimilation techniques applied to simple dynamical models, and approaches for optimally accounting for high-frequency internal wave dynamics in mapping algorithms. In order to assess the potential and the limitations of the above algorithms, we have implemented a series of collaborative data challenges that have been shared across different research groups. These collaborative data challenges have been implemented on the basis of observational data, state-of-the-art numerical model simulations and evaluation metrics. They are meant to provide well-posed benchmarks for intercomparing inversion methods for the mapping of altimeter data. In this presentation, we will take a look back at the BOOST-SWOT results and at the benefits and issues raised during the challenges. On the basis of the BOOST-SWOT experience, we will try to describe what are, from our perspective, the key elements for a fruitful collaboration across different research groups through open data challenges: easy to access open source codes and data, co-designed evaluation metrics and blind evaluation references. We will conclude on what could be the first steps towards developing and maintaining systematic collaboration frameworks for our community in the future.

The reconstruction of sea surface currents is a key challenge in spatial oceanography. We recently proposed the so-called 4DVarNN algorithm, a generic end-to-end deep learning scheme for inverse problems using a variational formulation. Based on Observing System Simulation Experiment (OSSE) involving high-resolution numerical simulations in the Gulf Stream region, the preliminary applications of the 4DVarNN algorithm using an LSTM-based parametrization of the solver have shown promising results. We propose here to present the recent evolutions of the 4DVarNN framework applied to spatio-temporal interpolations of satellite-derived datasets. First, 4DVarNN embeds a variational formalism which is a natural framework for exploiting multi-tracer synergies (e.g. SSH and SST) in the reconstruction of altimetric fields benefiting from high-resolution satellite products. Using a similar OSSE configuration, we demonstrate how the use of SST may help for the identification of ocean fronts resulting in a better reconstruction of the SSH. Second, the variational formulation also enables to design optimal monitoring and sampling strategies to retrieve the best reconstruction of the submesoscale processes. Last, from an operational perspective, a new version of the code able to deal with datasets scaling up to an ocean basin has been recently distributed. Training the model involves a new strategy based on iterating the entire datasets in small batches. This code is open-source and enables its future use for both design and participation in ocean data challenges.

International audience; This paper addresses physics-informed deep learning schemes for satellite ocean remote sensing data. Such observation datasets are characterized by the irregular space-time sampling of the ocean surface due to sensors' characteristics and satellite orbits. With a focus on satellite altimetry, we show that end-to-end learning schemes based on variational formulations provide new means to explore and exploit such observation datasets. Through Observing System Simulation Experiments (OSSE) using numerical ocean simulations and real nadir and wide-swath altimeter sampling patterns, we demonstrate their relevance w.r.t. state-of-the-art and operational methods for space-time interpolation and short-term forecasting issues. We also stress and discuss how they could contribute to the design and calibration of ocean observing systems.

Additional file 1.

Geostatistical methods are commonly used in air quality mapping. In France, since 2003, the PREV'AIR system has been broadcasting daily three-days forecasts as well as national and European air quality maps. They are built by a kriging of the observations using the deterministic chemistry-transport model CHIMERE outputs as an external drift. However, many issues or developments remain unresolved.The first part of the thesis aims at improving the local, national and european maps of the main regulated pollutants. Regarding the analysis (mapping of the past observations), we get back to the question of the use of explanatory variables in kriging. What are the best options for a covariate-based modelling of the underlying non stationarity, that also enables downscaling the model outputs? The statistical modelling for the deterministic component of the stochastic process is investigated, as well as the modelling for the covariance of the residual. A focus is also made on the spatial sampling of the monitoring network. The use of PM10 observations to map PM2.5, whose monitoring network is less dense, is studied and a thought is also given to the estimation in far-off extrapolation. Last, we discuss how to extend these methods to the prediction problem (mapping of the future, where no observations are available). Therefore, we could improve the PREV'AIR system which currently dissociates the time and the spatial components of the underlying process. A review of spatio-temporal methods is carried out, and some of them are evaluated.In a second part, some pragmatic though justified approximations are presented to deal with regulatory requirements. At both local and national levels, how the analyses can be used to deduce probability maps of exceedances of the regulatory thresholds? Along this line, we also come back to the question of the spatial representativeness of the monitoring stations. Last, the pragmatic approximations are confronted to non-linear estimations, theoretically more convenient to deal with non-linear functions of the concentrations. We engage a discussion to show the need of considering the appropriate estimation method to compute the surface exceeding the limit value.; Les méthodes géostatistiques sont couramment utilisées pour la cartographie de la qualité de l'air. En France, le système PREV'AIR diffuse depuis 2003 des prévisions quotidiennes à trois jours ainsi que des cartographies de qualité de l'air à l'échelle nationale et européenne. Ces cartographies sont construites par krigeage des observations en utilisant les sorties du modèle déterministe de chimie-transport CHIMERE en dérive externe. Dans ce domaine, de nombreuses questions et développements restent en suspens.La première partie de la thèse vise à améliorer les cartographies locales, nationales et européennes des principaux polluants réglementaires. En mode analyse (réalisation de cartographies à partir des observations passées, et donc connues), on revient sur l'utilisation des variables explicatives dans le krigeage : il s'agit de voir comment ces variables peuvent être intégrées à l'estimation pour la modélisation des non stationnarités et l’élaboration de cartographies de plus haute résolution spatiale que les simulations du modèle de chimie-transport. La réflexion porte sur la modélisation de la composante déterministe du processus stochastique, mais aussi sur la covariance du résidu. Une attention particulière est portée au réseau de mesures. L'utilisation des observations PM10 pour cartographier les PM2.5, dont le réseau de mesures est moins dense, est explorée. L'estimation dans les zones peu informées est également examinée. Enfin, une réflexion est menée sur l'extension de ces méthodes au mode prévision, afin d'améliorer le système PREV'AIR, qui dissocie les composantes temporelles et spatiales de la prévision. Après une revue des méthodes spatio-temporelles d'estimation, différentes méthodes sont évaluées.Dans la deuxième partie sont présentées des approximations pragmatiques permettant d'exploiter au mieux les cartographies produites quotidiennement ou ponctuellement afin de satisfaire aux exigencesréglementaires. A l'échelle nationale, l'exploitation des sorties des cartes analysées (horaires, journalières) de PREV'AIR permet de déduire des cartes de probabilités de dépassement des seuils réglementaires. Ces travaux amènent à reconsidérer la question de la représentativité spatiale des stations de mesures. Enfin, on insiste sur la nécessité d'utiliser des méthodes d’estimation appropriées pour le calcul des surfaces en dépassement de valeurs réglementaires.

International audience; For large N and when no variables is predominant over the others, the central limit theorem (CLT) shall apply to the sum of random variables with negative values reset to zero. The parameters of the normal distribution are simply obtained by computing the expected value and the variance of each left rectified distributions. But for small N, the distribution of the sum is clearly not Gaussian and can present several modes and a strong skewness. In this paper, a way of computing the probability density function of the sum of N independent rectified Gaussian variables is presented, so that the calculation issues raised by the convolution product is solved. Some numerical examples are given and the validity of this approach is assessed through a comparison with a Monte-Carlo approach and an application to the PAH’s (Polycyclic Aromatic Hydrocarbon) batch filters measurements is provided.; Pour N grand, le thèorème de la limite centrale s’applique si l’on cherche à sommer des variables aléatoires gaussiennes rectifiées, i.e. dont les valeurs négatives sont remises à zéro. Les paramètres de la gaussienne sont obtenus en sommant les espérances et les variances de chaque gaussienne censurée. Pour N petit, la distribution de la somme n’est évidemment pas gaussienne : elle peut présenter plusieurs modes et une forte asymétrie. Dans ce papier, le calcul de la densité de probabilité associée à la somme de variables aléatoires gaussiennes rectifiées indépendantes est présenté. Il est obtenu classiquement à partir d’un produit de convolution mais il présente des complications calculatoires qui sont détaillées et résolues. Une comparaison avec des simulations Monte-Carlo est fournie pour valider les développements. Enfin, une application à des données de mesures par filtres de HAP (Hydrocarbure Aromatique Polycyclique) est également présentée.

International audience; The national PREV'AIR system (www2.prevair.org) delivers daily analyses and forecasts of different atmospheric pollutant concentrations over Europe and France. Analysed maps of the previous day situation (D-1) are produced by combining in-situ measurement data with output data from the chemistry-transport model CHIMERE. To perform this combination, kriging with external drift is applied using a moving neighbourhood. Forecast maps for the current and next two days (D+0, D+1, D+2) are computed according to the same methodology, except that the observation data are replaced by statistical forecasts at the monitoring sites obtained by station specific multilinear regression models. A constraint in that approach { denoted as statistical adaptation - lies in the availability of one to several years of training and validation data in order to build robust and efficient regression models. A way to get round this issue is to consider the analysis and adjustment of the forecast as a unique procedure in a spatio-temporal framework. In this study a spatio-temporal external drift kriging using a spatio-temporal moving neighbourhood has been developed and evaluated over one year on both the European and national scales. In that approach the prediction of D+0 concentrations is similar to an analysis where the available data in the neighbourhood are past observations until D-1 and CHIMERE outputs until D+0. The spatio-temporal covariance model and the results of the evaluation will be presented and commented. Those results will be compared to the performance of the current statistical adaptation. Possible areas for improvement and development will be discussed.

International audience

Soumission; We present an overview of (geo-)statistical models, methods and techniques for the analysis and predictionof continuous spatio-temporal processes residing in continuous space. Various approaches exist for building statistical models for such processes, estimating their parameters and performing predictions. We cover the Gaussian process approach, very common in spatial statistics and geostatistics, and we focus onR-based implementations of numerical procedures. To illustrate and compare the use of some of the most relevant packages, we treat a real-world application with high-dimensional data. The target variable is the daily mean PM10 concentration predicted thanks to a chemistry- transport model and observation series collected at monitoring stations across France in 2014. We give R code covering the full work-flow from importing data sets to the prediction of PM10 concentrations with a fitted parametric model, including the visualization of data, estimation of the parameters of the spatio-temporal covariance function and model selection. We conclude with some elements of comparison between the packages that are available today and some discussion for future developments.; Nous présentons un aperçu des modèles, méthodes et techniques (géo-)statistiques pour l’analyse et laprévision de processus spatio-temporels continus. De nombreuses approches sont possibles pour la construction de modèles statistiques pour ces processus, l’estimation de leurs paramètres et leur prédiction. Nous avons choisi de présenter l’approche par processus gaussien, la plus communément utilisée en statistiques spatiales et en géostatistiques, ainsi que son implémentation avec le logiciel R. Le variable cible est la moyenne de la concentration quotidienne PM 10 à l’échelle de la France, prédite à l’aide d’un modèle de transport en chimie de l’atmosphère et de séries d’observations obtenues à des stations de surveillance de la qualité de l’air. En suivant le fil d’une application réelle de grande dimension, nous comparons certains des paquets R les plus utilisés. Le code R permettant la visualisation des données, l’estimation des paramètres de la fonction de covariance spatio-temporelle ainsi que la sélection d’un modèle et la prédiction de la concentration de PM 10 est également présenté afin d’illustrer l’enchaînement des étapes. Nous concluons avec une comparaison entre les paquets qui sont disponibles aujourd’hui et ainsi que les pistes de développement qui nous paraissent intéressantes

We are presenting an evaluation of the outcomes of the FAIRMODE & AQUILA intercomparison exercise (IE) on spatial representativeness (SR). The assessment of the spatial representativeness (SR) of air quality monitoring stations is an important subject that is linked to several highly topical areas, including risk assessment and population exposure, the design of monitoring networks, model development, model evaluation and data assimilation. Nevertheless, European regulations lack a clear definition and provisions to determine the SR of the stations. Also in the scientific literature, there is no unified agreement to address this complex problem. In order to further explore this topic and to make progress in the harmonization of the related assessment procedures, the FAIRMODE (Forum for Air Quality Modelling in Europe) Cross-Cutting Activity group on SR organized a comprehensive intercomparison exercise (IE). The main objective of this IE was to evaluate the scattering of spatial representativeness results obtained by applying the range of different contemporary approaches to a jointly used example case study. In order to ensure a broad participation in this exercise, a collaborative effort has been established between FAIRMODE and AQUILA (the European Network of Air Quality Reference Laboratories). As a working basis, a shared dataset has been collected among a set of monitoring, emission and modelling data from the city of Antwerp. Within this IE, 11 different teams from 9 different countries provided their SR estimates for PM10 and NO2 at one traffic site, and for PM10, NO2 and O3 at two urban background sites. In order to narrow down the range of conceivable SR approaches and definitions, it was beforehand suggested to use the area of SR of the monitoring sites as a general concept to work with. During the course of the exercise, this concept of the SR area in fact turned out to be a useful indicator, and 10 of 11 teams were able to define shapes surrounding the stations under investigation, whereas one team rather worked towards a classification of the stations, as this was more common practice for SR evaluation in their member state. The resulting SR areas nevertheless revealed a considerable range of dissimilarity between the different teams - not only in terms of the extent and position of the SR perimeters, but also in the technical procedures and the extent of input data effectively used. These differences required detailed evaluations in order to identify the major factors triggering and controlling this spread, which can be found amongst (1) the basic principles of the methods, (2) the parameterization of the similarity criteria and thresholds, (3) the effective use of input data, and (4) the detailed conceptualization and definitions of SR. These outcomes do underline the need for (i) a more harmonized definition of the concept of “the area of representativeness” and (ii) consistent and transparent criteria used for its quantification. A comprehensive concluding section (chapter 10) is highlighting the challenges that the expert community working on spatial representativeness is currently facing. Recommendations are given for the directions to be focused on SR in the near to mid-term future. In this regards, we are outlining a roadmap towards a modular approach for better SR characterization. It is stressed that that for the aim of harmonization the concept of spatial representativeness will probably require a paradigm shift in its definition. Beyond the questions of harmonization, it should not be disregarded that alternative interpretations for the strong variability of the SR results might exist. The observed divergences in SR results could for example point us to some more fundamental discrepancies related to the evaluation of the air quality data. It is advised to take care, that in the endeavour for methodological harmonization such alternative explanations are not overlooked. An example could be a potential inconsistency within the input data coming from emission, monitoring and modelled data. Furthermore, the findings of this study are not only relevant with regard to the SR of a single monitoring station. It also gives evidence that questions need to be raised about what is the real representativeness of network monitoring data in general since it seems that there is no current consensus on its evaluation. Example given: Is there a need for the European Commission to check the criteria for the number and the siting of Air Quality Monitoring Stations set in the Air Quality Directive when a consensus on SR is reached?, JRC.C.5-Air and Climate

Soumission; We present an overview of (geo-)statistical models, methods and techniques for the analysis and prediction of continuous spatio-temporal processes residing in continuous space. Various approaches exist for building statistical models for such processes, estimating their parameters and performing predictions. We cover the Gaussian process approach, very common in spatial statistics and geostatistics, and we focus on R-based implementations of numerical procedures. To illustrate and compare the use of some of the most relevant packages, we treat a real-world application with high-dimensional data. The target variable is the daily mean PM10 concentration predicted thanks to a chemistry-transport model and observation series collected at monitoring stations across France in 2014. We give R code covering the full work-flow from importing data sets to the prediction of PM10 concentrations with a fitted parametric model, including the visualization of data, estimation of the parameters of the spatio-temporal covariance function and model selection. We conclude with some elements of comparison between the packages that are available today and some discussion for future developments.; Nous présentons un aperçu des modèles, méthodes et techniques (géo-)statistiques pour l’analyse et la prévision de processus spatio-temporels continus. De nombreuses approches sont possibles pour la construction de modèles statistiques pour ces processus, l’estimation de leurs paramètres et leur prédiction. Nous avons choisi de présenter l’approche par processus gaussien, la plus communément utilisée en statistiques spatiales et en géostatistiques, ainsi que son implémentation avec le logiciel R. La variable cible est la moyenne de la concentration quotidienne PM10 à l’échelle de la France, prédite à l’aide d’un modèle de transport en chimie de l’atmosphère et de séries d’observations obtenues à des stations de surveillance de la qualité de l’air. En suivant le fil d’une application réelle de grande dimension, nous comparons certains des paquets R les plus utilisés. Le code R permettant la visualisation des données, l’estimation des paramètres de la fonction de covariance spatio-temporelle ainsi que la sélection d’un modèle et la prédiction de la concentration de PM10 est également présenté afin d’illustrer l’enchaînement des étapes. Nous concluons avec une comparaison entre les paquets qui sont disponibles aujourd’hui et ainsi que les pistes de développement qui nous paraissent intéressantes.

International audience

Like all activity sectors, agriculture is responsible for emissions of many pollutants and particularly gaseous precursors of particulate matter, as ammonia. The French official emission inventory indicates that more than 97% (CITEPA, 2015) of anthropogenic ammonia emissions are due to agriculture (mainly due to breeding and nitrogen fertilizers applications). Ammonia combined with nitric acid is the source of secondary particles formation into the atmosphere contributing to the total particle load. Ammonia, under favorable weather conditions, reacts with nitrogen oxides and forms ammonium nitrate. During pollution episodes, this compound can represent over 50% of the PM10 mass. Anticyclonic situations conditions in late winter favor the accumulation of pollutants in the lowest layers of the atmosphere and the formation of ammonium nitrate. Scenario simulations using air quality models show that ammonia emission reductions lead to a significant reduction in ammonium nitrate concentrations, this decrease is even more important if other measures to reduce nitrogen oxides emissions are taken. Long-term policies to weaken ammonia emissions should reduce the number of exceedances of the PM10 and PM2.5 limit values. Other components like soil particles are also issued from agriculture, they are generally larger than 2.5 μm in diameter. Anyway, agriculture can play an important role in air quality management., Comme tous les secteurs d’activité anthropique, le secteur agricole est responsable de l’émission de polluants et précurseurs de polluants atmosphériques, notamment particulaires. L’agriculture se distingue par de très fortes émissions d’ammoniac, le recensement national des émissions indique que plus de 97 % [1] des émissions anthropiques de ce composé sont dues à l’agriculture (élevage et épandages d’engrais azotés) en France. L’ammoniac combiné à l’acide nitrique est à la source de la formation de particules secondaires dans l’atmosphère contribuant ainsi à la charge totale en particules. En effet, dans des conditions météorologiques favorables, l’ammoniac réagit avec les oxydes d’azote pour former le nitrate d’ammonium. À la fin de l’hiver et au début du printemps, la France et une bonne partie de l’Europe font régulièrement face à des épisodes de particules d’ampleur importante qui trouvent leurs origines dans un cumul d’émissions auquel contribuent les activités agricoles du fait de la période propice aux activités d’épandage d’engrais. Par ailleurs, les situations météorologiques facilitent l’évaporation de l’ammoniac et les mécanismes chimiques de formation de nitrate d’ammonium. En situation d’épisode, le nitrate d’ammonium peut représenter plus de 50 % de la masse de PM10.

Like all activity sectors, agriculture is responsible for the emission of many pollutants and particularly gaseous precursors of particulate matter, it is characterized by very high ammonia emissions. The French official emission inventory indicates that more than 97% (CITEPA, 2015) of anthropogenic emissions of this compound are due to agriculture (mainly due to breeding and nitrogen fertilizers applications). Ammonia combined with nitric acid is the source of secondary particles formation into the atmosphere contributing to the total particle load. Other components like soil particles are also issued from agriculture, they are generally greater than 2.5 μm in diameter. Thus, agriculture has an important role in air quality. Ammonia, under favorable weather conditions, reacts with the nitric acid and forms ammonium nitrate. During pollution episodes, this compound can represent over 50% of the PM10 mass. Anticyclonic situations conditions in late winter favor the accumulation of pollutants in the lower layers of the atmosphere and the formation of ammonium nitrate. Scenario simulations using air quality models show that ammonia emission reductions lead to a significant reduction in ammonium nitrate concentrations, the decrease is even more important if other measures to reduce nitrogen oxides emissions are taken. Long-term policies to weaken ammonia emissions will reduce the number of exceedances of PM10 and PM2.5 limit values., Comme tous les secteurs d’activité, le secteur agricole est responsable de l’émission de nombreux polluants et précurseurs de polluants notamment particulaires. L’agriculture se distingue par de très fortes émissions d’ammoniac, le recensement officiel des émissions indique que plus de 97 % (CITEPA, 2015) des émissions anthropiques de ce composé sont dues à l’agriculture (élevage et épandages d’engrais azotés) en France. L’ammoniac combiné à l’acide nitrique est à la source de la formation de particules secondaires dans l’atmosphère, contribuant ainsi à la charge totale en particules. D’autres émissions, notamment de particules terrigènes, sont imputables à l’agriculture lesquelles sont de taille généralement supérieure à 2,5 µm. L’agriculture a ainsi un rôle important sur la qualité de l’air. L’ammoniac, dans des conditions météorologiques favorables, réagit avec l’acide nitrique pour former le nitrate d’ammonium. En situation d’épisode, le nitrate d’ammonium peut représenter plus de 50 % de la masse de PM10, les situations anticycloniques de fin d’hiver présentent des conditions idéales pour favoriser l’accumulation des polluants dans les basses couches de l’atmosphère et la réaction de formation du nitrate d’ammonium. Les simulations de scénarios montrent que des réductions d’émissions d’ammoniac aboutissent à une baisse sensible des concentrations de nitrate d’ammonium, cette baisse est d’autant plus importante si elle s’accompagne d’autres mesures visant à réduire les émissions des oxydes d’azote. Des politiques de long terme visant à réduire l’ensemble des émissions d’ammoniac permettront également de réduire le nombre de dépassements des valeurs réglementaires des PM10 et PM2.5.

International audience