Search

Hundreds of petabytes are produced annually at weather and climate forecast centres worldwide. Compression is essential to reduce storage and to facilitate data sharing. Current techniques do not distinguish the real from the false information in data, leaving the level of meaningful precision unassessed or often subjectively chosen. Many of the trailing mantissa bits in floating-point numbers occur independently with high information entropy, reducing the efficiency of compression algorithms. Here we define the bitwise real information content from information theory as the mutual information of bits in adjacent grid points. The analysis automatically determines a precision from the data itself, based on the separation of real and false information bits. Applied to data from the Copernicus Atmospheric Monitoring Service (CAMS), most variables contain fewer than 7 bits of real information per value and are highly compressible due to spatio-temporal correlation. Rounding bits without real information to zero facilitates lossless compression algorithms and encodes the uncertainty within the data itself. The removal of bits with high entropy but low real information allows us to minimize information loss but maximize the efficiency of the compression algorithms. All CAMS data are 17x compressed in the longitudinal dimension and relative to 64-bit floats, while preserving 99% of real information. Combined with four-dimensional compression using the floating-point compressor Zfp, factors beyond 60x are achieved, with no significant increase of the forecast error. For multidimensional compression it is generally advantageous to include as many highly correlated dimensions as possible. A data compression Turing test is proposed to optimize compressibility while minimizing information loss for the end use of weather and climate forecast data.

The Copernicus Atmosphere Monitoring Service (CAMS) reanalysis is the latest global reanalysis dataset of atmospheric composition produced by the European Centre for Medium-Range Weather Forecasts (ECMWF), consisting of three-dimensional time-consistent atmospheric composition fields, including aerosols and chemical species. The dataset currently covers the period 2003–2016 and will be extended in the future by adding 1 year each year. A reanalysis for greenhouse gases is being produced separately. The CAMS reanalysis builds on the experience gained during the production of the earlier Monitoring Atmospheric Composition and Climate (MACC) reanalysis and CAMS interim reanalysis. Satellite retrievals of total column CO; tropospheric column NO2; aerosol optical depth (AOD); and total column, partial column and profile ozone retrievals were assimilated for the CAMS reanalysis with ECMWF's Integrated Forecasting System. The new reanalysis has an increased horizontal resolution of about 80 km and provides more chemical species at a better temporal resolution (3-hourly analysis fields, 3-hourly forecast fields and hourly surface forecast fields) than the previously produced CAMS interim reanalysis. The CAMS reanalysis has smaller biases compared with most of the independent ozone, carbon monoxide, nitrogen dioxide and aerosol optical depth observations used for validation in this paper than the previous two reanalyses and is much improved and more consistent in time, especially compared to the MACC reanalysis. The CAMS reanalysis is a dataset that can be used to compute climatologies, study trends, evaluate models, benchmark other reanalyses or serve as boundary conditions for regional models for past periods.

As part of the Copernicus Atmosphere Monitoring Service (CAMS), operated by ECMWF on behalf of the European Commission, global analyses and forecasts of atmospheric composition have been produced operationally since 2015. These were built on many years of previous work under the GEMS and MACC projects, which began producing regular forecasts in 2007.Since the transition to an operational service, there have continued to be many new developments and improvements to the system in five major upgrades, including increased horizontal and vertical resolution, updated emissions and paramterisations, additional species such as nitrate aerosol, as well as updates to the underlying meteorological model and data assimilation. The components of this system (aerosols, gas-phase chemistry, meteorology and the ocean) are also now coupled more tightly via active feedbacks then ever before.In this interactive presentation, we will demonstrate the impact of a number of these developments on the performance of the resulting global air quality forecasts, alongside the continuing evolution of our approaches to assessing model improvement against independent in-situ and remote-sensing observations from a variety of platforms.Because the continuing evolution of an operational system can make the analysis of long-term trends problematic, we will also contrast this with the CAMS global reanalysis product, which (while not using the very latest version of the model) do provide a consistent long-term dataset from 2003 onwards.

A portable Fourier transform spectrometer (FTS), model EM27/SUN, was deployed onboard the research vessel Polarstern to measure the column-average dry air mole fractions of carbon dioxide (XCO2) and methane (XCH4) by means of direct sunlight absorption spectrometry. We report on technical developments as well as data calibration and reduction measures required to achieve the targeted accuracy of fractions of a percent in retrieved XCO2 and XCH4 while operating the instrument under field conditions onboard the moving platform during a 6-week cruise on the Atlantic from Cape Town (South Africa, 34° S, 18° E; 5 March 2014) to Bremerhaven (Germany, 54° N, 19° E; 14 April 2014). We demonstrate that our solar tracker typically achieved a tracking precision of better than 0.05° toward the center of the sun throughout the ship cruise which facilitates accurate XCO2 and XCH4 retrievals even under harsh ambient wind conditions. We define several quality filters that screen spectra, e.g., when the field of view was partially obstructed by ship structures or when the lines-of-sight crossed the ship exhaust plume. The measurements in clean oceanic air, can be used to characterize a spurious air-mass dependency. After the campaign, deployment of the spectrometer alongside the TCCON (Total Carbon Column Observing Network) instrument at Karlsruhe, Germany, allowed for determining a calibration factor that makes the entire campaign record traceable to World Meteorological Organization (WMO) standards. Comparisons to observations of the GOSAT satellite and concentration fields modeled by the European Centre for Medium-Range Weather Forecasts (ECMWF) Copernicus Atmosphere Monitoring Service (CAMS) demonstrate that the observational setup is well suited to provide validation opportunities above the ocean and along interhemispheric transects.

Aerosols affect the atmosphere through direct interaction with short-wave and long-wave radiation and the microphysical properties of clouds. In this paper we report in detail on several mechanisms by which the short-term impact of dust on surface radiative fluxes can affect the dust loading of the atmosphere via modification of boundary-layer meteorology. This in turn affects the aerosol radiative forcing itself. Examples of these feedbacks between dust and boundary layer meteorology were observed during a series of dust storms in the Sahara and the eastern Mediterranean in April 2012. These case studies have been analysed using the Monitoring Atmospheric Composition and Climate – Interim Implementation (MACC-II) system. The radiative fluxes in the short-wave and long-wave spectra were both significantly affected by the prognostic aerosol–radiation interaction, which in turn impacted the meteorological simulation. Reduced incoming solar radiation below the aerosol layers led to a decrease in maximum surface temperatures and to a more stable thermal stratification of the lower atmosphere. This in turn forced weaker surface wind speeds and eventually smaller dust emissions. Moreover, we also observed a secondary impact of the aerosol radiative forcing, whereby horizontal gradients of surface temperature were increased at the edge of the dust plume, which led to local increases of surface wind speeds due to the thermal wind effect. The differentiated impact of the aerosol layer on surface pressure also contributed to the increase in surface wind speed and dust production in the same area. Enhanced long-wave radiative fluxes by the dust mass were associated with opposite processes. Less stable thermal stratification at night, brought mainly by higher minimum temperatures at the surface, caused stronger surface winds. At the edge of the dust storm, weaker horizontal temperature and pressure gradients forced lower winds and reduced dust production. Regarding dust emissions, short-wave radiative forcing had a larger impact than long-wave radiative forcing, corroborating several previous studies. For surface temperature, short-wave and long-wave contribution were close in intensity. These feedbacks were amplified when using data assimilation to build the aerosol analysis of the MACC-II global system. This led to an improvement in the short-term forecasts of thermal radiative fluxes and surface temperatures.

The Global Fire Assimilation System (GFASv1.0) calculates biomass burning emissions by assimilating Fire Radiative Power (FRP) observations from the MODIS instruments onboard the Terra and Aqua satellites. It corrects for gaps in the observations, which are mostly due to cloud cover, and filters spurious FRP observations of volcanoes, gas flares and other industrial activity. The combustion rate is subsequently calculated with land cover-specific conversion factors. Emission factors for 40 gas-phase and aerosol trace species have been compiled from a literature survey. The corresponding daily emissions have been calculated on a global 0.5° × 0.5° grid from 2003 to the present. General consistency with the Global Fire Emission Database version 3.1 (GFED3.1) within its accuracy is achieved while maintaining the advantages of an FRP-based approach: GFASv1.0 makes use of the quantitative information on the combustion rate that is contained in the FRP observations, and it detects fires in real time at high spatial and temporal resolution. GFASv1.0 indicates omission errors in GFED3.1 due to undetected small fires. It also exhibits slightly longer fire seasons in South America and North Africa and a slightly shorter fire season in Southeast Asia. GFASv1.0 has already been used for atmospheric reactive gas simulations in an independent study, which found good agreement with atmospheric observations. We have performed simulations of the atmospheric aerosol distribution with and without the assimilation of MODIS aerosol optical depth (AOD). They indicate that the emissions of particulate matter need to be boosted by a factor of 2–4 to reproduce the global distribution of organic matter and black carbon. This discrepancy is also evident in the comparison of previously published top-down and bottom-up estimates. For the time being, a global enhancement of the particulate matter emissions by 3.4 is recommended. Validation with independent AOD and PM10 observations recorded during the Russian fires in summer 2010 show that the global Monitoring Atmospheric Composition and Change (MACC) aerosol model with GFASv1.0 aerosol emissions captures the smoke plume evolution well when organic matter and black carbon are enhanced by the recommended factor. In conjunction with the assimilation of MODIS AOD, the use of GFASv1.0 with enhanced emission factors quantitatively improves the forecast of the aerosol load near the surface sufficiently to allow air quality warnings with a lead time of up to four days.

A detailed 3-D evaluation of an ensemble of five regional Chemistry Transport Models (RCTM) and one global CTM with focus on free tropospheric ozone over Europe is presented. It is performed over a summer period (June to August 2008) in the context of the GEMS-RAQ project. A data set of about 400 vertical ozone profiles from balloon soundings and commercial aircraft at 11 different locations is used for model evaluation, in addition to satellite measurements with the infrared nadir sounder (IASI) showing largest sensitivity to free tropospheric ozone. In the middle troposphere, the four regional models using the same top and boundary conditions from IFS-MOZART exhibit a systematic negative bias with respect to observed profiles of about −20%. Root Mean Square Error (RMSE) values are constantly growing with altitude, from 22% to 32% to 53%, respectively for 0–2 km, 2–8 km and 8–10 km height ranges. Lowest correlation is found in the middle troposphere, with minimum coefficients (R) between 0.2 to 0.45 near 8 km, as compared to 0.7 near the surface and similar values around 10 km. A sensitivity test made with the CHIMERE mode also shows that using hourly instead of monthly chemical boundary conditions generally improves the model skill (i.e. improve RMSE and correlation). Lower tropospheric 0–6 km partial ozone columns derived from IASI show a clear North-South gradient over Europe, which is qualitatively reproduced by the models. Also the temporal variability showing decreasing ozone concentrations in the lower troposphere (0–6 km columns) during summer is well reproduced by models even if systematic bias remains (the value of the bias being also controlled by the type of used boundary conditions). A multi-day case study of a trough with low tropopause was conducted and showed that both IASI and models were able to resolve strong horizontal gradients of middle and upper tropospheric ozone occurring in the vicinity of an upper tropospheric frontal zone.

Daily global analyses and 5-day forecasts are generated in the context of the European Monitoring Atmospheric Composition and Climate (MACC) project using an extended version of the Integrated Forecasting System (IFS) of the European Centre for Medium-Range Weather Forecasts (ECMWF). The IFS now includes modules for chemistry, deposition and emission of reactive gases, aerosols, and greenhouse gases, and the 4-dimensional variational data assimilation scheme makes use of multiple satellite observations of atmospheric composition in addition to meteorological observations. This paper describes the data assimilation setup of the new Composition-IFS (C-IFS) with respect to reactive gases and validates analysis fields of ozone (O3), carbon monoxide (CO), and nitrogen dioxide (NO2) for the year 2008 against independent observations and a control run without data assimilation. The largest improvement in CO by assimilation of Measurements of Pollution in the Troposphere (MOPITT) CO columns is seen in the lower troposphere of the Northern Hemisphere (NH) extratropics during winter, and during the South African biomass-burning season. The assimilation of several O3 total column and stratospheric profile retrievals greatly improves the total column, stratospheric and upper tropospheric O3 analysis fields relative to the control run. The impact on lower tropospheric ozone, which comes from the residual of the total column and stratospheric profile O3 data, is smaller, but nevertheless there is some improvement particularly in the NH during winter and spring. The impact of the assimilation of tropospheric NO2 columns from the Ozone Monitoring Instrument (OMI) is small because of the short lifetime of NO2, suggesting that NO2 observations would be better used to adjust emissions instead of initial conditions. The results further indicate that the quality of the tropospheric analyses and of the stratospheric ozone analysis obtained with the C-IFS system has improved compared to the previous "coupled" model system of MACC.

The European MACC (Monitoring Atmospheric Composition and Climate) project is preparing the operational Copernicus Atmosphere Monitoring Service (CAMS), one of the services of the European Copernicus Programme on Earth observation and environmental services. MACC uses data assimilation to combine in situ and remote sensing observations with global and regional models of atmospheric reactive gases, aerosols, and greenhouse gases, and is based on the Integrated Forecasting System of the European Centre for Medium-Range Weather Forecasts (ECMWF). The global component of the MACC service has a dedicated validation activity to document the quality of the atmospheric composition products. In this paper we discuss the approach to validation that has been developed over the past 3 years. Topics discussed are the validation requirements, the operational aspects, the measurement data sets used, the structure of the validation reports, the models and assimilation systems validated, the procedure to introduce new upgrades, and the scoring methods. One specific target of the MACC system concerns forecasting special events with high-pollution concentrations. Such events receive extra attention in the validation process. Finally, a summary is provided of the results from the validation of the latest set of daily global analysis and forecast products from the MACC system reported in November 2014.

Aerosols affect the atmosphere through the aerosol-radiation and the aerosol-clouds interactions. In this paper we report on a new mechanism whereby the radiative effect of dust aerosol on surface fluxes acts to increase the dust loading of the atmosphere via modification of boundary-layer stability, thereby acting to enhance the radiative aerosol effect. This positive feedback between dust aerosol and boundary layer stability occurred during a series of dust storms in the Sahara and the Eastern Mediterranean in April 2012, which were studied using the Monitoring Atmospheric Composition and Climate – Interim Implementation (MACC-II) system. The radiative fluxes in the shortwave and long-wave spectra were both significantly affected by the prognostic aerosols-radiation interation, which strongly influenced the meteorological simulation. Reduced incoming solar radiation below the aerosol layers caused a decrease in maximum surface temperatures, and consequently a more stable thermal stratification of the lower atmosphere. The increased thermal stability led to decreased surface wind speeds and therefore to smaller amounts of dust aerosol emissions. Larger downwelling long-wave fluxes were associated with the opposite processes: less stable thermal stratification at night, brought mainly by higher minimum temperatures at the surface, caused stronger surface winds. Overall, the impact by the long-wave radiative forcing was more important than the short-wave contribution. This feedback was amplified when taken into account in the aerosol analysis of the MACC-II global system. It lead to a notable improvement in short term forecast of short and long-wave radiative fluxes, of surface temperature but also of the aerosol burden itself. Forecasts of radiative fluxes in the shortwave and long-wave spectrum were also improved. At a longer range the improvement were less important as the forecast error of the aerosol load increased, thereby highlighting the importance of accurate aerosol representation in the study of aerosol-radiation interaction.

An eight-year long reanalysis of atmospheric composition data covering the period 2003-2010 was constructed as part of the FP7-funded Monitoring Atmospheric Composition and Climate project by assimilating satellite data into a global model and data assimilation system. This reanalysis provides fields of chemically reactive gases, namely carbon monoxide, ozone, nitrogen oxides, and formaldehyde, as well as aerosols and greenhouse gases globally at a horizontal resolution of about 80 km for both the troposphere and the stratosphere. This paper describes the assimilation system for the reactive gases and presents validation results for the reactive gas analysis fields to document the data set and to give a first indication of its quality. Tropospheric CO values from the MACC reanalysis are on average 10-20% lower than routine observations from commercial aircrafts over airports through most of the troposphere, and have larger negative biases in the boundary layer at urban sites affected by air pollution, possibly due to an underestimation of CO or precursor emissions. Stratospheric ozone fields from the MACC reanalysis agree with ozonesondes and ACE-FTS data to within ±10% in most seasons and regions. In the troposphere the reanalysis shows biases of -5% to +10% with respect to ozonesondes and aircraft data in the extratropics, but has larger negative biases in the tropics. Area-averaged total column ozone agrees with ozone fields from a multi-sensor reanalysis data set to within a few percent. NO2 fields from the reanalysis show the right seasonality over polluted urban areas of the NH and over tropical biomass burning areas, but underestimate wintertime NO2 maxima over anthropogenic pollution regions and overestimate NO2 in northern and southern Africa during the tropical biomass burning seasons. Tropospheric HCHO is well simulated in the MACC reanalysis even though no satellite data are assimilated. It shows good agreement with independent SCIAMACHY retrievals over regions dominated by biogenic emissions with some anthropogenic input, such as the eastern US and China, and also over African regions influenced by biogenic sources and biomass burning. © Author(s) 2013., SCOPUS: re.j, info:eu-repo/semantics/published

[1] This study presents the new aerosol assimilation system, developed at the European Centre for Medium-Range Weather Forecasts, for the Global and regional Earth-system Monitoring using Satellite and in-situ data (GEMS) project. The aerosol modeling and analysis system is fully integrated in the operational four-dimensional assimilation apparatus. Its purpose is to produce aerosol forecasts and reanalyses of aerosol fields using optical depth data from satellite sensors. This paper is the second of a series which describes the GEMS aerosol effort. It focuses on the theoretical architecture and practical implementation of the aerosol assimilation system. It also provides a discussion of the background errors and observations errors for the aerosol fields, and presents a subset of results from the 2-year reanalysis which has been run for 2003 and 2004 using data from the Moderate Resolution Imaging Spectroradiometer on the Aqua and Terra satellites. Independent data sets are used to show that despite some compromises that have been made for feasibility reasons in regards to the choice of control variable and error characteristics, the analysis is very skillful in drawing to the observations and in improving the forecasts of aerosol optical depth.

International audience; This paper presents the aerosol modeling now part of the ECMWF Integrated Forecasting System (IFS). It includes new prognostic variables for the mass of sea salt, dust, organic matter and black carbon, and sulphate aerosols, interactive with both the dynamics and the physics of the model. It details the various parameterizations used in the IFS to account for the presence of tropospheric aerosols. Details are given of the various formulations and data sets for the sources of the different aerosols and of the parameterizations describing their sinks. Comparisons of monthly mean and daily aerosol quantities like optical depths against satellite and surface observations are presented. The capability of the forecast model to simulate aerosol events is illustrated through comparisons of dust plume events. The ECMWF IFS provides a good description of the horizontal distribution and temporal variability of the main aerosol types. The forecastonly model described here generally gives the total aerosol optical depth within 0.12 of the relevant observations and can therefore provide the background trajectory information for the aerosol assimilation system described in part 2 of this paper.

The organization of a knowledge base within a laboratory expert system for solving C-13 NMR spectroscopy oriented problems is discussed. Four different forms for storing knowledge (formulas, tables, libraries, and hierarchical trees) used in the expert system CARBON are described. The emphasis is put on describing the flow of information and organization of knowledge in the system, which is designed to run on personal computers. In particular, we describe the implementation of the knowledge in such a way that it not only yields relevant (requested) information but also informs the user about missing information and possibly locates wrong data or wrong conclusions inferred. The two-way information flow enables the user to check retrieved information: if the query is a structure and the retrieved result is a spectrum, the check is made by spectral simulation; or, on the other hand, if the query is a spectrum and the yielded result a structure, the structural features can be checked by a decision tree or by assignation of spectra. Due to the built-in two-way handling of information the user can obtain the answer to his problem in an iterative way, testing a number of hypotheses.

Over the past several years, there has been a rapid development in the number and quality of global aerosol models intended for operational forecasting use. Indeed, most centers with global numerical weather prediction (NWP) capabilities have some program for aerosol prediction. These aerosol models typically have differences in their underlying meteorology as well as aerosol sources, sinks, microphysics and transformations. However, like similar diversity in aerosol climate models, the aerosol forecast models have fairly similar overall bulk error statistics for aerosol optical thickness (AOT)-one of the few aerosol metrics that is globally available. Experience in climate and weather prediction has shown that in situations such as this where there are several independent models, a multi-model ensemble or consensus will be top performing in many key error metrics. Further, multi-model ensembles provide a highly valuable tool for forecasters attempting to predict severe aerosol events. Here we present the first steps in developing a global multi-model aerosol forecasting ensemble intended for eventual operational and basic research use. Drawing from members of the International Cooperative for Aerosol Prediction (ICAP) latest generation of quasi-operational aerosol models, five day AOT forecasts are analyzed for December 2011 through November 2012 from four institutions: ECMWF, JMA, NASA GSFC, and NRL/FNMOC. For dust, we also include the NOAA NGAC product in our analysis. The Barcelona Supercomputing Centre (NMMC) and UK Met office dust product have also recent become available with ICAP, but have insufficient data to be included in this analysis period. A simple consensus ensemble of member and mean AOT fields for modal species (e.g., fine and coarse mode, and a separate dust ensemble) is used to create the ICAP Multi-Model Ensemble (ICAP-MME). The ICAP-MME is run daily at 0Z for 6 hourly forecasts out to 120 h. Basing metrics on comparisons to 21 regionally representative Aerosol Robotic Network (AERONET) sites, all models generally captured the basic aerosol features of the globe. However, there is an overall AOT low bias among models, particularly for high AOT events. Biomass burning regions have the most diversity in seasonal average AOT. The southern oceans, though low in AOT, nevertheless also have high diversity. In regard to root mean square error, as expected the ICAP-MME placed first over all models worldwide, and was typically first or second in ranking against all models at individual sites. These results are encouraging; as more global operational aerosol models come on line, we expect their inclusion in a robust operational multi-model ensemble will provide valuable aerosol forecasting guidance.

A system for the digitization of infrared spectra and subsequent automatic generation, management and search of spectral data files is described. The older version of the search system was further developed and upgraded. In the present version, the user can constantly update not only the spectral files but also the inverted files of peak positions, needed during the search. The system was tested in routine industrial analysis and can be compared favourably with the existing commercialized IR search systems.