Search

The assimilation of clear-sky radiance in the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric analysis relies on the clear-sky radiances observation operator. Some of these radiances have frequencies that make them sensitive to both the surface and atmosphere. Because the atmospheric and surface analyses are currently not strongly coupled, a specific treatment of the surface is required. The observation operator specifically expects a skin temperature value at the observation location and time as well as the profiles of the atmospheric variables along the viewing path. This skin temperature is added to the control variable and optimised simultaneously with all of the atmospheric variables to produce optimal simulated radiances. We present two approaches to add the skin temperature to the control variable. In the current TOVS Control Variable (TOVSCV) approach, a series of skin temperature values per observation location is added to the control variable. Effectively, in the optimisation process, the skin temperature acts as a sink variable in observation space and is uncoupled from the skin temperature at other locations. In the novel SKin Temperature in the Extended Control Vector (SKTECV) approach, two-dimensional skin temperature fields are added to the control variable. All clear-sky radiances then participate in the optimisation of these two-dimensional fields, and the analysis produces temporally and spatially consistent skin temperature fields. We compare the two approaches over two seasons of 3 months each. Overall, there is a neutral impact of the new approach on the analysis and forecast. Moreover, there is some evidence that the contribution of the subsurface layers should be represented in the new approach for the skin temperature associated with the microwave instruments.

Recent studies have demonstrated that it is possible to combine machine learning with data assimilation to reconstruct the dynamics of a physical model partially and imperfectly observed. Data assimilation is used to estimate the system state from the observations, while machine learning computes a surrogate model of the dynamical system based on those estimated states. The surrogate model can be defined as an hybrid combination where a physical model based on prior knowledge is enhanced with a statistical model estimated by a neural network. The training of the neural network is typically done offline, once a large enough dataset of model state estimates is available. By contrast, with online approaches the surrogate model is improved each time a new system state estimate is computed. Online approaches naturally fit the sequential framework encountered in geosciences where new observations become available with time. In a recent methodology paper, we have developed a new weak-constraint 4D-Var formulation which can be used to train a neural network for online model error correction. In the present article, we develop a simplified version of that method, in the incremental 4D-Var framework adopted by most operational weather centres. The simplified method is implemented in the ECMWF Object-Oriented Prediction System, with the help of a newly developed Fortran neural network library, and tested with a two-layer two-dimensional quasi geostrophic model. The results confirm that online learning is effective and yields a more accurate model error correction than offline learning. Finally, the simplified method is compatible with future applications to state-of-the-art models such as the ECMWF Integrated Forecasting System.

The idea of using machine learning (ML) methods to reconstruct the dynamics of a system is the topic of recent studies in the geosciences, in which the key output is a surrogate model meant to emulate the dynamical model. In order to treat sparse and noisy observations in a rigorous way, ML can be combined to data assimilation (DA). This yields a class of iterative methods in which, at each iteration a DA step assimilates the observations, and alternates with a ML step to learn the underlying dynamics of the DA analysis. In this article, we propose to use this method to correct the error of an existent, knowledge-based model. In practice, the resulting surrogate model is an hybrid model between the original (knowledge-based) model and the ML model. We demonstrate numerically the feasibility of the method using a two-layer, two-dimensional quasi-geostrophic channel model. Model error is introduced by the means of perturbed parameters. The DA step is performed using the strong-constraint 4D-Var algorithm, while the ML step is performed using deep learning tools. The ML models are able to learn a substantial part of the model error and the resulting hybrid surrogate models produce better short- to mid-range forecasts. Furthermore, using the hybrid surrogate models for DA yields a significantly better analysis than using the original model.

Model errors are increasingly seen as a fundamental performance limiter in both Numerical Weather Prediction and Climate Prediction simulations run with state of the art Earth system digital twins.This has motivated recent efforts aimed at estimating and correcting the systematic, predictable components of model error in a consistent data assimilation framework. While encouraging results have been obtained with a careful examination of the spatial aspects of the model error estimates, less attention has been devoted to the time correlation aspects of model errors and their impact on the assimilation cycle. In this work we employ a Lagged Analysis Increment Covariance (LAIG) diagnostic to gain insight in the temporal evolution of systematic model errors in the ECMWF operational data assimilation system, evaluate the effectiveness of the current weak constraint 4DVar algorithm in reducing these types of errors and, based on these findings,start exploring new ideas for the development of model error estimation and correction strategies in data assimilation.

The idea of using machine learning (ML) methods to reconstruct the dynamics of a system is the topic of recent studies in the geosciences in which the key output is a surrogate model meant to emulate the dynamical model. In order to treat sparse and noisy observations in a rigorous way, ML can be combined with data assimilation (DA). This yields a class of iterative methods in which, at each iteration a DA step estimates the system's state, and alternates with a ML step to learn the system's dynamics from the DA analysis.This framework can be used to correct the error of an existent, physical model. The resulting surrogate model is hybrid, with a physical and a statistical part. In practice, the correction can be added as an integrated term (i.e. in the model resolvent) or directly inside the tendencies of the physical model. The resolvent correction is easy to implement but is not suited for short-term predictions. The tendency correction is more technical since it requires the adjoint of the physical model, but also more flexible and can be used for any forecast lead time.In this presentation, we start by a proof of concept for the use of joint DA and ML tools to correct model error. We use the resolvent correction with simple neural networks to correct the error of a two-dimensional, two layer quasi-geostrophic layer. The difference between the resolvent and the tendency correction is then illustrated with the two- scale Lorenz model. Finally, we show that the tendency correction opens the possibility to make online model error correction, i.e. improving the model progressively as new observations become available. We compare online and offline learning using the same twin experiment with the two-scale Lorenz model.

International audience; Recent studies have shown that it is possible to combine machine learning methods with data assimilation to reconstruct a dynamical system using only sparse and noisy observations of that system. The same approach can be used to correct the error of a knowledge-based model. The resulting surrogate model is hybrid, with a statistical part supplementing a physical part. In practice, the correction can be added as an integrated term (i.e. in the model resolvent) or directly inside the tendencies of the physical model. The resolvent correction is easy to implement. The tendency correction is more technical, in particular it requires the adjoint of the physical model, but also more flexible. We use the two-scale Lorenz model to compare the two methods. The accuracy in long-range forecast experiments is somewhat similar between the surrogate models using the resolvent correction and the tendency correction. By contrast, the surrogate models using the tendency correction significantly outperform the surrogate models using the resolvent correction in data assimilation experiments. Finally, we show that the tendency correction opens the possibility to make online model error correction, i.e. improving the model progressively as new observations become available. The resulting algorithm can be seen as a new formulation of weak-constraint 4D-Var. We compare online and offline learning using the same framework with the two-scale Lorenz system, and show that with online learning, it is possible to extract all the information from sparse and noisy observations.

Tropical cyclones are some of the most devastating natural hazards and the “three beasts”—Harvey, Irma, and Maria—during the Atlantic hurricane season 2017 are recent examples. The European Centre for Medium-Range Weather Forecasts (ECMWF) is working on fulfilling its 2016–25 strategy in which early warnings for extreme events will be made possible by a high-resolution Earth system ensemble forecasting system. Several verification reports acknowledge deterministic and probabilistic tropical cyclone tracks from ECMWF as world leading. However, producing reliable intensity forecasts is still a difficult task for the ECMWF global forecasting model, especially regarding maximum wind speed. This article will put the ECMWF strategy into a tropical cyclone perspective and highlight some key research activities, using Harvey, Irma, and Maria as examples. We describe the observation usage around tropical cyclones in data assimilation and give examples of their impact. From a model perspective, we show the impact of running at 5-km resolution and also the impact of applying ocean coupling. Finally, we discuss the future challenges to tackle the errors in intensity forecasts for tropical cyclones.

In both Numerical Weather Prediction and Climate Prediction, achieving improved accuracy and reliability is fundamentally dependent on identifying the sources and reducing the effects of model error. It has been recently demonstrated (Laloyaux et al., 2020) that weak constraint 4D-Var can estimate and correct for a large fraction of model error in the stratosphere, where the current global observing system is sufficiently dense and homogeneous. Accounting for the model error in the entire atmospheric column, specifically in the troposphere, remains challenging due to the difficulty in disentangling different sources of errors with similar spatial scales, and is the focus of current research.In this work we demonstrate how Deep Learning techniques can be applied to the problem of estimation and online correction of model error. Recent results (Bonavita and Laloyaux, 2020) in the ECMWF Integrated Forecasting System (IFS) have shown that model error can be learned by an Artificial Neural Network (ANN) and applied in a weak constraint 4D-Var data assimilation framework as a model tendency forcing term. Moreover, the error estimation can extend to the whole atmospheric column and result in significantly improved analyses and forecasts. We have recently implemented in the ECMWF IFS the capability of applying online such ANN-based model error. This allows us to extend the application of the ANN-based model error parameterization from the data assimilation cycle to the long forecast step, where a model error tendency correction is continuously estimated and applied as a model forcing. We show preliminary results of the experiments conducted in the IFS framework and discuss our current understanding of the advantages and limitations of these techniques.

Recent developments in machine learning (ML) have demonstrated impressive skills in reproducing complex spatiotemporal processes. However, contrary to data assimilation (DA), the underlying assumption behind ML methods is that the system is fully observed and without noise, which is rarely the case in numerical weather prediction. In order to circumvent this issue, it is possible to embed the ML problem into a DA formalism characterised by a cost function similar to that of the weak-constraint 4D-Var (Bocquet et al., 2019; Bocquet et al., 2020). In practice ML and DA are combined to solve the problem: DA is used to estimate the state of the system while ML is used to estimate the full model. In realistic systems, the model dynamics can be very complex and it may not be possible to reconstruct it from scratch. An alternative could be to learn the model error of an already existent model using the same approach combining DA and ML. In this presentation, we test the feasibility of this method using a quasi geostrophic (QG) model. After a brief description of the QG model model, we introduce a realistic model error to be learnt. We then asses the potential of ML methods to reconstruct this model error, first with perfect (full and noiseless) observation and then with sparse and noisy observations. We show in either case to what extent the trained ML models correct the mid-term forecasts. Finally, we show how the trained ML models can be used in a DA system and to what extent they correct the analysis.Bocquet, M., Brajard, J., Carrassi, A., and Bertino, L.: Data assimilation as a learning tool to infer ordinary differential equation representations of dynamical models, Nonlin. Processes Geophys., 26, 143–162, 2019Bocquet, M., Brajard, J., Carrassi, A., and Bertino, L.: Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization, Foundations of Data Science, 2 (1), 55-80, 2020Farchi, A., Laloyaux, P., Bonavita, M., and Bocquet, M.: Using machine learning to correct model error in data assimilation and forecast applications, arxiv:2010.12605, submitted.

N/A

A new configuration of the European Centre for Medium-Range Weather Forecasts (ECMWF) incremental 4D-Var data assimilation (DA) system is introduced which builds upon the quasi-continuous DA concept proposed in the mid-1990s. Rather than working with a fixed set of observations, the new 4D-Var configuration exploits the near-continuous stream of incoming observations by introducing recently arrived observations at each outer loop iteration of the assimilation. This allows the analysis to benefit from more recent observations. Additionally, by decoupling the start time of the DA calculations from the observational data cut-off time, real-time forecasting applications can benefit from more expensive analysis configurations that previously could not have been considered. In this work we present results of a systematic comparison of the performance of a Continuous DA system against that of two more traditional baseline 4D-Var configurations. We show that the quality of the analysis produced by the new, more continuous configuration is comparable to that of a conventional baseline that has access to all of the observations in each of the outer loops, which is a configuration not feasible in real-time operational numerical weather prediction. For real-time forecasting applications, the Continuous DA framework allows configurations which clearly outperform the best available affordable non-continuous configuration. Continuous DA became operational at ECMWF in June 2019 and led to significant 2 to 3% reductions in medium-range forecast root mean square errors, which is roughly equivalent to 2-3 hr of additional predictive skill.

Recent success in assimilating cloud- and precipitation-affected satellite observations using the “all-sky” approach is thought to have benefitted from variational data assimilation, particularly its ability to handle moderate nonlinearity and non-Gaussianity and to extract wind information through the generalized tracer effect. Ensemble assimilation relies on assumptions including linearity and Gaussianity that might cause difficulties when using all-sky observations. Here, all-sky assimilation is evaluated in a global ensemble Kalman filter (EnKF) system of near-operational quality, derived from an operational four-dimensional variational (4D-Var) system. To get EnKF working successfully required a new all-sky observation error model (the most successful approach was to inflate error as a multiple of the ensemble spread) and adjustments to localization. With these improvements, assimilation of eight microwave humidity instruments gave 2%–4% improvement in forecast scores whether using EnKF or 4D-Var. Correlations from the ensemble showed that all-sky observations generated sensitivity to wind, temperature, and humidity. EnKF increments shared many similarities with those in 4D-Var. Hence both 4D-Var and ensemble data assimilation were able to make good use of all-sky observations, including the extraction of wind information. In absolute terms the EnKF forecast performance in the troposphere was still worse than that that with 4D-Var, although the gap could be reduced by going from 50 to 100 ensemble members. EnKF errors were larger in the stratosphere, where there are excessive gravity wave increments that are not connected with all-sky assimilation.

This study aims at improving the assimilation of scatterometer wind observations in global Numerical Weather Prediction (NWP) model by refining the background quality control and optimizing the observation sampling strategy. To improve the background quality control, different Huber Norm distribution implementations are tested and compared against the current “Gaussian plus flat” distribution. Sensitivity experiments show that the usage of the Huber Norm distribution improves the analysis and forecasts. The benefit is mainly seen in the lower model levels in the tropics and extra-tropical Southern Hemisphere. The optimal wind sampling is investigated by testing several combinations of thinning scheme and observation error standard deviation. The impact is demonstrated with a large sample and illustrated by a case study. The case study shows the impact of different settings on the analysis and forecast of a tropical cyclone. A revised wind sampling setting, where four times more observations and a higher observation error than the current operational one are used, showed slightly positive impact on the European Centre for Medium-Range Weather Forecasts (ECMWF) global NWP analyses and forecasts.

The ECMWF twentieth century reanalysis (ERA-20C; 1900–2010) assimilates surface pressure and marine wind observations. The reanalysis is single-member, and the background errors are spatiotemporally varying, derived from an ensemble. The atmospheric general circulation model uses the same configuration as the control member of the ERA-20CM ensemble, forced by observationally based analyses of sea surface temperature, sea ice cover, atmospheric composition changes, and solar forcing. The resulting climate trend estimations resemble ERA-20CM for temperature and the water cycle. The ERA-20C water cycle features stable precipitation minus evaporation global averages and no spurious jumps or trends. The assimilation of observations adds realism on synoptic time scales as compared to ERA-20CM in regions that are sufficiently well observed. Comparing to nighttime ship observations, ERA-20C air temperatures are 1 K colder. Generally, the synoptic quality of the product and the agreement in terms of climate indices with other products improve with the availability of observations. The MJO mean amplitude in ERA-20C is larger than in 20CR version 2c throughout the century, and in agreement with other reanalyses such as JRA-55. A novelty in ERA-20C is the availability of observation feedback information. As shown, this information can help assess the product’s quality on selected time scales and regions.

With the ever increasing amount of meteorological data available, from satellites in particular, it becomes more and more important to use as large fraction of this data as practically possible for operational weather forecasts. Two applications are shown where more data are used in ensemble data assimilation and forecasting by distributing satellite data among different ensemble members. For the Ensemble of Data Assimilations (EDA), a version of the perturbed observations ensemble Kalman filter, the ensemble mean error can in theory be reduced by replacing observation perturbations by distributing different subsets of observations to different members, but in practice this is complicated by observation error correlations and the need to maintain the same level of spread in the ensemble as before. For the ensemble forecasts, it is shown that the initial conditions can be improved by re-centring EDA perturbations on multiple equally good analyses that use different subsets of observations instead of re-centring all EDA perturbations on a single one of those analyses. It is shown how the ensemble mean error decreases for decreased error correlation between analyses. We estimate that in practice only 5–6 analysis are needed to obtain 80% of the root mean square error reduction that would be achieved with infinite number of analyses.

The desire to do detailed comparisons between variational and more scalable ensemble-based data assimilation systems in a semioperational environment has led to the development of a state-of-the-art EnKF system at ECMWF. A broad description of the ECMWF EnKF is given in this paper, focusing on highlighting differences compared to standard EnKF practice. In particular, a discussion of the novel algorithm used to control imbalances between the mass and wind fields in the EnKF analysis is given. The scalability and computational properties of the EnKF are reviewed and the implementation choices adopted at ECMWF described. The sensitivity of the ECMWF EnKF to ensemble size, horizontal resolution, and representation of model errors is also discussed. A comparison with 4DVar will be found in Part II of this two-part study.

The desire to do detailed comparisons between variational and more scalable ensemble-based data assimilation systems in a semioperational environment has led to the development of a state-of-the-art EnKF system at ECMWF, which has been described in Part I of this two-part study. In this part the performance of the EnKF system is evaluated compared to a 4DVar of similar resolution. It is found that there is not a major difference between the forecast skill of the two systems. However, similarly to the operational hybrid 4DVar–EDA, a hybrid EnKF–variational system [which we refer to as the hybrid gain ensemble data assimilation (HG-EnDA)] is capable of significantly outperforming both component systems. The HG-EnDA has been implemented with relatively little effort following Penny’s recent study. Results of numerical experimentation comparing the HG-EnDA with the hybrid 4DVar–EDA used operationally at ECMWF are presented, together with diagnostic results, which help characterize the behavior of the proposed ensemble data assimilation system. A discussion of these results in the context of hybrid data assimilation in global NWP is also provided.

Many weather forecasting centres generate the initial conditions for their ensemble prediction system by adding a set of perturbations to a single analysis. At the European Centre for Medium-Range Weather Forecasts (ECMWF), initial conditions from an ensemble of data assimilations (EDA) are re-centred on the deterministic high-resolution analysis. This study assesses how the re-centring step affects the ensemble forecasts and what benefits starting directly from the perturbed EDA members may bring. Numerical experiments are based on suitably modified configurations of the ECMWF medium-range ensemble. Re-centring leads to an artificial increase of the climatological variance of the initial conditions for variables and spatial scales that are not well constrained in the analysis. For those variables and spatial scales, the EDA variance can reach a similar magnitude as the climatological variance. Re-centring on the unperturbed member of the EDA leads to a degradation of the ensemble skill in comparison to starting directly from the perturbed EDA members. Depending on the amplitude of the EDA spread and which scales are less constrained, different lead times are affected. Furthermore, the re-centring step increases the jumpiness of the ensemble forecasts in general and alters the precipitation frequency during the first 12 h of the forecasts. It also becomes apparent from our results that the benefit for the ECMWF ensemble of starting directly from EDA initial conditions is sensitive to several factors: the difference in analysis accuracy between the unperturbed EDA control and the high-resolution analysis, the number of EDA members, and the EDA spread. At present, the forecast skill still benefits from centring the ensemble initial conditions on the high-resolution analysis, but future changes to the EDA may change the balance.

Members in ensemble forecasts differ due to the representations of initial uncertainties and model uncertainties. The inclusion of stochastic schemes to represent model uncertainties has improved the probabilistic skill of the ECMWF ensemble by increasing reliability and reducing the error of the ensemble mean. Recent progress, challenges and future directions regarding stochastic representations of model uncertainties at ECMWF are described in this paper. The coming years are likely to see a further increase in the use of ensemble methods in forecasts and assimilation. This will put increasing demands on the methods used to perturb the forecast model. An area that is receiving a greater attention than 5 to 10 years ago is the physical consistency of the perturbations. Other areas where future efforts will be directed are the expansion of uncertainty representations to the dynamical core and to other components of the Earth system as well as the overall computational efficiency of representing model uncertainty.

The impact of Radio Occultation observations from Global Positioning System satellites (GPSRO) on global Numerical Weather Prediction (NWP) has been analysed with a recent version of the European Centre for Medium-range Weather Forecasting (ECMWF) Integrated Forecasting System. As in previous studies, the use of GPSRO was found to improve the NWP forecast skill and to drastically decrease model-induced temperature biases in the analysis. The maximum forecast impact is in the lower and middle stratosphere, where the GPSRO observations have the smallest errors, but it is also visible in the troposphere. The tropospheric impact of GPSRO comes in part from direct tropospheric measurements and in part from stratosphere–troposphere interactions: this second mechanism is found to be particularly important during the Northern Hemisphere winter. The forecast impact of GPSRO observations is compared with that of conventional and hyperspectral satellite nadir sounders. It is found that although GPSRO data have a smaller impact than those of either class of nadir sounders, they are still able to account for a considerable fraction (30–70%) of the global forecast error reduction afforded by the use of the full observing system over a system that uses only conventional observations. When forecast verification is performed against radiosonde observations, GPSRO is found to be the most valuable satellite observing system in the lower stratosphere. This is remarkable in view of the relative sparseness of the GPSRO spatial and temporal coverage and an indication of the potential improvements that a denser GPSRO observing network would be able to provide. The forecast impact of GPSRO observations is also evaluated in the context of a data denial assimilation experiment with respect to the full observing system. Results are found to be consistent with those from the reduced baseline observational network and also indicate a statistically significant positive impact on tropospheric synoptic skill scores.

Ensemble of data assimilations (EDA) methods have been shown to be able to provide flow-dependent estimates of analysis and background error statistics. For this reason, they potentially present a way to overcome one of the main limitations of current variational data assimilation systems. However, the limited number of ensemble members which can be realistically run in an operational context and the stochastic nature of the EDA approach lead to high levels of sampling noise in the relevant ensemble statistics. To answer this problem, an ‘objective filtering’ technique of the sample ensemble variances proposed by Raynaud et al. in 2009 has been implemented at ECMWF. In this article we present a comparison of the ability of ensemble data assimilation systems of different sizes (10 to 50 members) to represent flow-dependent background-error variances. In particular, the ensemble-based variances are examined in the cases of the severe storm Klaus (24 January 2009) over France and the Atlantic tropical hurricane Ike (1–14 September 2008). Our results show that, while a relatively small ensemble (10 members) can be sufficient to resolve the larger-scale error structures connected to an extratropical cyclogenesis, a larger ensemble is beneficial to resolve more localised anomalies like those connected with a hurricane. In this sense, the objective filtering technique provides a useful indication of the spatial scales the ensemble is able to resolve in a statistically robust way. The day-to-day variability of the ensemble statistics and how this affects the objective spatial filtering procedure are also examined. Our conclusion is that a time-independent implementation of the filter based on a climatology of truncation wave numbers results in more robust ensemble statistics estimates, and ultimately in improved forecast skill scores. Copyright © 2011 Royal Meteorological Society

The Ensemble Kalman Filter (EnKF) is likely to become a viable alternative to variational methods for the next generation of meteorological and oceanographic data assimilation systems. In this work we present results from real-data assimilation experiments using the CNMCA regional numerical weather prediction (NWP) forecasting system and compare them to the currently operational variational-based analysis. The set of observations used is the same as the one ingested in the operational data stream, with the exception of satellite radiances and scatterometer winds. Results show that the EnKF-based assimilation cycle is capable of producing analyses and forecasts of consistently superior skill in the root mean square error metric than CNMCA operational 3D-Var. One of the most important issues in EnKF implementations lies in the filter tendency to become underdispersive for practical ensemble sizes. To combat this problem a number of different parametrizations of the model error unaccounted for in the assimilation cycle have been proposed. In the CNMCA system a combination of adaptive multiplicative and additive background covariance inflations has been found to give adequate results and to be capable of avoiding filter divergence in extended assimilation trials. The additive component of the covariance inflation has been implemented through the use of scaled forecast differences. Following suggestions that ensemble square-root filters can violate the gaussianity assumption when used with nonlinear prognostic models, the statistical distribution of the forecast and analysis ensembles has been studied. No sign of the ensemble collapsing onto one or a few model states has been found, and the forecast and analysis ensembles appear to stay remarkably close to the assumed probability distribution functions. Copyright © 2010 Royal Meteorological Society

The ensemble Kalman filter (EnKF) has been widely tested as a possible candidate for the next generation of meteorological and oceanographic data assimilation algorithms. While a number of tests with models of varying realism have been successfully performed, the EnKF has been seldom evaluated in an operational regional NWP environment at realistic spatial resolution. In this work one particular EnKF implementation (Local Ensemble Transform Kalman Filter, LETKF) has been implemented and its performance evaluated in comparison with CNMCA operational 3D-Var. One of the most important issues in EnKF implementations lies in the filter tendency to become underdispersive for practical ensemble sizes. While multiplicative (or additive) covariance inflation has been used to deal with this problem, tuning its values is an expensive and possibly never-ending task. Following ideas from linear estimation theory, we test an adaptive estimation procedure to evaluate forecast covariance inflation factors and observation errors. Our results show that, differently from previous experiences, the online estimation technique can be successfully employed in a realistic state-of-the-art NWP system. More generally the LETKF analysis is shown to be of superior quality with respect to the operational 3D-Var and a likely candidate for its replacement in the not-too-distant future. Copyright © 2008 Royal Meteorological Society

The trend towards using flow-dependent, ensemble-based estimates of background-error covariances has been one of the main themes of atmospheric data assimilation research and development in recent years. In this work it is documented how flow-dependent ensemble information from the ECMWF ensemble of data assimilations (EDA) has gradually been incorporated into the B model which describes the background-error covariance matrix at the start of the ECMWF 4D-Var assimilation window. Starting with background-error variances for the balanced part of the control vector and observation quality control, the current article extends the flow-dependency to background-error variances for the unbalanced part of the control vector and for background-error correlation structures. The correlations are determined either online from previous days or from a hybrid of climatological and current cycle estimates. Each of these changes is shown to improve both the realism of the modelled B and the accuracy of the analysis and forecast fields produced by the 4D-Var assimilation cycle which makes use of the improved B. Finally, increasing the resolution at which the EDA 4D-Vars are run is shown to reduce the underestimation of the EDA-based error estimates.

The excellent forecasts made by ECMWF predicting the devastating landfall of Hurricane Sandy attracted a great deal of publicity and praise in the immediate aftermath of the event. The almost unprecedented and sudden “left hook” of the storm toward the coast of New Jersey was attributed to interactions with the large-scale atmospheric flow. This led to speculation that satellite observations may play an important role in the successful forecasting of this event. To investigate the role of satellite data a number of experiments have been performed at ECMWF where different satellite observations are deliberately withheld and forecasts of the hurricane rerun. Without observations from geostationary satellites the correct landfall of the storm is still reasonably well predicted albeit with a slight timing shift compared to the control forecast. On the other hand, without polar-orbiting satellites (which represent 90% of the volume of currently ingested observations) the ECMWF system would have given no useful guidance 4–5 days ahead that the storm would make landfall on the New Jersey coast. Instead the hurricane is predicted to stay well offshore in the Atlantic and hit the Maine coast 24 h later. If background errors estimated from the ECMWF Ensemble of Data Assimilations (EDA) are allowed to evolve and adapt to the depleted observing system, then some of the performance loss suffered by withholding polar satellite data can be recovered. The use of the appropriate EDA errors results in a more enhanced use of geostationary satellite observations, which partly compensates for the loss of polar satellite data.

Medium-range weather prediction has become more skillful over recent decades, but forecast centers still suffer from occasional very poor forecasts, which are often referred to as “dropouts” or “busts.” This study focuses on European Centre for Medium-Range Weather Forecasts (ECMWF) day-6 forecasts for Europe. Although busts are defined by gross scores, bust composites reveal a coherent “Rex type” blocking situation, with a high over northern Europe and a low over the Mediterranean. Initial conditions for these busts also reveal a coherent flow, but this is located over North America and involves a trough over the Rockies, with high convective available potential energy (CAPE) to its east. This flow type occurs in spring and is often associated with a Rossby wave train that has crossed the Pacific. A composite on this initial flow type displays enhanced day-6 random forecast errors and some-what enhanced ensemble forecast spread, indicating reduced inherent predictability. Mesoscale convective systems, as...

A hybrid assimilation system which uses sample statistics from an Ensemble of Data Assimilations (EDA) to estimate background error variances has been implemented at ECMWF. We show that the new system is beneficial in terms of deterministic forecast skill, provided that random and systematic errors in the estimation of variances are properly accounted for. The mechanisms through which EDA sample variances influence the deterministic analysis are clarified. An interesting aspect is that the use of flow-dependent variances alone is able to introduce a significant degree of flow dependency in the analysis increments. Further possible improvements and extensions to the current methodology are also discussed. Copyright © 2012 Royal Meteorological Society