Search

Systematic, routine, and comprehensive evaluation of Earth system models (ESMs) facilitates benchmarking improvement across model generations and identifying the strengths and weaknesses of different model configurations. By gauging the consistency between models and observations, this endeavor is becoming increasingly necessary to objectively synthesize the thousands of simulations contributed to the Coupled Model Intercomparison Project (CMIP) to date. The Program for Climate Model Diagnosis and Intercomparison (PCMDI) Metrics Package (PMP) is an open-source Python software package that provides quick-look objective comparisons of ESMs with one another and with observations. The comparisons include metrics of large- to global-scale climatologies, tropical inter-annual and intra-seasonal variability modes such as the El Niño–Southern Oscillation (ENSO) and Madden–Julian Oscillation (MJO), extratropical modes of variability, regional monsoons, cloud radiative feedbacks, and high-frequency characteristics of simulated precipitation, including its extremes. The PMP comparison results are produced using all model simulations contributed to CMIP6 and earlier CMIP phases. An important objective of the PMP is to document the performance of ESMs participating in the recent phases of CMIP, together with providing version-controlled information for all datasets, software packages, and analysis codes being used in the evaluation process. Among other purposes, this also enables modeling groups to assess performance changes during the ESM development cycle in the context of the error distribution of the multi-model ensemble. Quantitative model evaluation provided by the PMP can assist modelers in their development priorities. In this paper, we provide an overview of the PMP, including its latest capabilities, and discuss its future direction. [ABSTRACT FROM AUTHOR]

Systematic, routine, and comprehensive evaluation of Earth System Models (ESMs) facilitates benchmarking improvement across model generations and identifying the strengths and weaknesses of different model configurations. By gauging the consistency between models and observations, this endeavor is becoming increasingly necessary to objectively synthesize thousands of simulations contributed to the Coupled Model Intercomparison Project (CMIP) to date. The PCMDI Metrics Package (PMP) is an open-source Python software package that provides 'quick-look' objective comparisons of ESMs with one another and with observations. The comparisons include metrics of large- to global-scale climatologies, tropical inter-annual and intra-seasonal variability modes such as El Niño-Southern Oscillation (ENSO) and Madden-Julian Oscillation (MJO), extratropical modes of variability, regional monsoons, cloud radiative feedbacks, and high-frequency characteristics of simulated precipitation, including extremes. The PMP results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. An important priority of the PMP is to document evaluation statistics for all Historical and AMIP simulations submitted to recent phases of CMIP, providing version-controlled information for all data sets and software packages being used. Among other purposes, this also enables modeling groups to assess performance changes during the ESM development cycle in the context of the error distribution of the multi-model ensemble. In this paper, we present an overview of the PMP including its history to date, capabilities, recent updates, and future direction. [ABSTRACT FROM AUTHOR]

As the resolution of global Earth system models increases, regional-scale evaluations are becoming ever more important. This study presents a framework for quantifying precipitation distributions at regional scales and applies it to evaluate Coupled Model Intercomparison Project (CMIP) 5 and 6 models. We employ the Intergovernmental Panel on Climate Change (IPCC) sixth assessment report (AR6) climate reference regions over land and propose refinements to the oceanic regions based on the homogeneity of precipitation distribution characteristics. The homogeneous regions are identified as heavy-, moderate-, and light-precipitating areas by K -means clustering of Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (GPM) version 6 final run product (IMERG) precipitation frequency and amount distributions. With the global domain partitioned into 62 regions, including 46 land and 16 ocean regions, we apply 10 established precipitation distribution metrics. The collection includes metrics focused on the maximum peak, lower 10th percentile, and upper 90th percentile in precipitation amount and frequency distributions; the similarity between observed and modeled frequency distributions; an unevenness measure based on cumulative amount; average total intensity on all days with precipitation; and number of precipitating days each year. We apply our framework to 25 CMIP5 and 41 CMIP6 models, as well as six observation-based products of daily precipitation. Our results indicate that many CMIP5 and 6 models substantially overestimate the observed light-precipitation amount and frequency, as well as the number of precipitating days, especially over midlatitude regions outside of some land regions in the Americas and Eurasia. Improvement from CMIP5 to 6 is shown in some regions, especially in midlatitude regions, but it is not evident globally, and over the tropics most metrics point toward degradation. [ABSTRACT FROM AUTHOR]

A framework for quantifying precipitation distributions at regional scales is presented and applied to CMIP 5 and 6 models. We employ the IPCC AR6 climate reference regions over land and propose refinements to the oceanic regions based on the homogeneity of precipitation distribution characteristics. The homogeneous regions are identified as heavy, moderate, and light precipitating areas by K-means clustering of IMERG precipitation frequency and amount distributions. With the global domain partitioned into 62 regions, including 46 land and 16 ocean regions, we apply 10 established precipitation distribution metrics. The collection includes metrics focused on the maximum peak, lower 10th percentile, and upper 90th percentile in precipitation amount and frequency distributions, the similarity between observed and modeled frequency distributions, an unevenness measure based on cumulative amount, average total intensity on all days with precipitation, and number of precipitating days each year. We apply our framework to 25 CMIP5 and 41 CMIP6 models, and 6 observation-based products of daily precipitation. Our results indicate that many CMIP 5 and 6 models substantially overestimate the observed light precipitation amount and frequency as well as the number of precipitating days, especially over mid-latitude regions outside of some land regions in the Americas and Eurasia. Improvement from CMIP 5 to 6 is shown in some regions, especially in mid-latitude regions, but it is not evident globally, and over the tropics most metrics point toward over degradation. [ABSTRACT FROM AUTHOR]

Objective performance metrics that measure precipitation variability across time scales from subdaily to interannual are presented and applied to Historical simulations of Coupled Model Intercomparison Project phase 5 and 6 (CMIP5 and CMIP6) models. Three satellite-based precipitation estimates (IMERG, TRMM, and CMORPH) are used as reference data. We apply two independent methods to estimate temporal variability of precipitation and compare the consistency in their results. The first method is derived from power spectra analysis of 3-hourly precipitation, measuring forced variability by solar insolation (diurnal and annual cycles) and internal variability at different time scales (subdaily, synoptic, subseasonal, seasonal, and interannual). The second method is based on time averaging and facilitates estimating the seasonality of subdaily variability. Supporting the robustness of our metric, we find a near equivalence between the results obtained from the two methods when examining simulated-to-observed ratios over large domains (global, tropics, extratropics, land, or ocean). Additionally, we demonstrate that our model evaluation is not very sensitive to the discrepancies between observations. Our results reveal that CMIP5 and CMIP6 models in general overestimate the forced variability while they underestimate the internal variability, especially in the tropical ocean and higher-frequency variability. The underestimation of subdaily variability is consistent across different seasons. The internal variability is overall improved in CMIP6, but remains underestimated, and there is little evidence of improvement in forced variability. Increased horizontal resolution results in some improvement of internal variability at subdaily and synoptic time scales, but not at longer time scales. [ABSTRACT FROM AUTHOR]

Daily and sub‐daily precipitation statistics are investigated from three global model ensembles: (a) global storm‐resolving models (GSRMs) with typical horizontal resolutions of ∼4 km, (b) "high"‐resolution global models with typical resolutions of ∼50 km and (c) "standard"‐resolution global models with typical resolutions of ∼100 km. Compared to two satellite rainfall datasets, GSRMs convincingly exhibit superior performance for statistics of heavier rain rate events including their diurnal cycle, spatial propagation and the amount contributed by intense precipitation, but not for statistics of weaker or shorter duration precipitation. Both high‐ and standard‐resolution models fail to simulate the correct phase and amplitude of diurnal cycle of precipitation and the propagating convection in the Central US, but high‐resolution models show relative improvement in the distribution of precipitation frequency and amount, especially for intense precipitation. Plain Language Summary: With the increasing in computation power in recent years, global storm‐resolving models (GSRMs), which have ultra‐high horizontal resolutions of 1–5 km and are capable of simulating convective storms directly, are now feasible to produce simulations beyond a month. In this study, we investigated how well these GSRMs simulate daily and sub‐daily precipitation statistics, and compared their performance with coarser‐resolution models (∼25–500 km). We demonstrated that these ultra‐high‐resolution global models outperform coarser‐resolution models in various aspects, such as the diurnal cycle of precipitation, the tropical precipitation intensity distribution of more intense events, and the propagating convection in the Central US. Key Points: Daily and sub‐daily precipitation statistics from global storm‐resolving models (GSRMs) and coarser resolution global models are compared to observationsGSRMs show superior performance for statistics of more intense precipitation events including their diurnal cycle and spatial propagationGSRMs are not superior for statistics of weaker or shorter duration precipitation [ABSTRACT FROM AUTHOR]

The present study examines the characteristics of the Madden–Julian Oscillation (MJO) events represented in the Energy Exascale Earth System Model version 1 (E3SMv1), DOE's new Earth system model. The coupled E3SMv1 realistically simulates the eastward propagation of precipitation and moist static energy (MSE) anomalies associated with the MJO. As in observations, horizontal moisture advection and longwave radiative feedback are found to be the dominant processes in E3SMv1 that lead to the eastward movement and maintenance of the MJO MSE anomalies, respectively. Modulation of the diurnal cycle of precipitation in the Maritime Continent region by the MJO is also well represented in the model despite systematic biases in the magnitude and phase of the precipitation diurnal cycle. On the MJO impact over the midlatitude, E3SMv1 reasonably captures the pattern of the MJO teleconnections across the North Pacific and North America, with improvement in the performance in a high‐resolution version, despite the magnitude being a bit weaker than the observed. Regarding the interannual variability of the MJO, the El Niño‐Southern Oscillation (ENSO) modulation of the zonal extent of MJO's eastward propagation, as well as associated changes in the mean state moisture gradient in the tropical west Pacific, are well reproduced in the model. However, MJO in E3SMv1 exhibits no sensitivity to the Quasi‐Biennial Oscillation (QBO), with the MJO propagation characteristics being almost identical between easterly QBO and westerly QBO years. Processes that have been suggested as critical to MJO simulation are also examined by utilizing recently developed process‐oriented diagnostics. Plain Language Summary: The United States Department of Energy developed a new computer model that simulates Earth's climate systems, called Energy Exascale Earth System Model version 1 (E3SMv1). This study examines how well the model reproduces the characteristics of the Madden–Julian Oscillation (MJO), a tropical climate phenomenon that impacts weather and climate around the globe. We find that the strength and eastward movement of the MJO is realistically represented in the model. Variability of water vapor and radiation are the dominant processes for the MJO simulation, which agree well with the real‐world observations. Despite some unrealistic features, E3SMv1 successfully simulates the impact of the MJO on tropical precipitation at shorter than daily time scale and on large‐scale atmospheric circulation in the midlatitude. The model also exhibits realistic year‐to‐year changes in east‐west expansion of the MJO by the El Niño‐Southern Oscillation, while no noticeable changes can be detected when stratospheric wind reverses its direction over the equator in every 1 or 2 years. Key Points: Energy Exascale Earth System Model version 1 (E3SMv1) simulates Madden‐Julian Oscillation (MJOs) that exhibit realistic eastward propagation over the Indo‐Pacific warm poolModeled processes of the MJO, revealed through column‐integrated moist static energy anomalies, match well with those in observationsDespite the decent MJO simulation fidelity, the observed MJO–Quasi‐Biennial Oscillation coupling is not simulated in E3SMv1 [ABSTRACT FROM AUTHOR]

This study examines the role of the mean state in the propagation of the Madden‐Julian oscillation (MJO) over the Maritime Continent (MC). We use an ensemble of simulations made with a single model—the Community Earth System Model version 2—to assess the effect of the mean state that is unaffected by that of model components such as parameterization schemes. Results show that the background meridional moisture gradient is much steeper over the MC region in the periods with a stronger MJO propagation. Column water vapor budget of the MJO strongly suggests that the simulated mean state affects MJO via its impacts on moisture dynamics—a greater advection of mean moisture by MJO wind in the MC region is responsible for the anomalous MJO activity. Plain Language Summary: The Madden‐Julian oscillation (MJO) is a planetary‐scale, eastward moving envelope of anomalous convection in the tropics. It is the dominant mode of sub‐seasonal variability in the tropics. Unfortunately, an accurate representation of the MJO has historically been a challenging task for many, if not most, global climate models. The mean state distribution of atmospheric moisture has been highlighted as a key aspect affecting the simulation of MJO propagation in many recent modeling studies. When many different models are compared, however, it is difficult to isolate the role of the mean state because different models use different parameterizations of moist physics that affect both the mean state and the MJO directly. In this study, we examine the relationship between the mean state and MJO propagation in an ensemble of simulations made with a single coupled model—the Community Earth System Model version 2 (CESM2). Each ensemble member differs only in its initial conditions and thus the parameterizations and resolution are identical. We found that MJO propagation over the MC in CESM2 is strongly affected by the background meridional moisture gradient (MMG), with MJO propagation being enhanced in the periods with a steeper MMG. Key Points: Each ensemble in the Community Earth System Model version 2 historical simulation shows a marked low‐frequency variability in Madden‐Julian oscillation (MJO) propagation across the Maritime Continent (MC)Simulation periods with enhanced MJO propagation across the MC exhibit a steeper background meridional moisture gradient (MMG) around the MCThe steeper background MMG strengthens moisture recharging to the east of MJO, leading its eastward propagation [ABSTRACT FROM AUTHOR]

The Radiative‐Convective Equilibrium Model Intercomparison Project (RCEMIP) is an intercomparison of multiple types of numerical models configured in radiative‐convective equilibrium (RCE). RCE is an idealization of the tropical atmosphere that has long been used to study basic questions in climate science. Here, we employ RCE to investigate the role that clouds and convective activity play in determining cloud feedbacks, climate sensitivity, the state of convective aggregation, and the equilibrium climate. RCEMIP is unique among intercomparisons in its inclusion of a wide range of model types, including atmospheric general circulation models (GCMs), single column models (SCMs), cloud‐resolving models (CRMs), large eddy simulations (LES), and global cloud‐resolving models (GCRMs). The first results are presented from the RCEMIP ensemble of more than 30 models. While there are large differences across the RCEMIP ensemble in the representation of mean profiles of temperature, humidity, and cloudiness, in a majority of models anvil clouds rise, warm, and decrease in area coverage in response to an increase in sea surface temperature (SST). Nearly all models exhibit self‐aggregation in large domains and agree that self‐aggregation acts to dry and warm the troposphere, reduce high cloudiness, and increase cooling to space. The degree of self‐aggregation exhibits no clear tendency with warming. There is a wide range of climate sensitivities, but models with parameterized convection tend to have lower climate sensitivities than models with explicit convection. In models with parameterized convection, aggregated simulations have lower climate sensitivities than unaggregated simulations. Plain Language Summary: This study investigates tropical clouds and climate using results from more than 30 different numerical models set up in a simplified framework. The data set of model simulations is unique in that it includes a wide range of model types configured in a consistent manner. We address some of the biggest open questions in climate science, including how cloud properties change with warming and the role that the tendency of clouds to form clusters plays in determining the average climate and how climate changes. While there are large differences in how the different models simulate average temperature, humidity, and cloudiness, in a majority of models, the amount of high clouds decreases as climate warms. Nearly all models simulate a tendency for clouds to cluster together. There is agreement that when the clouds are clustered, the atmosphere is drier with fewer clouds overall. We do not find a conclusive result for how cloud clustering changes as the climate warms. Key Points: Temperature, humidity, and clouds in radiative‐convective equilibrium vary substantially across modelsModels agree that self‐aggregation dries the atmosphere and reduces high cloudinessThere is no consistency in how self‐aggregation depends on warming [ABSTRACT FROM AUTHOR]

Many climate models struggle with a poor simulation of the Madden‐Julian Oscillation (MJO), especially its propagation across the Maritime Continent (MC). This study quantitatively evaluates the robustness of MJO propagation over the MC in climate models that participated in Coupled Model Intercomparison Project Phase 5 (CMIP5) and Phase 6 (CMIP6) with a newly developed MC propagation metric. The results show that the CMIP6 models simulate MJO propagation over the MC more realistically than the CMIP5 models. Lower free‐tropospheric moisture budget analysis highlights that the greater horizontal moisture advection is responsible for the enhanced MJO propagation over the MC. The increase in horizontal moisture advection in the CMIP6 models is mainly attributed to the steeper horizontal mean state moisture gradient around the MC, which is associated with the reduction of the equatorial dry bias. Plain Language Summary: The Madden‐Julian Oscillation (MJO), planetary‐scale eastward propagating tropical convective cloud clusters coupled with large‐scale circulation, is the dominant mode of intraseasonal variability in the tropics and thereby influences a wide range of weather and climate phenomena. Unfortunately, however, many contemporary climate models struggle to simulate a realistic MJO propagation across the Maritime Continent, and this common bias had persisted over the previous generations of the Coupled Model Intercomparison Project (CMIP). We show that, in the newly released CMIP Phase 6 (CMIP6) models, the simulation of the MJO propagation is significantly improved when compared to their predecessors—CMIP Phase 5 (CMIP5) models. The improvement in the MJO simulation is mainly due to the reduction of the dry bias that many CMIP5 models exhibit over the Indo‐Pacific Warm Pool region. Key Points: A metric indicating the robustness of MJO propagation across the MC is developed and applied to 30 CMIP5 and 34 CMIP6 modelsCMIP6 models represent MJO propagation over the MC more realistically than the CMIP5 modelsThe improvement in MJO propagation is due to the steepening of the mean state moisture gradient around the MC [ABSTRACT FROM AUTHOR]

Modifications in cumulus parameterizations that improve Madden‐Julian oscillation (MJO) simulation tend to degrade the mean state—known as the MJO‐mean state trade‐off. The impacts of parameterizing mesoscale convective organization on the relationship between the MJO and the mean state simulation fidelity are examined. A series of experiments are made with a general circulation model that parameterizes the mesoscale convective organization and simulates well both the MJO and the mean state. In the control simulation, a prognostic nondimensional variable (Ω) represents the degree of convective organization. In order to examine the effect of the parameterized convective organization, fixed Ω values are imposed in a series of constrained experiments. The fixed‐Ω simulations show a negative relationship between the MJO and the mean state simulation fidelity. The control simulation is deviated from the negative relationship, suggesting the parameterized mesoscale convective organization helps general circulation models to mitigate the MJO‐mean state trade‐off. Plain Language Summary: Most general circulation models (GCMs) have a common problem that the modifications in cumulus parameterizations for improving Madden‐Julian oscillation (MJO) simulation tend to degrade mean state, which is known as the MJO‐mean state trade‐off. Understanding and overcoming this problem is important to develop next‐generation GCMs. It has been speculated that for a GCM to overcome the MJO‐mean state trade‐off, the convection scheme should represent the mesoscale convective organization, but most GCMs do not fully represent this process. Seoul National University Atmosphere Model Version 0 with a Unified Convection Scheme parameterizes the mesoscale convective organization and simulates well both the MJO and the mean state implying mitigation of the MJO‐mean state trade‐off. Using this model, the current study shows the importance of parameterized convective organization to mitigate the MJO‐mean state trade‐off by comparing the control simulation with a series of experiments in which the convective organization is constrained. Key Points: Effects of parameterized mesoscale convective organization are examined with a focus on the MJO and mean state simulationsThe experiments without parameterized mesoscale convective organization exhibit the MJO‐mean state trade‐offWith a parameterized mesoscale convective organization, both the MJO and the mean state are realistically simulated [ABSTRACT FROM AUTHOR]