Search

Adaptation to climate change requires robust climate projections, yet the uncertainty in these projections performed by ensembles of Earth system models (ESMs) remains large. This is mainly due to uncertainties in the representation of subgrid-scale processes such as turbulence or convection that are partly alleviated at higher resolution. New developments in machine learning-based hybrid ESMs demonstrate great potential for systematically reduced errors compared to traditional ESMs. Building on the work of hybrid (physics + AI) ESMs, we here discuss the additional potential of further improving and accelerating climate models with quantum computing. We discuss how quantum computers could accelerate climate models by solving the underlying differential equations faster, how quantum machine learning could better represent subgrid-scale phenomena in ESMs even with currently available noisy intermediate-scale quantum devices, how quantum algorithms aimed at solving optimisation problems could assist in tuning the many parameters in ESMs, a currently time-consuming and challenging process, and how quantum computers could aid in the analysis of climate models. We also discuss hurdles and obstacles facing current quantum computing paradigms. Strong interdisciplinary collaboration between climate scientists and quantum computing experts could help overcome these hurdles and harness the potential of quantum computing for this urgent topic.

Data assimilation (DA) provides more accurate, physically consistent analysis fields and is used for estimating initial conditions in numerical weather forecasting. Traditional DA methods derive statistically optimal analyses in model space based on Bayesian theory. However, their effectiveness is limited by the difficulty of accurately estimating the background error covariances matrix B, which represents the intricate interdependencies among atmospheric variables, as well as the standard linearity assumptions required during the assimilation step. To address these limitations, we propose Latent Data Assimilation (LDA) for a multi-variable global atmosphere, performing non-linear Machine-Learning based Bayesian DA on an atmospheric latent representation learned by an autoencoder. The feasibility of LDA is supported by the near-linear relationship between increments in latent space (within the typical magnitude range for DA) and their corresponding impacts in model space, ensuring that the optimal analysis obtained in latent space approximates the optimal analysis in model space. Due to the relationships among the atmospheric variables encoded in the latent space, LDA can physically propagate observation information across unobserved regions and atmospheric variables, even with a fully diagonal B in latent space. We perform idealized experiments with simulated observations and demonstrate the superiority of LDA over traditional DA methods in model space, while the experiments assimilating real observations highlight its potential application for operational reanalysis and weather forecasting systems.

Accurate modeling of terrestrial carbon and water exchange requires robust ecological parameters that capture vegetation responses and adaptations to the local environment. The current generation of land models use Plant Functional Types (PFTs) to discretize vegetation functional diversity, but these coarse categorizations often overlook fine-scale variations shaped by local climate, soil, and forest age factors. The lack of governing equations for plant adaptation demands a paradigm shift in how we integrate diverse Earth observations to uncover ecological functional dependence on changing environments. To address this challenge, we developed DifferLand, a differentiable, hybrid physics and machine learning model that infers the spatial distributions of ecological parameters and their relationships with environmental factors constrained by satellite and in-situ observations. Our model unifies top-down and bottom-up observational constraints with process-based knowledge to generate a global analysis of ecological functions and their adaptation to environmental gradients. We found PFTs account for less than half of the explainable spatial parameter variations controlling carbon fluxes and vegetation states. The remaining parameter variability is largely driven by local climate and forest demography factors, and the learned environment-parameter relationships lead to enhanced spatial generalization at unseen locations. DifferLand identified growing season length, leaf economics, and agricultural intensity as the three orthogonal spatial gradients underlying parameter variations. Our novel framework can lead to new insights on global carbon cycling by learning directly from data and expanding our understanding of local responses of ecosystems to environmental drivers.

Robust integration of physical knowledge and data is key to improve computational simulations, such as Earth system models. Data assimilation is crucial for achieving this goal because it provides a systematic framework to calibrate model outputs with observations, which can include remote sensing imagery and ground station measurements, with uncertainty quantification. Conventional methods, including Kalman filters and variational approaches, inherently rely on simplifying linear and Gaussian assumptions, and can be computationally expensive. Nevertheless, with the rapid adoption of data-driven methods in many areas of computational sciences, we see the potential of emulating traditional data assimilation with deep learning, especially generative models. In particular, the diffusion-based probabilistic framework has large overlaps with data assimilation principles: both allows for conditional generation of samples with a Bayesian inverse framework. These models have shown remarkable success in text-conditioned image generation or image-controlled video synthesis. Likewise, one can frame data assimilation as observation-conditioned state calibration. In this work, we propose SLAMS: Score-based Latent Assimilation in Multimodal Setting. Specifically, we assimilate in-situ weather station data and ex-situ satellite imagery to calibrate the vertical temperature profiles, globally. Through extensive ablation, we demonstrate that SLAMS is robust even in low-resolution, noisy, and sparse data settings. To our knowledge, our work is the first to apply deep generative framework for multimodal data assimilation using real-world datasets; an important step for building robust computational simulators, including the next-generation Earth system models. Our code is available at: https://github.com/yongquan-qu/SLAMS, Comment: Best Student Paper Award @ CVPR2024 EarthVision

Accurate representations of unknown and sub-grid physical processes through parameterizations (or closure) in numerical simulations with quantified uncertainty are critical for resolving the coarse-grained partial differential equations that govern many problems ranging from weather and climate prediction to turbulence simulations. Recent advances have seen machine learning (ML) increasingly applied to model these subgrid processes, resulting in the development of hybrid physics-ML models through the integration with numerical solvers. In this work, we introduce a novel framework for the joint estimation of physical parameters and machine learning parameterizations with uncertainty quantification. Our framework incorporates online training and efficient Bayesian inference within a high-dimensional parameter space, facilitated by differentiable programming. This proof of concept underscores the substantial potential of differentiable programming in synergistically combining machine learning with differential equations, thereby enhancing the capabilities of hybrid physics-ML modeling., Comment: Accepted at ICLR 2024 Workshop on AI4Differential Equations in Science

Earth system models (ESMs) are vital for understanding past, present, and future climate, but they suffer from legacy technical infrastructure. ESMs are primarily implemented in Fortran, a language that poses a high barrier of entry for early career scientists and lacks a GPU runtime, which has become essential for continued advancement as GPU power increases and CPU scaling slows. Fortran also lacks differentiability - the capacity to differentiate through numerical code - which enables hybrid models that integrate machine learning methods. Converting an ESM from Fortran to Python/JAX could resolve these issues. This work presents a semi-automated method for translating individual model components from Fortran to Python/JAX using a large language model (GPT-4). By translating the photosynthesis model from the Community Earth System Model (CESM), we demonstrate that the Python/JAX version results in up to 100x faster runtimes using GPU parallelization, and enables parameter estimation via automatic differentiation. The Python code is also easy to read and run and could be used by instructors in the classroom. This work illustrates a path towards the ultimate goal of making climate models fast, inclusive, and differentiable.

Deep learning is a powerful tool to represent subgrid processes in climate models, but many application cases have so far used idealized settings and deterministic approaches. Here, we develop stochastic parameterizations with calibrated uncertainty quantification to learn subgrid convective and turbulent processes and surface radiative fluxes of a superparameterization (SP) embedded in an Earth System Model (ESM). We explore three methods to construct stochastic parameterizations: 1) a single Deep Neural Network (DNN) with Monte Carlo Dropout; 2) a multi-member parameterization; and 3) a Variational Encoder Decoder with latent space perturbation. We show that the multi-member (MM) parameterization improves the representation of convective processes, especially in the planetary boundary layer, compared to individual DNNs. The respective uncertainty quantification illustrates that methods 2) and 3) are advantageous compared to a dropout-based DNN parameterization regarding the spread of convective processes. Hybrid simulations with our best-performing MM parameterizations remained challenging and crash within the first days. Therefore, we develop a pragmatic partial coupling strategy relying on the SP for condensate emulation. Partial coupling reduces the computational efficiency of hybrid Earth-like simulations but enables model stability over 5 months with our MM parameterizations. However, our hybrid simulations exhibit biases in thermodynamic fields and differences in precipitation patterns. Despite this, the MM parameterizations enable improvements in reproducing tropical extreme precipitation compared to a traditional convection parameterization. Despite these challenges, our results indicate the potential of a new generation of MM machine learning parameterizations leveraging uncertainty quantification to improve the representation of stochasticity of subgrid effects., Comment: main: 40 pages, 8 figures, 1 table; supporting information: 76 pages, 55 figures, 7 tables ; submitted to Journal of Advances in Modeling Earth Systems (JAMES)

Accurate prediction of climate in the subseasonal-to-seasonal scale is crucial for disaster preparedness and robust decision making amidst climate change. Yet, forecasting beyond the weather timescale is challenging because it deals with problems other than initial condition, including boundary interaction, butterfly effect, and our inherent lack of physical understanding. At present, existing benchmarks tend to have shorter forecasting range of up-to 15 days, do not include a wide range of operational baselines, and lack physics-based constraints for explainability. Thus, we propose ChaosBench, a challenging benchmark to extend the predictability range of data-driven weather emulators to S2S timescale. First, ChaosBench is comprised of variables beyond the typical surface-atmospheric ERA5 to also include ocean, ice, and land reanalysis products that span over 45 years to allow for full Earth system emulation that respects boundary conditions. We also propose physics-based, in addition to deterministic and probabilistic metrics, to ensure a physically-consistent ensemble that accounts for butterfly effect. Furthermore, we evaluate on a diverse set of physics-based forecasts from four national weather agencies as baselines to our data-driven counterpart such as ViT/ClimaX, PanguWeather, GraphCast, and FourCastNetV2. Overall, we find methods originally developed for weather-scale applications fail on S2S task: their performance simply collapse to an unskilled climatology. Nonetheless, we outline and demonstrate several strategies that can extend the predictability range of existing weather emulators, including the use of ensembles, robust control of error propagation, and the use of physics-informed models. Our benchmark, datasets, and instructions are available at https://leap-stc.github.io/ChaosBench., Comment: NeurIPS 2024 D&B Track (Oral)

The increasing size and severity of wildfires across the western United States have generated dangerous levels of PM$_{2.5}$ concentrations in recent years. In a changing climate, expanding the use of prescribed fires is widely considered to be the most robust fire mitigation strategy. However, reliably forecasting the potential air quality impact from prescribed fires, which is critical in planning the prescribed fires' location and time, at hourly to daily time scales remains a challenging problem. In this paper, we introduce a spatial-temporal graph neural network (GNN) based forecasting model for hourly PM$_{2.5}$ predictions across California. Using a two-step approach, we leverage our forecasting model to estimate the PM$_{2.5}$ contribution of wildfires. Integrating the GNN-based PM$_{2.5}$ forecasting model with prescribed fire simulations, we propose a novel framework to forecast the PM$_{2.5}$ pollution of prescribed fires. This framework helps determine March as the optimal month for implementing prescribed fires in California and quantifies the potential air quality trade-offs involved in conducting more prescribed fires outside the fire season., Comment: 10 pages; multiple figures; matches version submitted to Environmental Data Science

Satellite-observed solar-induced chlorophyll fluorescence (SIF) is a powerful proxy for diagnosing the photosynthetic characteristics of terrestrial ecosystems. Despite the increasing spatial and temporal resolutions of these satellite retrievals, records of SIF are primarily limited to the recent decade, impeding their application in detecting long-term dynamics of ecosystem function and structure. In this study, we leverage the two surface reflectance bands (red and near-infrared) available both from Advanced Very High-Resolution Radiometer (AVHRR, 1982-2022) and MODerate-resolution Imaging Spectroradiometer (MODIS, 2001-2022). Importantly, we calibrate and orbit-correct the AVHRR bands against their MODIS counterparts during their overlapping period. Using the long-term bias-corrected reflectance data, a neural network is then built to reproduce the Orbiting Carbon Observatory-2 SIF using AVHRR and MODIS, and used to map SIF globally over the entire 1982-2022 period. Compared with the previous MODIS-based CSIF product relying on four reflectance bands, our two-band-based product has similar skill but can be advantageously extended to the bias-corrected AVHRR period. Further comparison with three widely used vegetation indices (NDVI, kNDVI, NIRv; all based empirically on red and near-infrared bands) shows a higher or comparable correlation of LCSIF with satellite SIF and site-level GPP estimates across vegetation types, ensuring a greater capacity of LCSIF for representing terrestrial photosynthesis. Globally, LCSIF-AVHRR shows an accelerating upward trend since 1982, with an average rate of 0.0025 mW m-2 nm-1 sr-1 per decade during 1982-2000 and 0.0038 mW m-2 nm-1 sr-1 per decade during 2001-2022. Our LCSIF data provide opportunities to better understand the long-term dynamics of ecosystem photosynthesis and their underlying driving processes.

Machine learning (ML)-based parameterizations have been developed for Earth System Models (ESMs) with the goal to better represent subgrid-scale processes or to accelerate computations. ML-based parameterizations within hybrid ESMs have successfully learned subgrid-scale processes from short high-resolution simulations. However, most studies used a particular ML method to parameterize the subgrid tendencies or fluxes originating from the compound effect of various small-scale processes (e.g., radiation, convection, gravity waves) in mostly idealized settings or from superparameterizations. Here, we use a filtering technique to explicitly separate convection from these processes in simulations with the Icosahedral Non-hydrostatic modelling framework (ICON) in a realistic setting and benchmark various ML algorithms against each other offline. We discover that an unablated U-Net, while showing the best offline performance, learns reverse causal relations between convective precipitation and subgrid fluxes. While we were able to connect the learned relations of the U-Net to physical processes this was not possible for the non-deep learning-based Gradient Boosted Trees. The ML algorithms are then coupled online to the host ICON model. Our best online performing model, an ablated U-Net excluding precipitating tracer species, indicates higher agreement for simulated precipitation extremes and mean with the high-resolution simulation compared to the traditional scheme. However, a smoothing bias is introduced both in water vapor path and mean precipitation. Online, the ablated U-Net significantly improves stability compared to the non-ablated U-Net and runs stable for the full simulation period of 180 days. Our results hint to the potential to significantly reduce systematic errors with hybrid ESMs.

Machine-learning (ML) parameterizations of subgrid processes (here of turbulence, convection, and radiation) may one day replace conventional parameterizations by emulating high-resolution physics without the cost of explicit simulation. However, uncertainty about the relationship between offline and online performance (i.e., when integrated with a large-scale general circulation model (GCM)) hinders their development. Much of this uncertainty stems from limited sampling of the noisy, emergent effects of upstream ML design decisions on downstream online hybrid simulation. Our work rectifies the sampling issue via the construction of a semi-automated, end-to-end pipeline for $\mathcal{O}(100)$ size ensembles of hybrid simulations, revealing important nuances in how systematic reductions in offline error manifest in changes to online error and online stability. For example, removing dropout and switching from a Mean Squared Error (MSE) to a Mean Absolute Error (MAE) loss both reduce offline error, but they have opposite effects on online error and online stability. Other design decisions, like incorporating memory, converting moisture input from specific humidity to relative humidity, using batch normalization, and training on multiple climates do not come with any such compromises. Finally, we show that ensemble sizes of $\mathcal{O}(100)$ may be necessary to reliably detect causally relevant differences online. By enabling rapid online experimentation at scale, we can empirically settle debates regarding subgrid ML parameterization design that would have otherwise remained unresolved in the noise., Comment: Main text: 21 pages, 5 figures. SI: 24 pages, 35 figures

Precise and reliable climate projections are required for climate adaptation and mitigation, but Earth system models still exhibit great uncertainties. Several approaches have been developed to reduce the spread of climate projections and feedbacks, yet those methods cannot capture the non-linear complexity inherent in the climate system. Using a Transfer Learning approach, we show that Machine Learning can be used to optimally leverage and merge the knowledge gained from Earth system models simulations and historical observations to reduce the spread of global surface air temperature fields projected in the 21st century. We reach an uncertainty reduction of more than 50% with respect to state-of-the-art approaches, while giving evidence that our novel method provides improved regional temperature patterns together with narrower projections uncertainty, urgently required for climate adaptation.

Machine-learning-based parameterizations (i.e. representation of sub-grid processes) of global climate models or turbulent simulations have recently been proposed as a powerful alternative to physical, but empirical, representations, offering a lower computational cost and higher accuracy. Yet, those approaches still suffer from a lack of generalization and extrapolation beyond the training data, which is however critical to projecting climate change or unobserved regimes of turbulence. Here we show that a multi-fidelity approach, which integrates datasets of different accuracy and abundance, can provide the best of both worlds: the capacity to extrapolate leveraging the physically-based parameterization and a higher accuracy using the machine-learning-based parameterizations. In an application to climate modeling, the multi-fidelity framework yields more accurate climate projections without requiring major increase in computational resources. Our multi-fidelity randomized prior networks (MF-RPNs) combine physical parameterization data as low-fidelity and storm-resolving historical run's data as high-fidelity. To extrapolate beyond the training data, the MF-RPNs are tested on high-fidelity warming scenarios, $+4K$, data. We show the MF-RPN's capacity to return much more skillful predictions compared to either low- or high-fidelity (historical data) simulations trained only on one regime while providing trustworthy uncertainty quantification across a wide range of scenarios. Our approach paves the way for the use of machine-learning based methods that can optimally leverage historical observations or high-fidelity simulations and extrapolate to unseen regimes such as climate change., Comment: 27 pages, 16 figures

Modern climate projections lack adequate spatial and temporal resolution due to computational constraints, leading to inaccuracies in representing critical processes like thunderstorms that occur on the sub-resolution scale. Hybrid methods combining physics with machine learning (ML) offer faster, higher fidelity climate simulations by outsourcing compute-hungry, high-resolution simulations to ML emulators. However, these hybrid ML-physics simulations require domain-specific data and workflows that have been inaccessible to many ML experts. As an extension of the ClimSim dataset (Yu et al., 2024), we present ClimSim-Online, which also includes an end-to-end workflow for developing hybrid ML-physics simulators. The ClimSim dataset includes 5.7 billion pairs of multivariate input/output vectors, capturing the influence of high-resolution, high-fidelity physics on a host climate simulator's macro-scale state. The dataset is global and spans ten years at a high sampling frequency. We provide a cross-platform, containerized pipeline to integrate ML models into operational climate simulators for hybrid testing. We also implement various ML baselines, alongside a hybrid baseline simulator, to highlight the ML challenges of building stable, skillful emulators. The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res) and code (https://leap-stc.github.io/ClimSim and https://github.com/leap-stc/climsim-online) are publicly released to support the development of hybrid ML-physics and high-fidelity climate simulations., Comment: This manuscript is an expanded version of our paper that received the Outstanding Paper Award at the NeurIPS 2023 conference

Climate models are essential to understand and project climate change, yet long-standing biases and uncertainties in their projections remain. This is largely associated with the representation of subgrid-scale processes, particularly clouds and convection. Deep learning can learn these subgrid-scale processes from computationally expensive storm-resolving models while retaining many features at a fraction of computational cost. Yet, climate simulations with embedded neural network parameterizations are still challenging and highly depend on the deep learning solution. This is likely associated with spurious non-physical correlations learned by the neural networks due to the complexity of the physical dynamical system. Here, we show that the combination of causality with deep learning helps removing spurious correlations and optimizing the neural network algorithm. To resolve this, we apply a causal discovery method to unveil causal drivers in the set of input predictors of atmospheric subgrid-scale processes of a superparameterized climate model in which deep convection is explicitly resolved. The resulting causally-informed neural networks are coupled to the climate model, hence, replacing the superparameterization and radiation scheme. We show that the climate simulations with causally-informed neural network parameterizations retain many convection-related properties and accurately generate the climate of the original high-resolution climate model, while retaining similar generalization capabilities to unseen climates compared to the non-causal approach. The combination of causal discovery and deep learning is a new and promising approach that leads to stable and more trustworthy climate simulations and paves the way towards more physically-based causal deep learning approaches also in other scientific disciplines.

A promising method for improving the representation of clouds in climate models, and hence climate projections, is to develop machine learning-based parameterizations using output from global storm-resolving models. While neural networks can achieve state-of-the-art performance within their training distribution, they can make unreliable predictions outside of it. Additionally, they often require post-hoc tools for interpretation. To avoid these limitations, we combine symbolic regression, sequential feature selection, and physical constraints in a hierarchical modeling framework. This framework allows us to discover new equations diagnosing cloud cover from coarse-grained variables of global storm-resolving model simulations. These analytical equations are interpretable by construction and easily transferable to other grids or climate models. Our best equation balances performance and complexity, achieving a performance comparable to that of neural networks ($R^2=0.94$) while remaining simple (with only 11 trainable parameters). It reproduces cloud cover distributions more accurately than the Xu-Randall scheme across all cloud regimes (Hellinger distances $<0.09$), and matches neural networks in condensate-rich regimes. When applied and fine-tuned to the ERA5 reanalysis, the equation exhibits superior transferability to new data compared to all other optimal cloud cover schemes. Our findings demonstrate the effectiveness of symbolic regression in discovering interpretable, physically-consistent, and nonlinear equations to parameterize cloud cover., Comment: 35 pages, 10 figures, Submitted to 'Journal of Advances in Modeling Earth Systems' (JAMES)

Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage large datasets. ML methods, especially deep networks, presented strong predictive skills yet lacked the ability to answer specific scientific questions. While various methods have been proposed for ML-physics integration, an important underlying theme -- differentiable modeling -- is not sufficiently recognized. Here we outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG). "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables, critically enabling the learning of high-dimensional unknown relationships. DG refers to a range of methods connecting varying amounts of prior knowledge to neural networks and training them together, capturing a different scope than physics-guided machine learning and emphasizing first principles. Preliminary evidence suggests DG offers better interpretability and causality than ML, improved generalizability and extrapolation capability, and strong potential for knowledge discovery, while approaching the performance of purely data-driven ML. DG models require less training data while scaling favorably in performance and efficiency with increasing amounts of data. With DG, geoscientists may be better able to frame and investigate questions, test hypotheses, and discover unrecognized linkages.

In this article, we consider the problem of estimating fractional processes based on noisy high-frequency data. Generalizing the idea of pre-averaging to a fractional setting, we exhibit a sequence of consistent estimators for the unknown parameters of interest by proving a law of large numbers for associated variation functionals. In contrast to the semimartingale setting, the optimal window size for pre-averaging depends on the unknown roughness parameter of the underlying process. We evaluate the performance of our estimators in a simulation study and use them to empirically verify Kolmogorov's 2/3-law in turbulence data contaminated by instrument noise.

Physical parameterizations are used as representations of unresolved subgrid processes within weather and global climate models or coarse-scale turbulent models, whose resolutions are too coarse to resolve small-scale processes. These parameterizations are typically grounded on physically-based, yet empirical, representations of the underlying small-scale processes. Machine learning-based parameterizations have recently been proposed as an alternative and have shown great promises to reduce uncertainties associated with small-scale processes. Yet, those approaches still show some important mismatches that are often attributed to stochasticity in the considered process. This stochasticity can be due to noisy data, unresolved variables or simply to the inherent chaotic nature of the process. To address these issues, we develop a new type of parameterization (closure) which is based on a Bayesian formalism for neural networks, to account for uncertainty quantification, and includes memory, to account for the non-instantaneous response of the closure. To overcome the curse of dimensionality of Bayesian techniques in high-dimensional spaces, the Bayesian strategy is based on a Hamiltonian Monte Carlo Markov Chain sampling strategy that takes advantage of the likelihood function and kinetic energy's gradients with respect to the parameters to accelerate the sampling process. We apply the proposed Bayesian history-based parameterization to the Lorenz '96 model in the presence of noisy and sparse data, similar to satellite observations, and show its capacity to predict skillful forecasts of the resolved variables while returning trustworthy uncertainty quantifications for different sources of error. This approach paves the way for the use of Bayesian approaches for closure problems.

We constructed a frequently updated, near-real-time global power generation dataset: Carbon Monitor-Power since January, 2016 at national levels with near-global coverage and hourly-to-daily time resolution. The data presented here are collected from 37 countries across all continents for eight source groups, including three types of fossil sources (coal, gas, and oil), nuclear energy and four groups of renewable energy sources (solar energy, wind energy, hydro energy and other renewables including biomass, geothermal, etc.). The global near-real-time power dataset shows the dynamics of the global power system, including its hourly, daily, weekly and seasonal patterns as influenced by daily periodical activities, weekends, seasonal cycles, regular and irregular events (i.e., holidays) and extreme events (i.e., the COVID-19 pandemic). The Carbon Monitor-Power dataset reveals that the COVID-19 pandemic caused strong disruptions in some countries (i.e., China and India), leading to a temporary or long-lasting shift to low carbon intensity, while it had only little impact in some other countries (i.e., Australia). This dataset offers a large range of opportunities for power-related scientific research and policy-making.

Global Storm-Resolving Models (GSRMs) have gained widespread interest because of the unprecedented detail with which they resolve the global climate. However, it remains difficult to quantify objective differences in how GSRMs resolve complex atmospheric formations. This lack of comprehensive tools for comparing model similarities is a problem in many disparate fields that involve simulation tools for complex data. To address this challenge we develop methods to estimate distributional distances based on both nonlinear dimensionality reduction and vector quantization. Our approach automatically learns physically meaningful notions of similarity from low-dimensional latent data representations that the different models produce. This enables an intercomparison of nine GSRMs based on their high-dimensional simulation data (2D vertical velocity snapshots) and reveals that only six are similar in their representation of atmospheric dynamics. Furthermore, we uncover signatures of the convective response to global warming in a fully unsupervised way. Our study provides a path toward evaluating future high-resolution simulation data more objectively., Comment: 26 pages, 21 figures. In revision at Scientific Reports

Evapotranspiration (ET) represents the largest water loss flux in drylands, but ET and its partition into plant transpiration (T) and soil evaporation (E) are poorly quantified, especially at fine temporal scales. Physically-based remote sensing models relying on sensible heat flux estimates, like the two-source energy balance model, could benefit from considering more explicitly the key effect of stomatal regulation on dryland ET. The objective of this study is to assess the value of solar-induced chlorophyll fluorescence (SIF), a proxy for photosynthesis, to constrain the canopy conductance (Gc) of an optimal stomatal model within a two-source energy balance model in drylands. We assessed our ET estimation using in situ eddy covariance GPP as a benchmark, and compared with results from using the Contiguous solar-induced chlorophyll fluorescence (CSIF) remote sensing product instead of GPP, with and without the effect of root-zone soil moisture on the Gc. The estimated ET was robust across four steppes and two tree-grass dryland ecosystem. Comparison of ET simulated against in situ GPP yielded an average R2 of 0.73 (0.86) and RMSE of 0.031 (0.36) mm at half-hourly (daily) timescale. Including explicitly the soil moisture effect on Gc, increased the R2 to 0.76 (0.89). For the CSIF model, the average R2 for ET estimates also improved when including the effect of soil moisture: from 0.65 (0.79) to 0.71 (0.84), with RMSE ranging between 0.023 (0.22) and 0.043 (0.54) mm depending on the site. Our results demonstrate the capacity of SIF to estimate subdaily and daily ET fluxes under very low ET conditions. SIF can provide effective vegetation signals to constrain stomatal conductance and partition ET into T and E in drylands. This approach could be extended for regional estimates using remote sensing SIF estimates such as CSIF, TROPOMI-SIF, or the upcoming FLEX mission, among others.