Search

Full-complexity Earth system models (ESMs) are computationally very expensive, limiting their use in exploring the climate outcomes of multiple emission pathways. More efficient emulators that approximate ESMs can directly map emissions onto climate outcomes, and benchmarks are being used to evaluate their accuracy on standardized tasks and datasets. We investigate a popular benchmark in data-driven climate emulation, ClimateBench, on which deep learning-based emulators are currently achieving the best performance. We implement a linear regression-based emulator, akin to pattern scaling, and find that it outperforms the incumbent 100M-parameter deep learning foundation model, ClimaX, on 3 out of 4 regionally-resolved surface-level climate variables. While emulating surface temperature is expected to be predominantly linear, this result is surprising for emulating precipitation. We identify that this outcome is a result of high levels of internal variability in the benchmark targets. To address internal variability, we update the benchmark targets with ensemble averages from the MPI-ESM1.2-LR model that contain 50 instead of 3 climate simulations per emission pathway. Using the new targets, we show that linear pattern scaling continues to be more accurate on temperature, but can be outperformed by a deep learning-based model for emulating precipitation. We publish our code, data, and an interactive tutorial at github.com/blutjens/climate-emulator.

Gyrokinetic and kinetic-MHD simulations are performed for the fishbone instability in the DIII-D discharge #178631, chosen for validation of first-principles simulations to predict the energetic particle (EP) transport in an ITER prefusion baseline scenario. Fishbone modes are found to generate zonal flows, which dominate the fishbone saturation. The underlying mechanisms of the two-way fishbone-zonal flows nonlinear interplay are discussed in details. Numerical and analytical analyses identify the fishbone-induced EP redistribution as the dominant generation mechanism for zonal flows. The zonal flows modify the nonlinear dynamics of phase space zonal structures, which reduces the amount of EPs able to resonate with the mode, leading to an early fishbone saturation. Simulation results including zonal flows agree quantitatively with DIII-D experimental measurements of the fishbone saturation amplitude and EP transport, supporting this novel saturation mechanism by self-generated zonal flows. Moreover, the wave-particle mode-locking mechanism is shown to determine quantitatively the fishbone frequency down-chirping, as evident in GTC simulation results in agreement with predictions from analytical theory. Finally, the fishbone-induced zonal flows are possibly responsible for the formation of an ion-ITB in the DIII-D discharge. Based on the low EP transport and the large zonal flow shearing rates associated with the fishbone instability in gyrokinetic simulations of the ITER scenario, it is conjectured that high performance scenarios could be designed in ITER burning plasmas through fishbone-induced ITBs.

Modern climate projections lack adequate spatial and temporal resolution due to computational constraints, leading to inaccuracies in representing critical processes like thunderstorms that occur on the sub-resolution scale. Hybrid methods combining physics with machine learning (ML) offer faster, higher fidelity climate simulations by outsourcing compute-hungry, high-resolution simulations to ML emulators. However, these hybrid ML-physics simulations require domain-specific data and workflows that have been inaccessible to many ML experts. As an extension of the ClimSim dataset (Yu et al., 2024), we present ClimSim-Online, which also includes an end-to-end workflow for developing hybrid ML-physics simulators. The ClimSim dataset includes 5.7 billion pairs of multivariate input/output vectors, capturing the influence of high-resolution, high-fidelity physics on a host climate simulator's macro-scale state. The dataset is global and spans ten years at a high sampling frequency. We provide a cross-platform, containerized pipeline to integrate ML models into operational climate simulators for hybrid testing. We also implement various ML baselines, alongside a hybrid baseline simulator, to highlight the ML challenges of building stable, skillful emulators. The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res) and code (https://leap-stc.github.io/ClimSim and https://github.com/leap-stc/climsim-online) are publicly released to support the development of hybrid ML-physics and high-fidelity climate simulations., Comment: This manuscript is an expanded version of our paper that received the Outstanding Paper Award at the NeurIPS 2023 conference

Recent progress in self-supervision has shown that pre-training large neural networks on vast amounts of unsupervised data can lead to substantial increases in generalization to downstream tasks. Such models, recently coined foundation models, have been transformational to the field of natural language processing. Variants have also been proposed for image data, but their applicability to remote sensing tasks is limited. To stimulate the development of foundation models for Earth monitoring, we propose a benchmark comprised of six classification and six segmentation tasks, which were carefully curated and adapted to be both relevant to the field and well-suited for model evaluation. We accompany this benchmark with a robust methodology for evaluating models and reporting aggregated results to enable a reliable assessment of progress. Finally, we report results for 20 baselines to gain information about the performance of existing models. We believe that this benchmark will be a driver of progress across a variety of Earth monitoring tasks., Comment: arXiv admin note: text overlap with arXiv:2112.00570

Computer vision can accelerate ecology research by automating the analysis of raw imagery from sensors like camera traps, drones, and satellites. However, computer vision is an emerging discipline that is rarely taught to ecologists. This work discusses our experience teaching a diverse group of ecologists to prototype and evaluate computer vision systems in the context of an intensive hands-on summer workshop. We explain the workshop structure, discuss common challenges, and propose best practices. This document is intended for computer scientists who teach computer vision across disciplines, but it may also be useful to ecologists or other domain experts who are learning to use computer vision themselves.

Gyrokinetic simulations of the fishbone instability in DIII-D tokamak plasmas find that self-generated zonal flows can dominate the nonlinear saturation by preventing coherent structures from persisting or drifting in the energetic particle phase space when the mode frequency down-chirps. Results from the simulation with zonal flows agree quantitatively, for the first time, with experimental measurements of the fishbone saturation amplitude and energetic particle transport. Moreover, the fishbone-induced zonal flows are likely responsible for the formation of an internal transport barrier that was observed after fishbone bursts in this DIII-D experiment. Finally, gyrokinetic simulations of a related ITER baseline scenario show that the fishbone induces insignificant energetic particle redistribution and may enable high performance scenarios in ITER burning plasma experiments., Comment: Accepted in Physical Review Letters : https://journals.aps.org/prl/accepted/b0073Yf2Q4c1a78cb8611348c9fa5932b12922776

Numerical simulations in climate, chemistry, or astrophysics are computationally too expensive for uncertainty quantification or parameter-exploration at high-resolution. Reduced-order or surrogate models are multiple orders of magnitude faster, but traditional surrogates are inflexible or inaccurate and pure machine learning (ML)-based surrogates too data-hungry. We propose a hybrid, flexible surrogate model that exploits known physics for simulating large-scale dynamics and limits learning to the hard-to-model term, which is called parametrization or closure and captures the effect of fine- onto large-scale dynamics. Leveraging neural operators, we are the first to learn grid-independent, non-local, and flexible parametrizations. Our \textit{multiscale neural operator} is motivated by a rich literature in multiscale modeling, has quasilinear runtime complexity, is more accurate or flexible than state-of-the-art parametrizations and demonstrated on the chaotic equation multiscale Lorenz96., Comment: Presented at International Conference on Machine Learning Workshop AI for Science, 2022

Forest biomass is a key influence for future climate, and the world urgently needs highly scalable financing schemes, such as carbon offsetting certifications, to protect and restore forests. Current manual forest carbon stock inventory methods of measuring single trees by hand are time, labour, and cost-intensive and have been shown to be subjective. They can lead to substantial overestimation of the carbon stock and ultimately distrust in forest financing. The potential for impact and scale of leveraging advancements in machine learning and remote sensing technologies is promising but needs to be of high quality in order to replace the current forest stock protocols for certifications. In this paper, we present ReforesTree, a benchmark dataset of forest carbon stock in six agro-forestry carbon offsetting sites in Ecuador. Furthermore, we show that a deep learning-based end-to-end model using individual tree detection from low cost RGB-only drone imagery is accurately estimating forest carbon stock within official carbon offsetting certification standards. Additionally, our baseline CNN model outperforms state-of-the-art satellite-based forest biomass and carbon stock estimates for this type of small-scale, tropical agro-forestry sites. We present this dataset to encourage machine learning research in this area to increase accountability and transparency of monitoring, verification and reporting (MVR) in carbon offsetting projects, as well as scaling global reforestation financing through accurate remote sensing., Comment: Accepted paper for the AI for Social Impact Track at the AAAI 2022

Recent progress in self-supervision shows that pre-training large neural networks on vast amounts of unsupervised data can lead to impressive increases in generalisation for downstream tasks. Such models, recently coined as foundation models, have been transformational to the field of natural language processing. While similar models have also been trained on large corpuses of images, they are not well suited for remote sensing data. To stimulate the development of foundation models for Earth monitoring, we propose to develop a new benchmark comprised of a variety of downstream tasks related to climate change. We believe that this can lead to substantial improvements in many existing applications and facilitate the development of new applications. This proposal is also a call for collaboration with the aim of developing a better evaluation process to mitigate potential downsides of foundation models for Earth monitoring.

Developing fast and accurate surrogates for physics-based coastal and ocean models is an urgent need due to the coastal flood risk under accelerating sea level rise, and the computational expense of deterministic numerical models. For this purpose, we develop the first digital twin of Earth coastlines with new physics-informed machine learning techniques extending the state-of-art Neural Operator. As a proof-of-concept study, we built Fourier Neural Operator (FNO) surrogates on the simulations of an industry-standard flood and ocean model (NEMO). The resulting FNO surrogate accurately predicts the sea surface height in most regions while achieving upwards of 45x acceleration of NEMO. We delivered an open-source \textit{CoastalTwin} platform in an end-to-end and modular way, to enable easy extensions to other simulations and ML-based surrogate methods. Our results and deliverable provide a promising approach to massively accelerate coastal dynamics simulators, which can enable scientists to efficiently execute many simulations for decision-making, uncertainty quantification, and other research activities.

Verification and validation of the internal kink instability in tokamak have been performed for both gyrokinetic (GTC) and kinetic-MHD codes (GAM-solver, M3D-C1-K, NOVA, XTOR-K). Using realistic magnetic geometry and plasma profiles from the same equilibrium reconstruction of the DIII-D shot #141216, these codes exhibit excellent agreement for the growth rate and mode structure of the internal kink mode when all kinetic effects are suppresed. The simulated radial mode structures agree quantitatively with the electron cyclotron emission measurement after adjusting, within the experimental uncertainty, the safety factor q=1 flux-surface location in the equilibrium reconstruction. Compressible magnetic perturbations strongly destabilize the kink, while poloidal variations of the equilibrium current density reduce the growth rate of the kink. Furthermore, kinetic effects of thermal ions are found to decrease the kink growth rate in kinetic-MHD simulations, but increase the kink growth rate in gyrokinetic simulations, due to the additional drive of the ion temperature gradient and parallel electric field. Kinetic thermal electrons are found to have negligible effects on the internal kink instability., Comment: 16 pages, 10 figures

The transition to green energy grids depends on detailed wind and solar forecasts to optimize the siting and scheduling of renewable energy generation. Operational forecasts from numerical weather prediction models, however, only have a spatial resolution of 10 to 20-km, which leads to sub-optimal usage and development of renewable energy farms. Weather scientists have been developing super-resolution methods to increase the resolution, but often rely on simple interpolation techniques or computationally expensive differential equation-based models. Recently, machine learning-based models, specifically the physics-informed resolution-enhancing generative adversarial network (PhIREGAN), have outperformed traditional downscaling methods. We provide a thorough and extensible benchmark of leading deep learning-based super-resolution techniques, including the enhanced super-resolution generative adversarial network (ESRGAN) and an enhanced deep super-resolution (EDSR) network, on wind and solar data. We accompany the benchmark with a novel public, processed, and machine learning-ready dataset for benchmarking super-resolution methods on wind and solar data.

Forest carbon offsets are increasingly popular and can play a significant role in financing climate mitigation, forest conservation, and reforestation. Measuring how much carbon is stored in forests is, however, still largely done via expensive, time-consuming, and sometimes unaccountable field measurements. To overcome these limitations, many verification bodies are leveraging machine learning (ML) algorithms to estimate forest carbon from satellite or aerial imagery. Aerial imagery allows for tree species or family classification, which improves the satellite imagery-based forest type classification. However, aerial imagery is significantly more expensive to collect and it is unclear by how much the higher resolution improves the forest carbon estimation. This proposal paper describes the first systematic comparison of forest carbon estimation from aerial imagery, satellite imagery, and ground-truth field measurements via deep learning-based algorithms for a tropical reforestation project. Our initial results show that forest carbon estimates from satellite imagery can overestimate above-ground biomass by up to 10-times for tropical reforestation projects. The significant difference between aerial and satellite-derived forest carbon measurements shows the potential for aerial imagery-based ML algorithms and raises the importance to extend this study to a global benchmark between options for carbon measurements., Comment: Spotlight talk at the Tackling Climate Change with Machine Learning workshop at the ICML 2021 https://www.climatechange.ai/papers/icml2021/79

Climate models project an uncertainty range of possible warming scenarios from 1.5 to 5 degree Celsius global temperature increase until 2100, according to the CMIP6 model ensemble. Climate risk management and infrastructure adaptation requires the accurate quantification of the uncertainties at the local level. Ensembles of high-resolution climate models could accurately quantify the uncertainties, but most physics-based climate models are computationally too expensive to run as ensemble. Recent works in physics-informed neural networks (PINNs) have combined deep learning and the physical sciences to learn up to 15k faster copies of climate submodels. However, the application of PINNs in climate modeling has so far been mostly limited to deterministic models. We leverage a novel method that combines polynomial chaos expansion (PCE), a classic technique for uncertainty propagation, with PINNs. The PCE-PINNs learn a fast surrogate model that is demonstrated for uncertainty propagation of known parameter uncertainties. We showcase the effectiveness in ocean modeling by using the local advection-diffusion equation., Comment: Presented at ICRL 2021 Workshop on AI for Modeling Oceans and Climate Change

Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni\~no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns. In comparison, graph neural networks (GNNs) are capable of modeling large-scale spatial dependencies and are more interpretable due to the explicit modeling of information flow through edge connections. We propose the first application of graph neural networks to seasonal forecasting. We design a novel graph connectivity learning module that enables our GNN model to learn large-scale spatial interactions jointly with the actual ENSO forecasting task. Our model, \graphino, outperforms state-of-the-art deep learning-based models for forecasts up to six months ahead. Additionally, we show that our model is more interpretable as it learns sensible connectivity structures that correlate with the ENSO anomaly pattern.

Deep generative vision models are now able to synthesize realistic-looking satellite imagery. But, the possibility of hallucinations prevents their adoption for risk-sensitive applications, such as generating materials for communicating climate change. To demonstrate this issue, we train a generative adversarial network (pix2pixHD) to create synthetic satellite imagery of future flooding and reforestation events. We find that a pure deep learning-based model can generate photorealistic flood visualizations but hallucinates floods at locations that were not susceptible to flooding. To address this issue, we propose to condition and evaluate generative vision models on segmentation maps of physics-based flood models. We show that our physics-conditioned model outperforms the pure deep learning-based model and a handcrafted baseline. We evaluate the generalization capability of our method to different remote sensing data and different climate-related events (reforestation). We publish our code and dataset which includes the data for a third case study of melting Arctic sea ice and $>$30,000 labeled HD image triplets -- or the equivalent of 5.5 million images at 128x128 pixels -- for segmentation guided image-to-image translation in Earth observation. Code and data is available at \url{https://github.com/blutjens/eie-earth-public}., Comment: arXiv admin note: text overlap with arXiv:2010.08103

Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni\~no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns called teleconnections. Hence, we propose the application of spatiotemporal Graph Neural Networks (GNN) to forecast ENSO at long lead times, finer granularity and improved predictive skill than current state-of-the-art methods. The explicit modeling of information flow via edges may also allow for more interpretable forecasts. Preliminary results are promising and outperform state-of-the art systems for projections 1 and 3 months ahead., Comment: Presented at the NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning

As climate change increases the intensity of natural disasters, society needs better tools for adaptation. Floods, for example, are the most frequent natural disaster, but during hurricanes the area is largely covered by clouds and emergency managers must rely on nonintuitive flood visualizations for mission planning. To assist these emergency managers, we have created a deep learning pipeline that generates visual satellite images of current and future coastal flooding. We advanced a state-of-the-art GAN called pix2pixHD, such that it produces imagery that is physically-consistent with the output of an expert-validated storm surge model (NOAA SLOSH). By evaluating the imagery relative to physics-based flood maps, we find that our proposed framework outperforms baseline models in both physical-consistency and photorealism. While this work focused on the visualization of coastal floods, we envision the creation of a global visualization of how climate change will shape our earth., Comment: Under Review

The fishbone-induced transport of alpha particles is computed for the ITER 15 MA baseline scenario, using the nonlinear hybrid Kinetic-MHD code XTOR-K. Two limit cases have been studied, in order to analyse the characteristic regimes of the fishbone instability : the weak kinetic drive limit and the strong kinetic drive limit. In both those regimes, characteristic features of the n = m = 1 fishbone instability are recovered, such as a strong up/down-chirping of the mode frequency, associated with a resonant transport of trapped and passing alpha particles. The effects of the n = m = 0 sheared poloidal and toroidal plasma rotation are taken into account in the simulations. The shear is not negligible, which implies that the fishbone mode frequency has a radial dependency, impacting the wave-particle resonance condition. Phase space hole and clump structures are observed in both nonlinear regimes, centered around the precessional and passing resonances. These structures remains attached to the resonances as the different mode frequencies chirp up and down. In the nonlinear phase, the transport of individual resonant trapped particles is identified to be linked to mode-particle synchronization. On this basis, a partial mechanism for the nonlinear coupling between particle transport and mode dominant down-chirping is proposed. The overall transport of alpha particles inside out the q = 1 surface is of order 2-5% of the initial population between the simulations. The loss of alpha power is found to be directly equal to the loss of alpha particles.

International stakeholders increasingly invest in offsetting carbon emissions, for example, via issuing Payments for Ecosystem Services (PES) to forest conservation projects. Issuing trusted payments requires a transparent monitoring, reporting, and verification (MRV) process of the ecosystem services (e.g., carbon stored in forests). The current MRV process, however, is either too expensive (on-ground inspection of forest) or inaccurate (satellite). Recent works propose low-cost and accurate MRV via automatically determining forest carbon from drone imagery, collected by the landowners. The automation of MRV, however, opens up the possibility that landowners report untruthful drone imagery. To be robust against untruthful reporting, we propose TrueBranch, a metric learning-based algorithm that verifies the truthfulness of drone imagery from forest conservation projects. TrueBranch aims to detect untruthfully reported drone imagery by matching it with public satellite imagery. Preliminary results suggest that nominal distance metrics are not sufficient to reliably detect untruthfully reported imagery. TrueBranch leverages metric learning to create a feature embedding in which truthfully and untruthfully collected imagery is easily distinguishable by distance thresholding., Comment: *Authors have contributed equally. Published as Spotlight Presentation at ICLR 2020 Workshop on Tackling Climate Change with Machine Learning

A new model of fast ion source induced by Neutral Beam Injection (NBI) in tokamaks has been implemented in the hybrid kinetic-magnetohydrodynamic code XTOR-K. This source, combined with the collisions also recently implemented, allows for the modeling of realistic slowing-down populations and their interplay with macroscopic modes. This paper describes the Neutral Beam injection and ionization model designed to reproduce experimental configurations and its validation for a typical discharge of the ASDEX Upgrade tokamak. The application to the interaction of NBI-induced fast particles with a kink mode in ASDEX Upgrade demonstrates a resonance between passing particles and the n=1 mode. This resonance partially stabilizes the kink mode and induces a radial transport of fast particles. Preliminary results in ITER-like circular geometry show that NBI induces a toroidal torque but has little impact on the kink mode dynamics.

The stability of the $n=m=1$ alpha-fishbone kinetic-MHD mode on the ITER 15 MA baseline scenario \cite{ITERB} is analyzed using the nonlinear hybrid Kinetic-MHD code XTOR-K. Quantitative agreement is found between the complex frequencies $\omega + i\gamma$ computed with the linear model in \cite{Brochard2018} and XTOR-K's linear simulations. Identical precessional resonance positions in phase space are also found between the linear model and XTOR-K. Linear hybrid simulations performed with XTOR-K on the ITER 15 MA scenario reveal that this configuration is likely to be unstable against the alpha fishbone mode. The fishbone thresholds for kinetic-MHD equilibria with flat q profiles with on-axis safety factor just below unity lies between $\beta_{\alpha,thres}/\beta_{tot} = 6-10\%$, whereas the expected beta ratio on ITER is $\beta_{\alpha}/\beta_{tot} = 15-20\%$ \cite{ITERB}., Comment: 22 pages, 12 figures, submitted to Nuclear Fusion

An increasing amount of companies and cities plan to become CO2-neutral, which requires them to invest in renewable energies and carbon emission offsetting solutions. One of the cheapest carbon offsetting solutions is preventing deforestation in developing nations, a major contributor in global greenhouse gas emissions. However, forest preservation projects historically display an issue of trust and transparency, which drives companies to invest in transparent, but expensive air carbon capture facilities. Preservation projects could conduct accurate forest inventories (tree diameter, species, height etc.) to transparently estimate the biomass and amount of stored carbon. However, current rainforest inventories are too inaccurate, because they are often based on a few expensive ground-based samples and/or low-resolution satellite imagery. LiDAR-based solutions, used in US forests, are accurate, but cost-prohibitive, and hardly-accessible in the Amazon rainforest. We propose accurate and cheap forest inventory analyses through Deep Learning-based processing of drone imagery. The more transparent estimation of stored carbon will create higher transparency towards clients and thereby increase trust and investment into forest preservation projects., Comment: Published at 2019 NeurIPS Workshop on Tackling Climate Change with Machine Learning

Deep Neural Network-based systems are now the state-of-the-art in many robotics tasks, but their application in safety-critical domains remains dangerous without formal guarantees on network robustness. Small perturbations to sensor inputs (from noise or adversarial examples) are often enough to change network-based decisions, which was already shown to cause an autonomous vehicle to swerve into oncoming traffic. In light of these dangers, numerous algorithms have been developed as defensive mechanisms from these adversarial inputs, some of which provide formal robustness guarantees or certificates. This work leverages research on certified adversarial robustness to develop an online certified defense for deep reinforcement learning algorithms. The proposed defense computes guaranteed lower bounds on state-action values during execution to identify and choose the optimal action under a worst-case deviation in input space due to possible adversaries or noise. The approach is demonstrated on a Deep Q-Network policy and is shown to increase robustness to noise and adversaries in pedestrian collision avoidance scenarios and a classic control task., Comment: Published at Conference on Robot Learning (CoRL) 2019; (v2) contains minor updates to related works; (v3) acknowledged AWS

Many current autonomous systems are being designed with a strong reliance on black box predictions from deep neural networks (DNNs). However, DNNs tend to be overconfident in predictions on unseen data and can give unpredictable results for far-from-distribution test data. The importance of predictions that are robust to this distributional shift is evident for safety-critical applications, such as collision avoidance around pedestrians. Measures of model uncertainty can be used to identify unseen data, but the state-of-the-art extraction methods such as Bayesian neural networks are mostly intractable to compute. This paper uses MC-Dropout and Bootstrapping to give computationally tractable and parallelizable uncertainty estimates. The methods are embedded in a Safe Reinforcement Learning framework to form uncertainty-aware navigation around pedestrians. The result is a collision avoidance policy that knows what it does not know and cautiously avoids pedestrians that exhibit unseen behavior. The policy is demonstrated in simulation to be more robust to novel observations and take safer actions than an uncertainty-unaware baseline., Comment: ICRA 2019; Presented at IROS 2018 Workshop on Machine Learning in Robot Motion Planning

A search for the production and non-standard decay of a Higgs boson, h, into four taus through intermediate pseudoscalars, a, is conducted on 683 pb-1 of data collected by the ALEPH experiment at centre-of-mass energies from 183 to 209 GeV. No excess of events above background is observed, and exclusion limits are placed on the combined production cross section times branching ratio, \xi^2 = \sigma(e+e- --> Zh)/\sigma_{SM}(e+e- --> Zh) x B(h --> aa)x B(a --> \tau^+\tau^-)^2. For mh < 107 GeV/c2 and 4 < ma < 10 GeV/c2, \xi^2 > 1 is excluded at the 95% confidence level., Comment: 18 pages, 16 figures