Search

The field of text privatization often leverages the notion of $\textit{Differential Privacy}$ (DP) to provide formal guarantees in the rewriting or obfuscation of sensitive textual data. A common and nearly ubiquitous form of DP application necessitates the addition of calibrated noise to vector representations of text, either at the data- or model-level, which is governed by the privacy parameter $\varepsilon$. However, noise addition almost undoubtedly leads to considerable utility loss, thereby highlighting one major drawback of DP in NLP. In this work, we introduce a new sentence infilling privatization technique, and we use this method to explore the effect of noise in DP text rewriting. We empirically demonstrate that non-DP privatization techniques excel in utility preservation and can find an acceptable empirical privacy-utility trade-off, yet cannot outperform DP methods in empirical privacy protections. Our results highlight the significant impact of noise in current DP rewriting mechanisms, leading to a discussion of the merits and challenges of DP in NLP, as well as the opportunities that non-DP methods present., Comment: 19 pages, 3 figures, 9 tables. Accepted to NAACL 2025 (Findings)

Near-extremal black holes have a long AdS$_2$ throat in their near-horizon region. Quantum fluctuations in the throat region are effectively governed by a quantum version of Jackiw-Teitelboim gravity with matter and are strongly coupled at low temperatures. We investigate how these quantum fluctuations affect the spectrum of emission of particles during Hawking radiation. We systematically consider the cases of Kerr and Kerr-Newman black holes for emission of scalar particles and discuss photon and graviton emission from the Kerr background. We find that at very low temperatures the quantum fluctuations radically change the nature of particle emission. Unlike the generic suppression of particle emission in the spherically symmetric Reissner-Nordstr\"om case, we uncover that for particles with non-vanishing angular momentum, the quantum-corrected emission can be substantially enhanced with respect to the standard semiclassical result., Comment: 45 pages, 15 figures

Quantum fluctuations of the spacetime metric induce an uncertainty in the horizon area of a black hole. Working in linearized quantum gravity, we derive the variance in the area of a four-dimensional Schwarzschild black hole from the renormalized graviton propagator. We find that the standard deviation of the horizon area scales as the product of the Schwarzschild radius and the Planck length. For macroscopic black holes, the quantum uncertainty is therefore enormous in Planck units., Comment: 27 pages, 2 figures

Classical neural ODEs trained with explicit methods are intrinsically limited by stability, crippling their efficiency and robustness for stiff learning problems that are common in graph learning and scientific machine learning. We present a semi-implicit neural ODE approach that exploits the partitionable structure of the underlying dynamics. Our technique leads to an implicit neural network with significant computational advantages over existing approaches because of enhanced stability and efficient linear solves during time integration. We show that our approach outperforms existing approaches on a variety of applications including graph classification and learning complex dynamical systems. We also demonstrate that our approach can train challenging neural ODEs where both explicit methods and fully implicit methods are intractable.

The James Webb Space Telescope (JWST) has recently discovered a new population of objects at high redshift referred to as `Little Red Dots' (LRDs). Their nature currently remains elusive, despite their surprisingly high inferred number densities. This emerging population of red point-like sources is reshaping our view of the early Universe and may shed light on the formation of high-redshift supermassive black holes. Here we present a spectroscopically confirmed LRD CANUCS-LRD-z8.6 at $z_{\rm spec}=8.6319\pm 0.0005$ hosting an Active Galactic Nucleus (AGN), using JWST data. This source shows the typical spectral shape of an LRD (blue UV and red optical continuum, unresolved in JWST imaging), along with broad H$\beta$ line emission, detection of high-ionization emission lines (CIV, NIV]) and very high electron temperature indicative of the presence of AGN. This is also combined with a very low metallicity ($Z<0.1 Z_\odot$). The presence of all these diverse features in one source makes CANUCS-LRD-z8.6 unique. We show that the inferred black hole mass of CANUCS-LRD-z8.6 ($M_{\rm BH}=1.0^{+0.6}_{-0.4}\times 10^{8}\rm ~M_\odot$) strongly challenges current standard theoretical models and simulations of black hole formation, and forces us to adopt `ad hoc' prescriptions. Indeed if massive seeds, or light seeds with super-Eddington accretion, are considered, the observed BH mass of CANUCS-LRD-z8.6 at $z=8.6$ can be reproduced. Moreover, the black hole is over-massive compared to its host, relative to the local $M_{\rm BH}-M_*$ relations, pointing towards an earlier and faster evolution of the black hole compared to its host galaxy., Comment: 4 main figures; 8 supplementary figures; 5 supplementary tables

Predicting the long-term behavior of chaotic systems remains a formidable challenge due to their extreme sensitivity to initial conditions and the inherent limitations of traditional data-driven modeling approaches. This paper introduces a novel framework that addresses these challenges by leveraging the recently proposed multi-step penalty (MP) optimization technique. Our approach extends the applicability of MP optimization to a wide range of deep learning architectures, including Fourier Neural Operators and UNETs. By introducing penalized local discontinuities in the forecast trajectory, we effectively handle the non-convexity of loss landscapes commonly encountered in training neural networks for chaotic systems. We demonstrate the effectiveness of our method through its application to two challenging use-cases: the prediction of flow velocity evolution in two-dimensional turbulence and ocean dynamics using reanalysis data. Our results highlight the potential of this approach for accurate and stable long-term prediction of chaotic dynamics, paving the way for new advancements in data-driven modeling of complex natural phenomena., Comment: 7 pages, 5 Figures, Submitted to CASML2024

Modern-world robotics involves complex environments where multiple autonomous agents must interact with each other and other humans. This necessitates advanced interactive multi-agent motion planning techniques. Generalized Nash equilibrium(GNE), a solution concept in constrained game theory, provides a mathematical model to predict the outcome of interactive motion planning, where each agent needs to account for other agents in the environment. However, in practice, multiple local GNEs may exist. Finding a single GNE itself is complex as it requires solving coupled constrained optimal control problems. Furthermore, finding all such local GNEs requires exploring the solution space of GNEs, which is a challenging task. This work proposes the MultiNash-PF framework to efficiently compute multiple local GNEs in constrained trajectory games. Potential games are a class of games for which a local GNE of a trajectory game can be found by solving a single constrained optimal control problem. We propose MultiNash-PF that integrates the potential game approach with implicit particle filtering, a sample-efficient method for non-convex trajectory optimization. We first formulate the underlying game as a constrained potential game and then utilize the implicit particle filtering to identify the coarse estimates of multiple local minimizers of the game's potential function. MultiNash-PF then refines these estimates with optimization solvers, obtaining different local GNEs. We show through numerical simulations that MultiNash-PF reduces computation time by up to 50\% compared to a baseline approach.

When working around humans, it is important to model their perception limitations in order to predict their behavior more accurately. In this work, we consider agents with a limited field of view, viewing range, and ability to miss objects within viewing range (e.g., transparency). By considering the observation model independently from the motion policy, we can better predict the agent's behavior by considering these limitations and approximating them. We perform a user study where human operators navigate a cluttered scene while scanning the region for obstacles with a limited field of view and range. Using imitation learning, we show that a robot can adopt a human's strategy for observing an environment with limitations on observation and navigate with minimal collision with dynamic and static obstacles. We also show that this learned model helps it successfully navigate a physical hardware vehicle in real time.

This work develops a distributed graph neural network (GNN) methodology for mesh-based modeling applications using a consistent neural message passing layer. As the name implies, the focus is on enabling scalable operations that satisfy physical consistency via halo nodes at sub-graph boundaries. Here, consistency refers to the fact that a GNN trained and evaluated on one rank (one large graph) is arithmetically equivalent to evaluations on multiple ranks (a partitioned graph). This concept is demonstrated by interfacing GNNs with NekRS, a GPU-capable exascale CFD solver developed at Argonne National Laboratory. It is shown how the NekRS mesh partitioning can be linked to the distributed GNN training and inference routines, resulting in a scalable mesh-based data-driven modeling workflow. We study the impact of consistency on the scalability of mesh-based GNNs, demonstrating efficient scaling in consistent GNNs for up to O(1B) graph nodes on the Frontier exascale supercomputer.

Ensemble Kalman Filtering (EnKF) is a popular technique for data assimilation, with far ranging applications. However, the vanilla EnKF framework is not well-defined when perturbations are nonlinear. We study two non-linear extensions of the vanilla EnKF - dubbed the conditional-Gaussian EnKF (CG-EnKF) and the normal score EnKF (NS-EnKF) - which sidestep assumptions of linearity by constructing the Kalman gain matrix with the `conditional Gaussian' update formula in place of the traditional one. We then compare these models against a state-of-the-art deep learning based particle filter called the score filter (SF). This model uses an expensive score diffusion model for estimating densities and also requires a strong assumption on the perturbation operator for validity. In our comparison, we find that CG-EnKF and NS-EnKF dramatically outperform SF for a canonical problem in high-dimensional multiscale data assimilation given by the Lorenz-96 system. Our analysis also demonstrates that the CG-EnKF and NS-EnKF can handle highly non-Gaussian additive noise perturbations, with the latter typically outperforming the former.

Dynamic games offer a versatile framework for modeling the evolving interactions of strategic agents, whose steady-state behavior can be captured by the Nash equilibria of the games. Nash equilibria are often computed in feedback, with policies depending on the state at each time, or in open-loop, with policies depending only on the initial state. Empirically, open-loop Nash equilibria (OLNE) could be more efficient to compute, while feedback Nash equilibria (FBNE) often encode more complex interactions. However, it remains unclear exactly which dynamic games yield FBNE and OLNE that differ significantly and which do not. To address this problem, we present a principled comparison study of OLNE and FBNE in linear quadratic (LQ) dynamic games. Specifically, we prove that the OLNE strategies of an LQ dynamic game can be synthesized by solving the coupled Riccati equations of an auxiliary LQ game with perturbed costs. The construction of the auxiliary game allows us to establish conditions under which OLNE and FBNE coincide and derive an upper bound on the deviation between FBNE and OLNE of an LQ game.

The celebrated Takens' embedding theorem provides a theoretical foundation for reconstructing the full state of a dynamical system from partial observations. However, the classical theorem assumes that the underlying system is deterministic and that observations are noise-free, limiting its applicability in real-world scenarios. Motivated by these limitations, we rigorously establish a measure-theoretic generalization that adopts an Eulerian description of the dynamics and recasts the embedding as a pushforward map between probability spaces. Our mathematical results leverage recent advances in optimal transportation theory. Building on our novel measure-theoretic time-delay embedding theory, we have developed a new computational framework that forecasts the full state of a dynamical system from time-lagged partial observations, engineered with better robustness to handle sparse and noisy data. We showcase the efficacy and versatility of our approach through several numerical examples, ranging from the classic Lorenz-63 system to large-scale, real-world applications such as NOAA sea surface temperature forecasting and ERA5 wind field reconstruction., Comment: 32 pages, 8 figures

A graph neural network (GNN) approach is introduced in this work which enables mesh-based three-dimensional super-resolution of fluid flows. In this framework, the GNN is designed to operate not on the full mesh-based field at once, but on localized meshes of elements (or cells) directly. To facilitate mesh-based GNN representations in a manner similar to spectral (or finite) element discretizations, a baseline GNN layer (termed a message passing layer, which updates local node properties) is modified to account for synchronization of coincident graph nodes, rendering compatibility with commonly used element-based mesh connectivities. The architecture is multiscale in nature, and is comprised of a combination of coarse-scale and fine-scale message passing layer sequences (termed processors) separated by a graph unpooling layer. The coarse-scale processor embeds a query element (alongside a set number of neighboring coarse elements) into a single latent graph representation using coarse-scale synchronized message passing over the element neighborhood, and the fine-scale processor leverages additional message passing operations on this latent graph to correct for interpolation errors. Demonstration studies are performed using hexahedral mesh-based data from Taylor-Green Vortex flow simulations at Reynolds numbers of 1600 and 3200. Through analysis of both global and local errors, the results ultimately show how the GNN is able to produce accurate super-resolved fields compared to targets in both coarse-scale and multiscale model configurations.

We show that birational hyper-K\"ahler varieties of $K3^{[n]}$-type are derived equivalent, establishing the D-equivalence conjecture in these cases. The Fourier-Mukai kernels of our derived equivalences are constructed from projectively hyperholomorphic bundles, following ideas of Markman. Our method also proves a stronger version of the D-equivalence conjecture for hyper-K\"ahler varieties of $K3^{[n]}$-type with Brauer classes., Comment: Updated the file to prove a generalized version of the D-equivalence conjecture for K3[n]-type with any Brauer class, 15 pages

Forecasting dynamical systems is of importance to numerous real-world applications. When possible, dynamical systems forecasts are constructed based on first-principles-based models such as through the use of differential equations. When these equations are unknown, non-intrusive techniques must be utilized to build predictive models from data alone. Machine learning (ML) methods have recently been used for such tasks. Moreover, ML methods provide the added advantage of significant reductions in time-to-solution for predictions in contrast with first-principle based models. However, many state-of-the-art ML-based methods for forecasting rely on neural networks, which may be expensive to train and necessitate requirements for large amounts of memory. In this work, we propose a quantum mechanics inspired ML modeling strategy for learning nonlinear dynamical systems that provides data-driven forecasts for complex dynamical systems with reduced training time and memory costs. This approach, denoted the quantum reservoir computing technique (QRC), is a hybrid quantum-classical framework employing an ensemble of interconnected small quantum systems via classical linear feedback connections. By mapping the dynamical state to a suitable quantum representation amenable to unitary operations, QRC is able to predict complex nonlinear dynamical systems in a stable and accurate manner. We demonstrate the efficacy of this framework through benchmark forecasts of the NOAA Optimal Interpolation Sea Surface Temperature dataset and compare the performance of QRC to other ML methods.

The purpose of this paper is to study motivic aspects of the Hitchin system for $\mathrm{GL}_n$. Our results include the following. (a) We prove the motivic decomposition conjecture of Corti-Hanamura for the Hitchin system; in particular, the decomposition theorem associated with the Hitchin system is induced by algebraic cycles. This yields an unconditional construction of the motivic perverse filtration for the Hitchin system, which lifts the cohomological/sheaf-theoretic perverse filtration. (b) We prove that the inverse of the relative Hard Lefschetz symmetry is induced by a relative algebraic correspondence, confirming the relative Lefschetz standard conjecture for the Hitchin system. (c) We show a strong perversity bound for the normalized Chern classes of a universal bundle with respect to the motivic perverse filtration; this specializes to the sheaf-theoretic result obtained earlier by Maulik-Shen. (d) We prove a $\chi$-independence result for the relative Chow motives associated with Hitchin systems. Our methods combine Fourier transforms for compactified Jacobian fibrations associated with integral locally planar curves, nearby and vanishing cycle techniques, and a Springer-theoretic interpretation of parabolic Hitchin moduli spaces., Comment: 46 pages. Minor changes

Applications of Differential Privacy (DP) in NLP must distinguish between the syntactic level on which a proposed mechanism operates, often taking the form of $\textit{word-level}$ or $\textit{document-level}$ privatization. Recently, several word-level $\textit{Metric}$ Differential Privacy approaches have been proposed, which rely on this generalized DP notion for operating in word embedding spaces. These approaches, however, often fail to produce semantically coherent textual outputs, and their application at the sentence- or document-level is only possible by a basic composition of word perturbations. In this work, we strive to address these challenges by operating $\textit{between}$ the word and sentence levels, namely with $\textit{collocations}$. By perturbing n-grams rather than single words, we devise a method where composed privatized outputs have higher semantic coherence and variable length. This is accomplished by constructing an embedding model based on frequently occurring word groups, in which unigram words co-exist with bi- and trigram collocations. We evaluate our method in utility and privacy tests, which make a clear case for tokenization strategies beyond the word level., Comment: 13 pages, 2 figures, 9 tables. Accepted to PrivateNLP 2024

The task of text privatization using Differential Privacy has recently taken the form of $\textit{text rewriting}$, in which an input text is obfuscated via the use of generative (large) language models. While these methods have shown promising results in the ability to preserve privacy, these methods rely on autoregressive models which lack a mechanism to contextualize the private rewriting process. In response to this, we propose $\textbf{DP-MLM}$, a new method for differentially private text rewriting based on leveraging masked language models (MLMs) to rewrite text in a semantically similar $\textit{and}$ obfuscated manner. We accomplish this with a simple contextualization technique, whereby we rewrite a text one token at a time. We find that utilizing encoder-only MLMs provides better utility preservation at lower $\varepsilon$ levels, as compared to previous methods relying on larger models with a decoder. In addition, MLMs allow for greater customization of the rewriting mechanism, as opposed to generative approaches. We make the code for $\textbf{DP-MLM}$ public and reusable, found at https://github.com/sjmeis/DPMLM ., Comment: 15 pages, 2 figures, 8 tables. Accepted to ACL 2024 (Findings)

Forecasting high-dimensional dynamical systems is a fundamental challenge in various fields, such as geosciences and engineering. Neural Ordinary Differential Equations (NODEs), which combine the power of neural networks and numerical solvers, have emerged as a promising algorithm for forecasting complex nonlinear dynamical systems. However, classical techniques used for NODE training are ineffective for learning chaotic dynamical systems. In this work, we propose a novel NODE-training approach that allows for robust learning of chaotic dynamical systems. Our method addresses the challenges of non-convexity and exploding gradients associated with underlying chaotic dynamics. Training data trajectories from such systems are split into multiple, non-overlapping time windows. In addition to the deviation from the training data, the optimization loss term further penalizes the discontinuities of the predicted trajectory between the time windows. The window size is selected based on the fastest Lyapunov time scale of the system. Multi-step penalty(MP) method is first demonstrated on Lorenz equation, to illustrate how it improves the loss landscape and thereby accelerates the optimization convergence. MP method can optimize chaotic systems in a manner similar to least-squares shadowing with significantly lower computational costs. Our proposed algorithm, denoted the Multistep Penalty NODE, is applied to chaotic systems such as the Kuramoto-Sivashinsky equation, the two-dimensional Kolmogorov flow, and ERA5 reanalysis data for the atmosphere. It is observed that MP-NODE provide viable performance for such chaotic systems, not only for short-term trajectory predictions but also for invariant statistics that are hallmarks of the chaotic nature of these dynamics., Comment: 25 pages, 17 Figures, submitted to Computer Methods in Applied Mechanics and Engineering

Elephant random walk is a special type of random walk that incorporates the memory of the past to determine its future steps. The probability of this walk taking a particular step (+1 or -1) at a time point, conditioned on the entire history, depends on a linear function of the proportion of that step till that time point. In this work, we investigate how the dynamics of the random walk will change if we replace this linear function by a generic map satisfying some analytic conditions. We also propose a new model, called the multidimensional generalized elephant random walk, that incorporates several variants of elephant random walk in one and higher dimensions and their generalizations thereof. Using tools from the theory of stochastic approximation, we derive the asymptotic behavior of our model leading to newer results on the phase transition boundary between diffusive and superdiffusive regimes. We also mention a few open problems in this context., Comment: 30 pages

In this work, we provide a mathematical formulation for error propagation in flow trajectory prediction using data-driven turbulence closure modeling. Under the assumption that the predicted state of a large eddy simulation prediction must be close to that of a subsampled direct numerical simulation, we retrieve an upper bound for the prediction error when utilizing a data-driven closure model. We also demonstrate that this error is significantly affected by the time step size and the Jacobian which play a role in amplifying the initial one-step error made by using the closure. Our analysis also shows that the error propagates exponentially with rollout time and the upper bound of the system Jacobian which is itself influenced by the Jacobian of the closure formulation. These findings could enable the development of new regularization techniques for ML models based on the identified error-bound terms, improving their robustness and reducing error propagation.

We present a lightweight, easy-to-train, low-resolution, fully data-driven climate emulator, LUCIE, that can be trained on as low as $2$ years of $6$-hourly ERA5 data. Unlike most state-of-the-art AI weather models, LUCIE remains stable and physically consistent for $100$ years of autoregressive simulation with $100$ ensemble members. Long-term mean climatology from LUCIE's simulation of temperature, wind, precipitation, and humidity matches that of ERA5 data, along with the variability. We further demonstrate how well extreme weather events and their return periods can be estimated from a large ensemble of long-term simulations. We further discuss an improved training strategy with a hard-constrained first-order integrator to suppress autoregressive error growth, a novel spectral regularization strategy to better capture fine-scale dynamics, and finally an optimization algorithm that enables data-limited (as low as $2$ years of $6$-hourly data) training of the emulator without losing stability and physical consistency. Finally, we provide a scaling experiment to compare the long-term bias of LUCIE with respect to the number of training samples. Importantly, LUCIE is an easy to use model that can be trained in just $2.4$h on a single A-100 GPU, allowing for multiple experiments that can explore important scientific questions that could be answered with large ensembles of long-term simulations, e.g., the impact of different variables on the simulation, dynamic response to external forcing, and estimation of extreme weather events, amongst others.

The study of privacy-preserving Natural Language Processing (NLP) has gained rising attention in recent years. One promising avenue studies the integration of Differential Privacy in NLP, which has brought about innovative methods in a variety of application settings. Of particular note are $\textit{word-level Metric Local Differential Privacy (MLDP)}$ mechanisms, which work to obfuscate potentially sensitive input text by performing word-by-word $\textit{perturbations}$. Although these methods have shown promising results in empirical tests, there are two major drawbacks: (1) the inevitable loss of utility due to addition of noise, and (2) the computational expensiveness of running these mechanisms on high-dimensional word embeddings. In this work, we aim to address these challenges by proposing $\texttt{1-Diffractor}$, a new mechanism that boasts high speedups in comparison to previous mechanisms, while still demonstrating strong utility- and privacy-preserving capabilities. We evaluate $\texttt{1-Diffractor}$ for utility on several NLP tasks, for theoretical and task-based privacy, and for efficiency in terms of speed and memory. $\texttt{1-Diffractor}$ shows significant improvements in efficiency, while still maintaining competitive utility and privacy scores across all conducted comparative tests against previous MLDP mechanisms. Our code is made available at: https://github.com/sjmeis/Diffractor., Comment: 12 pages, 7 figures, 7 tables, 10th ACM International Workshop on Security and Privacy Analytics (IWSPA 2024)

Effective verification and validation techniques for modern scientific machine learning workflows are challenging to devise. Statistical methods are abundant and easily deployed, but often rely on speculative assumptions about the data and methods involved. Error bounds for classical interpolation techniques can provide mathematically rigorous estimates of accuracy, but often are difficult or impractical to determine computationally. In this work, we present a best-of-both-worlds approach to verifiable scientific machine learning by demonstrating that (1) multiple standard interpolation techniques have informative error bounds that can be computed or estimated efficiently; (2) comparative performance among distinct interpolants can aid in validation goals; (3) deploying interpolation methods on latent spaces generated by deep learning techniques enables some interpretability for black-box models. We present a detailed case study of our approach for predicting lift-drag ratios from airfoil images. Code developed for this work is available in a public Github repository.

Closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

The neural ordinary differential equation (ODE) framework has shown promise in developing accelerated surrogate models for complex systems described by partial differential equations (PDEs). In PDE-based systems, neural ODE strategies use a two-step approach for acceleration: a nonlinear dimensionality reduction via an autoencoder, and a time integration step through a neural-network based model (neural ODE). This study explores the effectiveness of autoencoder-based neural ODE strategies for advection-dominated PDEs. It includes predictive demonstrations and delves into the sources of model acceleration, focusing on how neural ODEs achieve this. The research quantifies the impact of autoencoder and neural ODE components on system time-scales through eigenvalue analysis of dynamical system Jacobians. It examines how various training parameters, like training methods, latent space dimensionality, and training trajectory length, affect model accuracy and latent time-scales. Particularly, it highlights the influence of training trajectory length on neural ODE time-scales, noting that longer trajectories enhance limiting time-scales, with optimal neural ODEs capturing the largest time-scales of the actual system. The study conducts demonstrations on two distinct unsteady fluid dynamics settings influenced by advection: the Kuramoto-Sivashinsky equations and Hydrogen-Air channel detonations, using the compressible reacting Navier--Stokes equations.

In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scientific exploration, bringing significant advancements across sectors from drug development to renewable energy. To answer this call, we present DeepSpeed4Science initiative (deepspeed4science.ai) which aims to build unique capabilities through AI system technology innovations to help domain experts to unlock today's biggest science mysteries. By leveraging DeepSpeed's current technology pillars (training, inference and compression) as base technology enablers, DeepSpeed4Science will create a new set of AI system technologies tailored for accelerating scientific discoveries by addressing their unique complexity beyond the common technical approaches used for accelerating generic large language models (LLMs). In this paper, we showcase the early progress we made with DeepSpeed4Science in addressing two of the critical system challenges in structural biology research.

We report on a search for magnetic monopoles (MMs) produced in ultraperipheral Pb--Pb collisions during Run-1 of the LHC. The beam pipe surrounding the interaction region of the CMS experiment was exposed to 184.07 \textmu b$^{-1}$ of Pb--Pb collisions at 2.76 TeV center-of-mass energy per collision in December 2011, before being removed in 2013. It was scanned by the MoEDAL experiment using a SQUID magnetometer to search for trapped MMs. No MM signal was observed. The two distinctive features of this search are the use of a trapping volume very close to the collision point and ultra-high magnetic fields generated during the heavy-ion run that could produce MMs via the Schwinger effect. These two advantages allowed setting the first reliable, world-leading mass limits on MMs with high magnetic charge. In particular, the established limits are the strongest available in the range between 2 and 45 Dirac units, excluding MMs with masses of up to 80 GeV at 95\% confidence level., Comment: As accepted by PRL. 7 pages, 4 figures

Motivated by the Beauville decomposition of an abelian scheme and the "Perverse = Chern" phenomenon for a compactified Jacobian fibration, we study in this paper splittings of the perverse filtration for compactified Jacobian fibrations. On the one hand, we prove for the Beauville-Mukai system associated with an irreducible curve class on a K3 surface the existence of a Fourier-stable multiplicative splitting of the perverse filtration, which extends the Beauville decomposition for the nonsingular fibers. Our approach is to construct a Lefschetz decomposition associated with a Fourier-conjugate $\mathfrak{sl}_2$-triple, which relies heavily on recent work concerning the interaction between derived equivalences and LLV algebras for hyper-K\"ahler varieties. Motivic lifting and connections to the Beauville-Voisin conjectures are also discussed. On the other hand, we construct for any $g\geq 2$ a compactified Jacobian fibration of genus g curves such that each curve is integral with at worst simple nodes and the (multiplicative) perverse filtration does not admit a multiplicative splitting. This shows that in general an extension of the Beauville decomposition cannot exist for compactified Jacobian fibrations even when the simplest singular point appears., Comment: 38 pages. Comments are welcome!

The low-temperature thermodynamics of near-extremal rotating black holes has recently been revisited to incorporate one-loop contributions that are dominant in this regime. We discuss these quantum corrections to the gravitational path integral for asymptotically Anti de-Sitter black holes in four and five dimensions. In four dimensions we explicitly consider Kerr-AdS$_4$, Kerr-Newman-AdS$_4$ and the rotating black hole in ${\cal N}=4$ gauged supergravity with two scalars and two electric charges turned on. In five dimensions we explicitly address the asymptotically flat Myers-Perry black hole and the Kerr-AdS$_5$ black hole. In every case we find that tensor modes contribute $\frac{3}{2} \log T_{\rm Hawking}$ to the low-temperature thermodynamics. We identify the root cause of this universality in two facts: (i) the universal presence of a $SL(2,\mathbb{R})$ subgroup of isometries in the near-horizon geometry and (ii) a set of cancellations in the Lichnerowicz operator. We show that these two conditions hold for near-extremal black holes in asymptotically flat and asymptotically AdS spacetimes of various dimensions., Comment: 42 pages

Observations of $z \sim 6$ quasars powered by supermassive black holes (SMBHs; $M_{\rm BH} \sim 10^{8-10}\, M_\odot$) challenge our current understanding of early black hole (BH) formation and evolution. The advent of the James Webb Space Telescope (JWST) has enabled the study of massive BHs (MBHs; $M_{\rm BH}\sim 10^{6-7} \ \mathrm{M}_\odot$) up to $z\sim 11$, thus bridging the properties of $z\sim 6$ quasars to their ancestors. The JWST spectroscopic observations of GN-z11, a well-known $z=10.6$ star-forming galaxy, have been interpreted with the presence of a super-Eddington (Eddington ratio $\equiv \,\lambda_{\rm Edd}\sim 5.5$) accreting MBH. To test this hypothesis, we used a zoom-in cosmological simulation of galaxy formation and BH co-evolution. We first tested the simulation results against the observed probability distribution function (PDF) of $\lambda_{\rm Edd}$ found in $z\sim 6$ quasars. Then, in the simulation we selected the BHs that satisfy the following criteria: (a) $10 < z < 11 $, (b) $M_{\rm BH} > 10^6 \ \mathrm{M}_\odot$. Next, we apply the extreme value statistics to the PDF of $\lambda_{\rm Edd}$ resulting from the simulation and we find that the probability of observing a $z\sim 10-11$ MBH accreting with $\lambda_{\rm Edd} \sim 5.5$ in the volume surveyed by JWST is very low ($<0.2\%$). We compared our predictions with those in the literature, and discuss the main limitations of our work. Our simulation cannot explain the JWST observations of GN-z11. This might be due to (i) poor resolution and statistics in simulations, (ii) simplistic sub-grid models (e.g. BH accretion and seeding), (iii) uncertainties in the data analysis and interpretation., Comment: 8 pages, 2 figures; accepted for publication in A&A

This study explores the potential of physics-informed neural networks (PINNs) for the realization of digital twins (DT) from various perspectives. First, various adaptive sampling approaches for collocation points are investigated to verify their effectiveness in the mesh-free framework of PINNs, which allows automated construction of virtual representation without manual mesh generation. Then, the overall performance of the data-driven PINNs (DD-PINNs) framework is examined, which can utilize the acquired datasets in DT scenarios. Its scalability to more general physics is validated within parametric Navier-Stokes equations, where PINNs do not need to be retrained as the Reynolds number varies. In addition, since datasets can be often collected from different fidelity/sparsity in practice, multi-fidelity DD-PINNs are also proposed and evaluated. They show remarkable prediction performance even in the extrapolation tasks, with $42\sim62\%$ improvement over the single-fidelity approach. Finally, the uncertainty quantification performance of multi-fidelity DD-PINNs is investigated by the ensemble method to verify their potential in DT, where an accurate measure of predictive uncertainty is critical. The DD-PINN frameworks explored in this study are found to be more suitable for DT scenarios than traditional PINNs from the above perspectives, bringing engineers one step closer to seamless DT realization.

We study entanglement entropy in a non-relativistic Schr\"{o}dinger field theory at finite temperature and electric charge using the principle of gauge/gravity duality. The spacetime geometry is obtained from a charged AdS black hole by a null Melvin twist. By using an appropriate modification of the holographic Ryu-Takayanagi formula, we calculate the entanglement entropy, mutual information, and entanglement wedge cross-section for the simplest strip subsystem. The entanglement measures show non-trivial dependence on the black hole parameters., Comment: 19 pages, 6 figures

We consider quantum gravity fluctuations in a pair of nearby gravitational wave detectors. Quantum fluctuations of long-wavelength modes of the gravitational field induce coherent fluctuations in the detectors, leading to correlated noise. We determine the variance and covariance in the lengths of the arms of the detectors, and thereby obtain the graviton noise correlation. We find that the correlation depends on the angle between the detector arms as well as their separation distance., Comment: 14 pages, 2 figures, LaTeX

We consider a congruence of null geodesics in the presence of a quantized spacetime metric. The coupling to a quantum metric induces fluctuations in the congruence; we calculate the change in the area of a pencil of geodesics induced by such fluctuations. For the gravitational field in its vacuum state, we find that quantum gravity contributes a correction to the null Raychaudhuri equation which is of the same sign as the classical terms. We thus derive a quantum-gravitational focusing theorem valid for linearized quantum gravity., Comment: 15 pages, 1 figure, v2. published version in JHEP

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.

Weather forecasting is a fundamental problem for anticipating and mitigating the impacts of climate change. Recently, data-driven approaches for weather forecasting based on deep learning have shown great promise, achieving accuracies that are competitive with operational systems. However, those methods often employ complex, customized architectures without sufficient ablation analysis, making it difficult to understand what truly contributes to their success. Here we introduce Stormer, a simple transformer model that achieves state-of-the-art performance on weather forecasting with minimal changes to the standard transformer backbone. We identify the key components of Stormer through careful empirical analyses, including weather-specific embedding, randomized dynamics forecast, and pressure-weighted loss. At the core of Stormer is a randomized forecasting objective that trains the model to forecast the weather dynamics over varying time intervals. During inference, this allows us to produce multiple forecasts for a target lead time and combine them to obtain better forecast accuracy. On WeatherBench 2, Stormer performs competitively at short to medium-range forecasts and outperforms current methods beyond 7 days, while requiring orders-of-magnitude less training data and compute. Additionally, we demonstrate Stormer's favorable scaling properties, showing consistent improvements in forecast accuracy with increases in model size and training tokens. Code and checkpoints are available at https://github.com/tung-nd/stormer., Comment: Neural Information Processing Systems (NeurIPS 2024)

We propose a logarithmic enhancement of the Gromov-Witten/Donaldson-Thomas correspondence, with descendants, and study its behavior under simple normal crossings degenerations. The formulation of the logarithmic correspondence requires a matching of tangency conditions with relative insertions. This is achieved via a version of the Nakajima basis for the cohomology of the Hilbert schemes of points on a logarithmic surface. Next, we establish a strong form of the degeneration formula in logarithmic DT theory - the numerical DT invariants of the general fiber of a degeneration are determined by the numerical DT invariants attached to strata of the special fiber. The GW version of this result, which we prove in all target dimensions, strengthens currently known formulas. A key role is played by a certain exotic class of insertions, introduced here, that impose non-local incidence conditions coupled across multiple boundary strata of the target geometry. Finally, we prove compatibility of the new logarithmic GW/DT correspondence with degenerations. In particular, the logarithmic conjecture for all strata of the special fiber of a degeneration implies the traditional GW/DT conjecture on the general fiber. Compatibility is a strong constraint, and can be used to calculate logarithmic DT invariants. Several examples are included to illustrate the nature and utility of the formula., Comment: 90 pages. v2: Minor changes. Final version, to appear in Cambridge Journal of Mathematics

Data-driven surrogate modeling has surged in capability in recent years with the emergence of graph neural networks (GNNs), which can operate directly on mesh-based representations of data. The goal of this work is to introduce an interpretability enhancement procedure for GNNs, with application to unstructured mesh-based fluid dynamics modeling. Given a black-box baseline GNN model, the end result is an interpretable GNN model that isolates regions in physical space, corresponding to sub-graphs, that are intrinsically linked to the forecasting task while retaining the predictive capability of the baseline. These structures identified by the interpretable GNNs are adaptively produced in the forward pass and serve as explainable links between the baseline model architecture, the optimization goal, and known problem-specific physics. Additionally, through a regularization procedure, the interpretable GNNs can also be used to identify, during inference, graph nodes that correspond to a majority of the anticipated forecasting error, adding a novel interpretable error-tagging capability to baseline models. Demonstrations are performed using unstructured flow field data sourced from flow over a backward-facing step at high Reynolds numbers, with geometry extrapolations demonstrated for ramp and wall-mounted cube configurations.

Informed selection of drug candidates for laboratory experimentation provides an efficient means of identifying suitable anti-cancer treatments. The advancement of artificial intelligence has led to the development of computational models to predict cancer cell line response to drug treatment. It is important to analyze the false positive rate (FPR) of the models, to increase the number of effective treatments identified and to minimize unnecessary laboratory experimentation. Such analysis will also aid in identifying drugs or cancer types that require more data collection to improve model predictions. This work uses an attention based neural network classification model to identify responsive/non-responsive drug treatments across multiple types of cancer cell lines. Two data filtering techniques have been applied to generate 10 data subsets, including removing samples for which dose response curves are poorly fitted and removing samples whose area under the dose response curve (AUC) values are marginal around 0.5 from the training set. One hundred trials of 10-fold cross-validation analysis is performed to test the model prediction performance on all the data subsets and the subset with the best model prediction performance is selected for further analysis. Several error analysis metrics such as the false positive rate (FPR), and the prediction uncertainty are evaluated, and the results are summarized by cancer type and drug mechanism of action (MoA) category. The FPR of cancer type spans between 0.262 and 0.5189, while that of drug MoA category spans almost the full range of [0, 1]. This study identifies cancer types and drug MoAs with high FPRs. Additional drug screening data of these cancer and drug categories may improve response modeling. Our results also demonstrate that the two data filtering approaches help improve the drug response prediction performance.

We propose a parallel (distributed) version of the spectral proper orthogonal decomposition (SPOD) technique. The parallel SPOD algorithm distributes the spatial dimension of the dataset preserving time. This approach is adopted to preserve the non-distributed fast Fourier transform of the data in time, thereby avoiding the associated bottlenecks. The parallel SPOD algorithm is implemented in the PySPOD (https://github.com/MathEXLab/PySPOD) library and makes use of the standard message passing interface (MPI) library, implemented in Python via mpi4py (https://mpi4py.readthedocs.io/en/stable/). An extensive performance evaluation of the parallel package is provided, including strong and weak scalability analyses. The open-source library allows the analysis of large datasets of interest across the scientific community. Here, we present applications in fluid dynamics and geophysics, that are extremely difficult (if not impossible) to achieve without a parallel algorithm. This work opens the path toward modal analyses of big quasi-stationary data, helping to uncover new unexplored spatio-temporal patterns.

This study presents a methodology focusing on the use of computational model and experimental data fusion to improve the Spalart-Allmaras (SA) closure model for Reynolds-averaged Navier-Stokes solutions. In particular, our goal is to develop a technique that not only assimilates sparse experimental data to improve turbulence model performance, but also preserves generalization for unseen cases by recovering classical SA behavior. We achieve our goals using data assimilation, namely the Ensemble Kalman filtering approach (EnKF), to calibrate the coefficients of the SA model for separated flows. A holistic calibration strategy is implemented via the parameterization of the production, diffusion, and destruction terms. This calibration relies on the assimilation of experimental data collected in the form of velocity profiles, skin friction, and pressure coefficients. Despite using observational data from a single flow condition around a backward-facing step (BFS), the recalibrated SA model demonstrates generalization to other separated flows, including cases such as the 2D NASA wall mounted hump (2D-WMH) and modified BFS. Significant improvement is observed in the quantities of interest, i.e., skin friction coefficient ($C_f$) and pressure coefficient ($C_p$) for each flow tested. Finally, it is also demonstrated that the newly proposed model recovers SA proficiency for flows, such as a NACA-0012 airfoil and axisymmetric jet (ASJ), and that the individually calibrated terms in the SA model target specific flow-physics wherein the calibrated production term improves the re-circulation zone while destruction improves the recovery zone.

We study the interaction between Fourier-Mukai transforms and perverse filtrations for a certain class of dualizable abelian fibrations. Multiplicativity of the perverse filtration and the "Perverse $\supset$ Chern" phenomenon for these abelian fibrations are immediate consequences of our theory. We also show that our class of fibrations include families of compactified Jacobians of integral locally planar curves. Applications include the following: (a) we prove the motivic decomposition conjecture for this class (including compactified Jacobian fibrations), which generalizes Deninger-Murre's theorem for abelian schemes; (b) we provide a new proof of the P=W conjecture for $\mathrm{GL}_r$; (c) we prove half of the P=C conjecture concerning refined BPS invariants for the local $\mathbb{P}^2$; (d) we show that the perverse filtration for the compactified Jacobian associated with an integral locally planar curve is multiplicative, which generalizes a result of Oblomkov-Yun for homogeneous singularities. Our techniques combine Arinkin's autoduality for coherent categories, Ng\^o's support theorem for the decomposition theorem, Adams operations in operational K-theory, and Corti-Hanamura's theory of relative Chow motives., Comment: 55 pages; comments are welcome

In interactive multi-agent settings, decision-making complexity arises from agents' interconnected objectives. Dynamic game theory offers a formal framework for analyzing such intricacies. Yet, solving dynamic games and determining Nash equilibria pose computational challenges due to the need of solving coupled optimal control problems. To address this, our key idea is to leverage potential games, which are games with a potential function that allows for the computation of Nash equilibria by optimizing the potential function. We argue that dynamic potential games, can effectively facilitate interactive decision-making in many multi-agent interactions. We will identify structures in realistic multi-agent interactive scenarios that can be transformed into weighted potential dynamic games. We will show that the open-loop Nash equilibria of the resulting weighted potential dynamic game can be obtained by solving a single optimal control problem. We will demonstrate the effectiveness of the proposed method through various simulation studies, showing close proximity to feedback Nash equilibria and significant improvements in solve time compared to state-of-the-art game solvers.

Differentiable physical simulators are proving to be valuable tools for developing data-driven models in computational fluid dynamics (CFD). These simulators enable end-to-end training of machine learning (ML) models embedded within CFD solvers. This paradigm enables novel algorithms which combine the generalization power and low cost of physics-based simulations with the flexibility and automation of deep learning methods. In this study, we introduce a framework for embedding deep learning models within a generic finite element solver to solve the Navier-Stokes equations, specifically applying this approach to learn a subgrid scale closure with a graph neural network (GNN). We validate our method for flow over a backwards-facing step and test its performance on novel geometries, demonstrating the ability to generalize to novel geometries without sacrificing stability. Additionally, we show that our GNN-based closure model may be learned in a data-limited scenario by interpreting closure modeling as a solver-constrained optimization. Our end-to-end learning paradigm demonstrates a viable pathway for physically consistent and generalizable data-driven closure modeling across complex geometries.

We initiate an empirical investigation into differentially private graph neural networks on population graphs from the medical domain by examining privacy-utility trade-offs at different privacy levels on both real-world and synthetic datasets and performing auditing through membership inference attacks. Our findings highlight the potential and the challenges of this specific DP application area. Moreover, we find evidence that the underlying graph structure constitutes a potential factor for larger performance gaps by showing a correlation between the degree of graph homophily and the accuracy of the trained model.

Carrollian field theories at the classical level possess an infinite number of space-time symmetries, namely the supertranslations. In this article, we inquire whether these symmetries for interacting Carrollian scalar field theory survive in the presence of quantum effects. For interactions polynomial in the field, the answer is in the affirmative. We also study a renormalization group flow particularly tailored to respect the manifest Carroll invariance and analyze the consequences of introducing Carroll-breaking deformations. The renormalization group flow, with perturbative loop-level effects taken into account, indicates a new fixed point apart from the Gaussian ones., Comment: 20 pages, 6 figures. Version accepted for publication in Phys. Rev. D

Deep learning is increasingly becoming a promising pathway to improving the accuracy of sub-grid scale (SGS) turbulence closure models for large eddy simulations (LES). We leverage the concept of differentiable turbulence, whereby an end-to-end differentiable solver is used in combination with physics-inspired choices of deep learning architectures to learn highly effective and versatile SGS models for two-dimensional turbulent flow. We perform an in-depth analysis of the inductive biases in the chosen architectures, finding that the inclusion of small-scale non-local features is most critical to effective SGS modeling, while large-scale features can improve pointwise accuracy of the \textit{a-posteriori} solution field. The velocity gradient tensor on the LES grid can be mapped directly to the SGS stress via decomposition of the inputs and outputs into isotropic, deviatoric, and anti-symmetric components. We see that the model can generalize to a variety of flow configurations, including higher and lower Reynolds numbers and different forcing conditions. We show that the differentiable physics paradigm is more successful than offline, \textit{a-priori} learning, and that hybrid solver-in-the-loop approaches to deep learning offer an ideal balance between computational efficiency, accuracy, and generalization. Our experiments provide physics-based recommendations for deep-learning based SGS modeling for generalizable closure modeling of turbulence.

Quantum computing is an advancing area of research in which computer hardware and algorithms are developed to take advantage of quantum mechanical phenomena. In recent studies, quantum algorithms have shown promise in solving linear systems of equations as well as systems of linear ordinary differential equations (ODEs) and partial differential equations (PDEs). Reduced-order modeling (ROM) algorithms for studying fluid dynamics have shown success in identifying linear operators that can describe flowfields, where dynamic mode decomposition (DMD) is a particularly useful method in which a linear operator is identified from data. In this work, DMD is reformulated as an optimization problem to propagate the state of the linearized dynamical system on a quantum computer. Quadratic unconstrained binary optimization (QUBO), a technique for optimizing quadratic polynomials in binary variables, allows for quantum annealing algorithms to be applied. A quantum circuit model (quantum approximation optimization algorithm, QAOA) is utilized to obtain predictions of the state trajectories. Results are shown for the quantum-ROM predictions for flow over a 2D cylinder at Re = 220 and flow over a NACA0009 airfoil at Re = 500 and $\alpha = 15^\circ{}$. The quantum-ROM predictions are found to depend on the number of bits utilized for a fixed point representation and the truncation level of the DMD model. Comparisons with DMD predictions from a classical computer algorithm are made, as well as an analysis of the computational complexity and prospects for future, more fault-tolerant quantum computers.