Search

Recently, neural networks (NNs) have become a powerful tool for detecting quantum phases of matter. Unfortunately, NNs are black boxes and only identify phases without elucidating their properties. Novel physics benefits most from insights about phases, traditionally extracted in spin systems using spin correlators. Here, we combine two approaches and design TetrisCNN, a convolutional NN with parallel branches using different kernels that detects the phases of spin systems and expresses their essential descriptors, called order parameters, in a symbolic form based on spin correlators. We demonstrate this on the example of snapshots of the one-dimensional transverse-field Ising model taken in various bases. We show also that TetrisCNN can detect more complex order parameters using the example of two-dimensional Ising gauge theory. This work can lead to the integration of NNs with quantum simulators to study new exotic phases of matter., Comment: 13 pages, 7 figures. Accepted as a poster at the NeurIPS ML4PS 2024 workshop. Example code is available at https://github.com/kcybinski/TetrisCNN_for_spin_systems

While overparameterization is known to benefit generalization, its impact on Out-Of-Distribution (OOD) detection is less understood. This paper investigates the influence of model complexity in OOD detection. We propose an expected OOD risk metric to evaluate classifiers confidence on both training and OOD samples. Leveraging Random Matrix Theory, we derive bounds for the expected OOD risk of binary least-squares classifiers applied to Gaussian data. We show that the OOD risk depicts an infinite peak, when the number of parameters is equal to the number of samples, which we associate with the double descent phenomenon. Our experimental study on different OOD detection methods across multiple neural architectures extends our theoretical insights and highlights a double descent curve. Our observations suggest that overparameterization does not necessarily lead to better OOD detection. Using the Neural Collapse framework, we provide insights to better understand this behavior. To facilitate reproducibility, our code will be made publicly available upon publication.

Achieving high-quality Magnetic Resonance Imaging (MRI) reconstruction at accelerated acquisition rates remains challenging due to the inherent ill-posed nature of the inverse problem. Traditional Compressed Sensing (CS) methods, while robust across varying acquisition settings, struggle to maintain good reconstruction quality at high acceleration factors ($\ge$ 8). Recent advances in deep learning have improved reconstruction quality, but purely data-driven methods are prone to overfitting and hallucination effects, notably when the acquisition setting is varying. Plug-and-Play (PnP) approaches have been proposed to mitigate the pitfalls of both frameworks. In a nutshell, PnP algorithms amount to replacing suboptimal handcrafted CS priors with powerful denoising deep neural network (DNNs). However, in MRI reconstruction, existing PnP methods often yield suboptimal results due to instabilities in the proximal gradient descent (PGD) schemes and the lack of curated, noiseless datasets for training robust denoisers. In this work, we propose a fully unsupervised preprocessing pipeline to generate clean, noiseless complex MRI signals from multicoil data, enabling training of a high-performance denoising DNN. Furthermore, we introduce an annealed Half-Quadratic Splitting (HQS) algorithm to address the instability issues, leading to significant improvements over existing PnP algorithms. When combined with preconditioning techniques, our approach achieves state-of-the-art results, providing a robust and efficient solution for high-quality MRI reconstruction.

Intracranial Hemorrhage is a potentially lethal condition whose manifestation is vastly diverse and shifts across clinical centers worldwide. Deep-learning-based solutions are starting to model complex relations between brain structures, but still struggle to generalize. While gathering more diverse data is the most natural approach, privacy regulations often limit the sharing of medical data. We propose the first application of Federated Scene Graph Generation. We show that our models can leverage the increased training data diversity. For Scene Graph Generation, they can recall up to 20% more clinically relevant relations across datasets compared to models trained on a single centralized dataset. Learning structured data representation in a federated setting can open the way to the development of new methods that can leverage this finer information to regularize across clients more effectively.

In this paper we explore which part of the ideal lattice of a general ring is parametrized by its Cuntz semigroup $\mathrm{S}(R)$ and its ambient semigroup $\Lambda(R)$. We identify these classes of ideals as the quasipure ideals (a generalization of pure ideals) in the case of $\mathrm{S}(R)$, and what we term decomposable ideals in the case of $\Lambda(R)$. For an ($s$-)unital ring $R$, the latter class exhausts all ideals of the ring. We prove that these constructions behave well with respect to quotients. In order to study the passage to inductive limits, we introduce the classes of dense and left normal rings. We show that $\mathrm{S}(R)$ is an abstract Cu-semigroup whenever $R$ is left normal and, for such rings, the assignment $R\mapsto \mathrm{S}(R)$ is continuous. We prove a parallel result for $\Lambda(R)$ whenever $R$ is a dense ring., Comment: 38 pages. Comments welcome!

We present the largest Galactic neutral hydrogen HI absorption survey to date, utilizing the Australian SKA Pathfinder Telescope at an unprecedented spatial resolution of 30''. This survey, GASKAP-HI, unbiasedly targets 2,714 continuum background sources over 250 square degrees in the direction of the Magellanic Clouds, a significant increase compared to a total of 373 sources observed by previous Galactic absorption surveys across the entire Milky Way. We aim to investigate the physical properties of cold (CNM) and warm (WNM) neutral atomic gas in the Milky Way foreground, characterized by two prominent filaments at high Galactic latitudes (between $-45^{\circ}$ and $-25^{\circ}$). We detected strong HI absorption along 462 lines of sight above the 3$\sigma$ threshold, achieving an absorption detection rate of 17%. GASKAP-HI's unprecedented angular resolution allows for simultaneous absorption and emission measurements to sample almost the same gas clouds along a line of sight. A joint Gaussian decomposition is then applied to absorption-emission spectra to provide direct estimates of HI optical depths, temperatures, and column densities for the CNM and WNM components. The thermal properties of CNM components are consistent with those previously observed along a wide range of Solar neighborhood environments, indicating that cold HI properties are widely prevalent throughout the local interstellar medium. Across our region of interest, CNM accounts for ~30% of the total HI gas, with the CNM fraction increasing with column density toward the two filaments. Our analysis reveals an anti-correlation between CNM temperature and its optical depth, which implies that CNM with lower optical depth leads to a higher temperature., Comment: Largest Galactic HI Absorption Survey To Date (GASKAP-HI): Cold Atomic Gas in the Magellanic Cloud foreground using Australian SKA Pathfinder. This paper has 19 pages, 17 figures. This paper has been accepted for publication in Monthly Notices of the Royal Astronomical Society Main Journal

Dense cold molecular cores/filaments are surrounded by an envelope of translucent gas. Some of the low-J emission lines of CO and HCO$^+$ isotopologues are more sensitive to the conditions either in the translucent environment or in the dense cold one. We propose a cloud model composed of three homogeneous slabs of gas along each line of sight (LoS), representing an envelope and a shielded inner layer. IRAM-30m data from the ORION-B large program toward the Horsehead nebula are used to demonstrate the method's capability. We use the non-LTE radiative transfer code RADEX to model the line profiles from the kinetic temperature $T_{kin}$, the volume density $n_{H_2}$, kinematics and chemical properties of the different layers. We then use a maximum likelihood estimator to simultaneously fit the lines of the CO and HCO$^+$ isotopologues. We constrain column density ratios to limit the variance on the estimates. This simple heterogeneous model provides good fits of the fitted lines over a large part of the cloud. The decomposition of the intensity into three layers allows to discuss the distribution of the estimated physical/chemical properties along the LoS. About 80$\%$ the CO integrated intensity comes from the envelope, while $\sim55\%$ of that of the (1-0) and (2-1) lines of C$^{18}$O comes from the inner layer. The $N(^{13}CO)/N(C^{18}O)$ in the envelope increases with decreasing $A_v$, and reaches $25$ in the pillar outskirts. The envelope $T_{kin}$ varies from 25 to 40 K, that of the inner layer drops to $\sim 15$ K in the western dense core. The inner layer $n_{H_2}$ is $\sim 3\times10^4\,\text{cm}^{-3}$ toward the filament and it increases by a factor $10$ toward dense cores. The proposed method correctly retrieves the physical/chemical properties of the Horsehead nebula and offers promising prospects for less supervised model fits of wider-field datasets.

Observations of ionic, atomic, or molecular lines are performed to improve our understanding of the interstellar medium (ISM). However, the potential of a line to constrain the physical conditions of the ISM is difficult to assess quantitatively, because of the complexity of the ISM physics. The situation is even more complex when trying to assess which combinations of lines are the most useful. Therefore, observation campaigns usually try to observe as many lines as possible for as much time as possible. We search for a quantitative statistical criterion to evaluate the constraining power of a (or combination of) tracer(s) with respect to physical conditions in order to improve our understanding of the statistical relationships between ISM tracers and physical conditions and helps observers to motivate their observation proposals. The best tracers are obtained by comparing the mutual information between a physical parameter and different sets of lines. We apply this method to simulations of radio molecular lines emitted by a photodissociation region similar to the Horsehead Nebula that would be observed at the IRAM 30m telescope. We search for the best lines to constrain the visual extinction $A_v^{tot}$ or the far UV illumination $G_0$. The most informative lines change with the physical regime (e.g., cloud extinction). Short integration time of the CO isotopologue $J=1-0$ lines already yields much information on the total column density most regimes. The best set of lines to constrain the visual extinction does not necessarily combine the most informative individual lines. Precise constraints on $G_0$ are more difficult to achieve with molecular lines. They require spectral lines emitted at the cloud surface (e.g., [CII] and [CI] lines). This approach allows one to better explore the knowledge provided by ISM codes, and to guide future observation campaigns.

While the field of inverse graphics has been witnessing continuous growth, techniques devised thus far predominantly focus on learning individual scene representations. In contrast, learning large sets of scenes has been a considerable bottleneck in NeRF developments, as repeatedly applying inverse graphics on a sequence of scenes, though essential for various applications, remains largely prohibitive in terms of resource costs. We introduce a framework termed "scaled inverse graphics", aimed at efficiently learning large sets of scene representations, and propose a novel method to this end. It operates in two stages: (i) training a compression model on a subset of scenes, then (ii) training NeRF models on the resulting smaller representations, thereby reducing the optimization space per new scene. In practice, we compact the representation of scenes by learning NeRFs in a latent space to reduce the image resolution, and sharing information across scenes to reduce NeRF representation complexity. We experimentally show that our method presents both the lowest training time and memory footprint in scaled inverse graphics compared to other methods applied independently on each scene. Our codebase is publicly available as open-source. Our project page can be found at https://scaled-ig.github.io .

The idea of fractional derivatives has a long history that dates back centuries. Apart from their intriguing mathematical properties, fractional derivatives have been studied widely in physics, for example in quantum mechanics and generally in systems with nonlocal temporal or spatial interactions. However, systematic experiments have been rare due to challenges associated with the physical implementation. Here we report the observation and full characterization of a family of temporal optical solitons that are governed by a nonlinear wave equation with a fractional Laplacian. This equation has solutions with unique properties such as non-exponential tails and a very small time-bandwidth product.

In the era of Industry 4.0, system reliability engineering faces both challenges and opportunities. On the one hand, the complexity of cyber-physical systems, the integration of novel numerical technologies, and the handling of large amounts of data pose new difficulties for ensuring system reliability. On the other hand, innovations such as AI-driven prognostics, digital twins, and IoT-enabled systems enable the implementation of new methodologies that are transforming reliability engineering. Condition-based monitoring and predictive maintenance are examples of key advancements, leveraging real-time sensor data collection and AI to predict and prevent equipment failures. These approaches reduce failures and downtime, lower costs, and extend equipment lifespan and sustainability. However, it also brings challenges such as data management, integrating complexity, and the need for fast and accurate models and algorithms. Overall, the convergence of advanced technologies in Industry 4.0 requires a rethinking of reliability tasks, emphasising adaptability and real-time data processing. In this chapter, we propose to review recent innovations in the field, related methods and applications, as well as challenges and barriers that remain to be explored. In the red lane, we focus on smart manufacturing and automotive engineering applications with sensor-based monitoring and driver assistance systems., Comment: 34 pages, 19 figures

We study the ground state energy of trapped two-dimensional Bose gases with mean-field type interactions. We prove the stability of second kind of the many-body system and the convergence of the ground state energy per particle to that of a non-linear Schr\"odinger (NLS) energy functional. Notably, we can take the scaling of the interaction to be arbitrarily close to the Gross-Pitaevskii scaling, which is a drastic improvement on the best-known result for systems with attractive interactions. As a consequence of the stability of second kind we also obtain Bose-Einstein condensation for the many-body ground states and dynamics for a much improved range of the diluteness parameter., Comment: 16 pages

While pre-trained image autoencoders are increasingly utilized in computer vision, the application of inverse graphics in 2D latent spaces has been under-explored. Yet, besides reducing the training and rendering complexity, applying inverse graphics in the latent space enables a valuable interoperability with other latent-based 2D methods. The major challenge is that inverse graphics cannot be directly applied to such image latent spaces because they lack an underlying 3D geometry. In this paper, we propose an Inverse Graphics Autoencoder (IG-AE) that specifically addresses this issue. To this end, we regularize an image autoencoder with 3D-geometry by aligning its latent space with jointly trained latent 3D scenes. We utilize the trained IG-AE to bring NeRFs to the latent space with a latent NeRF training pipeline, which we implement in an open-source extension of the Nerfstudio framework, thereby unlocking latent scene learning for its supported methods. We experimentally confirm that Latent NeRFs trained with IG-AE present an improved quality compared to a standard autoencoder, all while exhibiting training and rendering accelerations with respect to NeRFs trained in the image space. Our project page can be found at https://ig-ae.github.io .

In our urbanised societies, the management and regulation of traffic and pedestrian flows is of considerable interest for public safety, economic development, and the conservation of the environment. However, modelling and controlling the collective dynamics of vehicles and pedestrians raises several challenges. Not only are the individual entities self-propelled and hard to describe, but their complex nonlinear physical and social interactions makes the multi-agent problem of crowd and traffic flow even more involved. In this chapter, we purport to review the suitability and limitations of classical modelling approaches through four examples of collective behaviour: stop-and-go waves in traffic flow, lane formation, long-term avoidance behaviour, and load balancing in pedestrian dynamics. While stop-and-go dynamics and lane formation can both be addressed by basic reactive models (at least to some extent), the latter two require anticipation and/or coordination at the level of the group. The results highlight the limitations of classical force-based models, but also the need for long-term anticipation mechanisms and multiscale modelling approaches. In response, we review new developments and modelling concepts., Comment: 20 pages, 6 figures

Supernova shocks into dense molecular cores in IC 443 (clumps B, C, and G) and 3C 391 were observed using the Stratospheric Observatory for Infrared Astronomy and complemented by archival data from the Herschel Space Observatory. The pure rotational transitions 0-0 S(1) and S(5) of H2, and the ground-state 110-101 transition of H2O, are all broadened, arising from molecules that survive the passage of the shock front. Theoretical models from the Paris-Durham shock code were analyzed to generate synthetic profiles that approximately match the observations. The observations can be fit with two shock conditions, which approximate the range of densities in the pre-shock molecular cloud. The width and brightness of the S(5) lines require shocks into gas with a density of order 2,000 cm-3, into which the IC 443 blast wave drives shocks with speed 60 km/s. The brightness and narrower width of the S(1) lines requires different shocks, into gas with density of order 10^5 cm-3, with shock speeds of 10 km/s. The H2O velocity distribution is also consistent with these shocks. The existence of shocks into dense gas shows that the bright shocked clumps in IC~443 were prestellar cores. It is unlikely that they will form stars soon after the passage of the shock front, given the input of kinetic and thermal energy from the shocks., Comment: accepted to Astrophysical Journal on 10/29/2024

Achieving the EU's climate neutrality goal requires retrofitting existing buildings to reduce energy use and emissions. A critical step in this process is the precise assessment of geometric building envelope characteristics to inform retrofitting decisions. Previous methods for estimating building characteristics, such as window-to-wall ratio, building footprint area, and the location of architectural elements, have primarily relied on applying deep-learning-based detection or segmentation techniques on 2D images. However, these approaches tend to focus on planar facade properties, limiting their accuracy and comprehensiveness when analyzing complete building envelopes in 3D. While neural scene representations have shown exceptional performance in indoor scene reconstruction, they remain under-explored for external building envelope analysis. This work addresses this gap by leveraging cutting-edge neural surface reconstruction techniques based on signed distance function (SDF) representations for 3D building analysis. We propose BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs. By integrating SDF-based representation with semantic modality, BuildNet3D recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics. Our framework is evaluated on a range of complex building structures, demonstrating high accuracy and generalizability in estimating window-to-wall ratio and building footprint. The results underscore the effectiveness of BuildNet3D for practical applications in building analysis and retrofitting.

In this work, we present a new unsupervised anomaly (outlier) detection (AD) method using the sliced-Wasserstein metric. This filtering technique is conceptually interesting for MLOps pipelines deploying machine learning models in critical sectors, e.g., energy, as it offers a conservative data selection. Additionally, we open the first dataset showcasing localized critical peak rebate demand response in a northern climate. We demonstrate the capabilities of our method on synthetic datasets as well as standard AD datasets and use it in the making of a first benchmark for our open-source localized critical peak rebate dataset.

This study explores the integration of large language models (LLMs) into classic inflation nowcasting frameworks, particularly in light of high inflation volatility periods such as the COVID-19 pandemic. We propose InflaBERT, a BERT-based LLM fine-tuned to predict inflation-related sentiment in news. We use this model to produce NEWS, an index capturing the monthly sentiment of the news regarding inflation. Incorporating our expectation index into the Cleveland Fed's model, which is only based on macroeconomic autoregressive processes, shows a marginal improvement in nowcast accuracy during the pandemic. This highlights the potential of combining sentiment analysis with traditional economic indicators, suggesting further research to refine these methodologies for better real-time inflation monitoring. The source code is available at https://github.com/paultltc/InflaBERT.

Using density functional theory calculations with spin-orbit coupling (SOC), we report on the temperature-dependent thermodynamical properties of Pb: electrical resistivity, thermal expansion (TE), heat capacity, bulk modulus and its pressure derivative. For the former, we employed the state-of-the-art ab initio Boltzmann Transport Equation formalism, and we calculated the effect of TE. In accordance with previous work, we show that SOC improves the description of the phonon dispersion and the resistivity. We argue that this is caused by a joint mutual effect of an increase in the electronic nesting and an increase in the electron-phonon coupling. Interestingly, including TE incorporates non-linearity into the resistivity at high temperatures, whose magnitude depends on whether SOC is included or not. We suggest that mechanisms beyond the quasi-harmonic approximation should be considered to get a better description of Pb with SOC at high temperatures., Comment: 19 pages, 10 figures

With the aim of progressing toward a practical implementation of an effective quantum-electrodynamics (QED) theory of atoms and molecules, which includes the effects of vacuum polarization through the creation of virtual electron-positron pairs but without the explicit photon degrees of freedom, we study a one-dimensional effective QED model of the hydrogen-like atom with delta-potential interactions. This model resembles the three-dimensional effective QED theory with Coulomb interactions while being substantially simpler. We provide some mathematical details about the definition of this model, calculate the vacuum-polarization density, and the Lamb-type shift of the bound-state energy, correcting and extending results of previous works. We also study the approximation of the model in a finite plane-wave basis, and in particular we discuss the basis convergence of the bound-state energy and eigenfunction, of the vacuum-polarization density, and of the Lamb-type shift of the bound-state energy. We highlight the difficulty of converging the vacuum-polarization density in a finite basis and we propose a way to improve it. The present work could give hints on how to perform similar calculations for the three-dimensional effective QED theory of atoms and molecules.

Laser-Wakefield Accelerators (LWFA) running at kHz repetition rates hold great potential for applications. They typically operate with low-energy, highly compressed laser pulses focused in high-pressure gas targets. Experiments have shown that the best-quality electron beams are achieved using Hydrogen gas targets. However, continuous operation with Hydrogen requires a dedicated pumping system. This work presents the design of a differential pumping system, enabling, for the first time, continuous operation of our kHz LWFA using a high-pressure Hydrogen gas jet. The system successfully maintained a pressure below 3e-4 mbar, even with a free-flowing gas jet operating at 140 bar backing pressure. Numerical fluid dynamics and optical simulations were used to guide and validate the system's design., Comment: 10 pages, 9 figures

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

The exponential growth of IoT networks necessitates a paradigm shift towards architectures that offer high flexibility and learning capabilities while maintaining low energy consumption, minimal communication overhead, and low latency. Traditional IoT systems, particularly when integrated with machine learning approaches, often suffer from high communication overhead and significant energy consumption. This work addresses these challenges by proposing a neuromorphic architecture inspired by biological systems. To illustrate the practical application of our proposed architecture, we present a case study focusing on water management in the Carinthian community of Neuhaus. Preliminary results regarding water consumption prediction and anomaly detection in this community are presented. We also introduce a novel neuromorphic IoT architecture that integrates biological principles into the design of IoT systems. This architecture is specifically tailored for edge computing scenarios, where low power and high efficiency are crucial. Our approach leverages the inherent advantages of neuromorphic computing, such as asynchronous processing and event-driven communication, to create an IoT framework that is both energy-efficient and responsive. This case study demonstrates how the neuromorphic IoT architecture can be deployed in a real-world scenario, highlighting its benefits in terms of energy savings, reduced communication overhead, and improved system responsiveness., Comment: Submitted to Edge AI conference, Cagliari, Italy

We construct a mean-field model that describes the nonlinear dynamics of a spin-polarized electron gas interacting with fixed, positively-charged ions possessing a magnetic moment that evolves in time. The mobile electrons are modeled by a four-component distribution function in the two-dimensional phase space $(x,v)$, obeying a Vlasov-Poisson set of equations. The ions are modeled by a Landau-Lifshitz equation for their spin density, which contains ion-ion and electron-ion magnetic exchange terms. We perform a linear response study of the coupled Vlasov-Poisson-Landau-Lifshitz (VPLL) equations for the case of a Maxwell-Boltzmann equilibrium, focussing in particular on the spin dispersion relation. Condition of stability or instability for the spin modes are identified, which depend essentially on the electron spin polarization rate $\eta$ and the electron-ion magnetic coupling constant $K$. We also develop an Eulerian grid-based computational code for the fully nonlinear VPLL equations, based on the geometric Hamiltonian method first developed in [N. Crouseilles et al. Journal of Plasma Physics, 89(2):905890215, 2023]. This technique allows us to achieve great accuracy for the conserved quantities, such as the modulus of the ion spin vector and the total energy. Numerical tests in the linear regime are in accordance with the estimations of the linear response theory. For two-stream equilibria, we study the interplay of instabilities occurring in both the charge and the spin sectors. The set of parameters used in the simulations, with densities close to those of solids ($\approx 10^{29} \rm m^{-3}$) and temperatures of the order of 10 eV, may be relevant to the warm dense matter regime appearing in some inertial fusion experiments.

Current progress in artificial intelligence is centered around so-called large language models that consist of neural networks processing long sequences of high-dimensional vectors called tokens. Statistical physics provides powerful tools to study the functioning of learning with neural networks and has played a recognized role in the development of modern machine learning. The statistical physics approach relies on simplified and analytically tractable models of data. However, simple tractable models for long sequences of high-dimensional tokens are largely underexplored. Inspired by the crucial role models such as the single-layer teacher-student perceptron (aka generalized linear regression) played in the theory of fully connected neural networks, in this paper, we introduce and study the bilinear sequence regression (BSR) as one of the most basic models for sequences of tokens. We note that modern architectures naturally subsume the BSR model due to the skip connections. Building on recent methodological progress, we compute the Bayes-optimal generalization error for the model in the limit of long sequences of high-dimensional tokens, and provide a message-passing algorithm that matches this performance. We quantify the improvement that optimal learning brings with respect to vectorizing the sequence of tokens and learning via simple linear regression. We also unveil surprising properties of the gradient descent algorithms in the BSR model.

The paper describes the algebraic structure of the graded algebra of differentially homogeneous polynomials of fixed finite order. We show that it is a finitely generated algebra, and we exhibit a minimal set of generators. Along the way, we provide a simpler proof of the so-called Schmidt--Kolchin conjecture (proved in a previous paper) . From the algebraic point of view, this provides natural compactifications of jet bundles of projective spaces. From the invariant theoretic point of view, this provides new examples, not covered (to our knowledge) by known conjectures in the subject, of unipotent sub-groups of general linear groups, whose algebras of invariants are finitely generated (and more precisely gives a First Fundamental Theorem for such groups, following the terminology in Invariant Theory)., Comment: 18 pages

Given a sequence of $d \times d$ symmetric matrices $\{\mathbf{W}_i\}_{i=1}^n$, and a margin $\Delta > 0$, we investigate whether it is possible to find signs $(\epsilon_1, \dots, \epsilon_n) \in \{\pm 1\}^n$ such that the operator norm of the signed sum satisfies $\|\sum_{i=1}^n \epsilon_i \mathbf{W}_i\|_\mathrm{op} \leq \Delta$. Kunisky and Zhang (2023) recently introduced a random version of this problem, where the matrices $\{\mathbf{W}_i\}_{i=1}^n$ are drawn from the Gaussian orthogonal ensemble. This model can be seen as a random variant of the celebrated Matrix Spencer conjecture and as a matrix-valued analog of the symmetric binary perceptron in statistical physics. In this work, we establish a satisfiability transition in this problem as $n, d \to \infty$ with $n / d^2 \to \tau > 0$. Our contributions are twofold. First, we prove that the expected number of solutions with margin $\Delta=\kappa \sqrt{n}$ has a sharp threshold at a critical $\tau_1(\kappa)$: for $\tau < \tau_1(\kappa)$ the problem is typically unsatisfiable, while for $\tau > \tau_1(\kappa)$ the average number of solutions becomes exponentially large. Second, combining a second-moment method with recent results from Altschuler (2023) on margin concentration in perceptron-type problems, we identify a second threshold $\tau_2(\kappa)$, such that for $\tau>\tau_2(\kappa)$ the problem admits solutions with high probability. In particular, we establish for the first time that a system of $n = \Theta(d^2)$ Gaussian random matrices can be balanced so that the spectrum of the resulting matrix macroscopically shrinks compared to the typical semicircle law. Our proofs rely on deriving concentration inequalities and large deviation properties for the law of correlated Gaussian matrices under spectral norm constraints, extending results in the literature and offering insights of independent interest., Comment: 29 pages, 2 figures

Variational Autoencoders (VAE) are popular generative models used to sample from complex data distributions. Despite their empirical success in various machine learning tasks, significant gaps remain in understanding their theoretical properties, particularly regarding convergence guarantees. This paper aims to bridge that gap by providing non-asymptotic convergence guarantees for VAE trained using both Stochastic Gradient Descent and Adam algorithms.We derive a convergence rate of $\mathcal{O}(\log n / \sqrt{n})$, where $n$ is the number of iterations of the optimization algorithm, with explicit dependencies on the batch size, the number of variational samples, and other key hyperparameters. Our theoretical analysis applies to both Linear VAE and Deep Gaussian VAE, as well as several VAE variants, including $\beta$-VAE and IWAE. Additionally, we empirically illustrate the impact of hyperparameters on convergence, offering new insights into the theoretical understanding of VAE training.

Creativity is the ability to produce novel, useful, and surprising ideas, and has been widely studied as a crucial aspect of human cognition. Machine creativity on the other hand has been a long-standing challenge. With the rise of advanced generative AI, there has been renewed interest and debate regarding AI's creative capabilities. Therefore, it is imperative to revisit the state of creativity in AI and identify key progresses and remaining challenges. In this work, we survey leading works studying the creative capabilities of AI systems, focusing on creative problem-solving, linguistic, artistic, and scientific creativity. Our review suggests that while the latest AI models are largely capable of producing linguistically and artistically creative outputs such as poems, images, and musical pieces, they struggle with tasks that require creative problem-solving, abstract thinking and compositionality and their generations suffer from a lack of diversity, originality, long-range incoherence and hallucinations. We also discuss key questions concerning copyright and authorship issues with generative models. Furthermore, we highlight the need for a comprehensive evaluation of creativity that is process-driven and considers several dimensions of creativity. Finally, we propose future research directions to improve the creativity of AI outputs, drawing inspiration from cognitive science and psychology., Comment: minor updates to content + figures

Hadronic contributions dominate the uncertainty of the Standard Model prediction for the anomalous magnetic moment of the muon. In this work, we present results on the hadronic light-by-light contribution obtained from the evaluation of the hadronic four-point function of electromagnetic currents using the position-space formalism developed by the Mainz group. The simulations are performed with staggered fermions directly at the physical point. Several physical volumes are used to estimate finite volume effects. This direct lattice study is supplemented by considering the contribution of the light pseudoscalar pole in both finite and infinite volumes, where we reuse the pseudoscalar transition form factors that have been evaluated in previous simulations on the same ensembles., Comment: 7 pages, 6 figures, LATTICE2024 conference proceedings, submitted to PoS

Reinforcement Learning from Human Feedback (RLHF) has become a popular approach to align language models (LMs) with human preferences. This method involves collecting a large dataset of human pairwise preferences across various text generations and using it to infer (implicitly or explicitly) a reward model. Numerous methods have been proposed to learn the reward model and align a LM with it. However, the costly process of collecting human preferences has received little attention and could benefit from theoretical insights. This paper addresses this issue and aims to formalize the reward training model in RLHF. We frame the selection of an effective dataset as a simple regret minimization task, using a linear contextual dueling bandit method. Given the potentially large number of arms, this approach is more coherent than the best-arm identification setting. We then propose an offline framework for solving this problem. Under appropriate assumptions - linearity of the reward model in the embedding space, and boundedness of the reward parameter - we derive bounds on the simple regret. Finally, we provide a lower bound that matches our upper bound up to constant and logarithmic terms. To our knowledge, this is the first theoretical contribution in this area to provide an offline approach as well as worst-case guarantees.

In this paper, we study the ability of large language models to learn specific mathematical rules such as distributivity or simplifying equations. We present an empirical analysis of their ability to generalize these rules, as well as to reuse them in the context of word problems. For this purpose, we provide a rigorous methodology to build synthetic data incorporating such rules, and perform fine-tuning of large language models on such data. Our experiments show that our model can learn and generalize these rules to some extent, as well as suitably reuse them in the context of word problems., Comment: NeurIPS'24 MATH-AI, the 4th Workshop on Mathematical Reasoning and AI

A field $K$ is quasi-classical $d$-local if there exist fields $K=k_d,\dots,k_0$ with $k_{i+1}$ Henselian admissible discretely valued with residue field $k_i$, and $k_0$ quasi-finite. We prove a duality theorem for the Galois cohomology of such $K$ with many coefficients, including finite coefficients of any order. Previously, such duality was only known in few cases : as a perfect pairing of finite groups for finite coefficients prime to $\mathrm{char} k_0$ in general, or for any finite coefficients when $k_1$ is $p$-adic ; or as a perfect pairing of locally compact Hausdorff groups for the $\mathrm{fppf}$ cohomology of finite group schemes when $K$ is local. With no obvious reasonable topology available, we abandon perfectness altogether and instead obtain nondegenerate pairings of abstract abelian groups. This is done with new diagram-chasing results for pairings of torsion groups, allowing a d\'evissage approach which reduces our results to the study of $K^M_r(K)/p\times H^{d+1-r}_p(K)\to\mathbb{Z}/p$ using results of Kato., Comment: 55 pages, comments welcome ! Version 2 : generalized from "complete" to "Henselian admissible" (i.e. the completion is a separable extension) in the definition of higher local fields and subsequent results ; included duality results for general coefficients of finite type ; corrected some typos and errors

We report on continuous high-harmonic generation at 1 kHz repetition rate from a liquid-sheet plasma mirror driven by relativistic-intensity near-single-cycle light transients. Through precise control of both the surface plasma density gradient and the driving light waveform, we can produce highly stable and reproducible extreme ultraviolet spectral quasi-continua, corresponding to the generation of stable kHz-trains of isolated attosecond pulses in the time domain. This confirms the exciting potential of liquid sheet targets as one of the building blocks of future high-power attosecond lasers.

Optical-based terahertz sources are important for many burgeoning scientific and technological applications. Among such applications is precision spectroscopy of molecules, which exhibit rotational transitions at terahertz frequencies. Stemming from precision spectroscopy is frequency discrimination and stabilization of terahertz sources. Because many molecular species exist in the gas phase at room temperature, their transitions are prime candidates for practical terahertz frequency references. We demonstrate the stabilization of a low phase-noise, dual-wavelength Brillouin laser (DWBL) terahertz oscillator to a rotational transition of carbonyl sulfide (\ce{OCS}). We achieve an instability of $1.2\times10^{-12}/\sqrt{\tau}$, where $\tau$ is the averaging time in seconds. The signal-to-noise ratio and intermodulation limitations of the experiment are also discussed. We thus demonstrate a highly stable and spectrally pure terahertz frequency source. Our presented architecture will likely benefit metrology, spectroscopy, precision terahertz studies, and beyond.

A unified algebraic interpretation of both finite families of orthogonal polynomials and biorthogonal rational functions of $q$-Hahn type is provided. The approach relies on the meta $q$-Hahn algebra and its finite-dimensional bidiagonal representations. The functions of $q$-Hahn type are identified as overlaps (up to global factors) between bases solving ordinary or generalized eigenvalue problems in the representation of the meta $q$-Hahn algebra. Moreover, (bi)orthogonality relations, recurrence relations, difference equations and some contiguity relations satisfied by these functions are recovered algebraically using the actions of the generators of the meta $q$-Hahn algebra on various bases., Comment: 27 pages

Transcranial ultrasound imaging is usually limited by skull-induced attenuation and high-order aberrations. By using contrast agents such as microbubbles in combination with ultrafast imaging, not only can the signal-to-noise ratio be improved, but super-resolution images down to the micrometer scale of the brain vessels can be obtained. However, ultrasound localization microscopy (ULM) remains impacted by wave-front distortions that limit the microbubble detection rate and hamper their localization. In this work, we show how matrix imaging, which relies on the prior recording of the reflection matrix, can provide a solution to those fundamental issues. As an experimental proof-of-concept, an in-vivo reconstruction of deep brain microvessels is performed on three anesthetized sheeps. The compensation of wave distortions is shown to drastically enhance the contrast and resolution of ULM. This experimental study thus opens up promising perspectives for a transcranial and non-ionizing observation of human cerebral microvascular pathologies, such as stroke., Comment: 43 pages, 11 figures, 3 tables

We propose a novel approach to quantify quantum coherence which, contrary to the previous ones, does not rely on resource theory but rather on ontological considerations. In this framework, coherence is understood as the ability for a quantum system's statistics to deviate from the total probability formula. After motivating the importance of the total probability formula in quantum foundations, we propose a new set of axioms that a measure of coherence should satisfy, and show that it defines a class of measures different from the main previous proposal. Finally, we prove a general result about the dependence of the l2-coherence norm on the basis of interest, and show that it is well approximated by the square root of the purity in most bases.

Large language models (LLMs) have demonstrated significant progress in various natural language generation and understanding tasks. However, their linguistic generalization capabilities remain questionable, raising doubts about whether these models learn language similarly to humans. While humans exhibit compositional generalization and linguistic creativity in language use, the extent to which LLMs replicate these abilities, particularly in morphology, is under-explored. In this work, we systematically investigate the morphological generalization abilities of LLMs through the lens of compositionality. We define morphemes as compositional primitives and design a novel suite of generative and discriminative tasks to assess morphological productivity and systematicity. Focusing on agglutinative languages such as Turkish and Finnish, we evaluate several state-of-the-art instruction-finetuned multilingual models, including GPT-4 and Gemini. Our analysis shows that LLMs struggle with morphological compositional generalization particularly when applied to novel word roots, with performance declining sharply as morphological complexity increases. While models can identify individual morphological combinations better than chance, their performance lacks systematicity, leading to significant accuracy gaps compared to humans., Comment: 33 pages

In Deep Reinforcement Learning models trained using gradient-based techniques, the choice of optimizer and its learning rate are crucial to achieving good performance: higher learning rates can prevent the model from learning effectively, while lower ones might slow convergence. Additionally, due to the non-stationarity of the objective function, the best-performing learning rate can change over the training steps. To adapt the learning rate, a standard technique consists of using decay schedulers. However, these schedulers assume that the model is progressively approaching convergence, which may not always be true, leading to delayed or premature adjustments. In this work, we propose dynamic Learning Rate for deep Reinforcement Learning (LRRL), a meta-learning approach that selects the learning rate based on the agent's performance during training. LRRL is based on a multi-armed bandit algorithm, where each arm represents a different learning rate, and the bandit feedback is provided by the cumulative returns of the RL policy to update the arms' probability distribution. Our empirical results demonstrate that LRRL can substantially improve the performance of deep RL algorithms.

The Leaf Area Index (LAI) is a critical parameter to understand ecosystem health and vegetation dynamics. In this paper, we propose a novel method for pixel-wise LAI prediction by leveraging the complementary information from Sentinel 1 radar data and Sentinel 2 multi-spectral data at multiple timestamps. Our approach uses a deep neural network based on multiple U-nets tailored specifically to this task. To handle the complexity of the different input modalities, it is comprised of several modules that are pre-trained separately to represent all input data in a common latent space. Then, we fine-tune them end-to-end with a common decoder that also takes into account seasonality, which we find to play an important role. Our method achieved 0.06 RMSE and 0.93 R2 score on publicly available data. We make our contributions available at https://github.com/valentingol/LeafNothingBehind for future works to further improve on our current progress.

We propose a simple method to estimate the parameters of a continuously measured quantum system, by fitting correlation functions of the measured signal. We demonstrate the approach in simulation, both on toy examples and on a recent superconducting circuits experiment which proved particularly difficult to characterise using conventional methods. The idea is applicable to any system whose evolution is described by a jump or diffusive stochastic master equation. It allows the simultaneous estimation of many parameters, is practical for everyday use, is suitable for large Hilbert space dimensions, and takes into account experimental constraints such as detector imperfections and signal filtering and digitisation. Unlike existing methods, it also provides a direct way to understand how each parameter is estimated from the measured signal. This makes the approach interpretable, facilitates debugging, and enables validating the adequacy of a model with the observed data., Comment: 26 pages, 15 figures

We theoretically and numerically study the evolution of soliton-like waves supported by stimulated Brillouin scattering. First, the emergence and unusual behaviour of resonant solitary waves are investigated for both backward and forward three wave interactions. We find that these waves can be characterized by the ratio between the optical and acoustic damping coefficients. We also examine a second class of non-resonant anti-symmetric soliton-like waves, which have a more complicated pulse shape than traditional solitons. These waves are superluminal, with pulse velocities that can be tuned by the input Stokes and pump fields. We discuss the excitation of these types of waves and the physical conditions required for their observation., Comment: 9 pages, 7 figures