Search

We consider the inverse problem consisting of the reconstruction of an inclusion $B$ contained in a bounded domain $\Omega\subset\mathbb{R}^d$ from a single pair of Cauchy data $(u|_{\partial\Omega},\partial_\nu u|_{\partial\Omega})$, where $\Delta u=0$ in $\Omega\setminus\overline B$ and $u=0$ on $\partial B$. We show that the reconstruction algorithm based on the range test, a domain sampling method, can be written as a neural network with a specific architecture. We propose to learn the weights of this network in the framework of supervised learning, and to combine it with a pre-trained classifier, with the purpose of distinguishing the inclusions based on their distance from the boundary. The numerical simulations show that this learned range test method provides accurate and stable reconstructions of polygonal inclusions. Furthermore, the results are superior to those obtained with the standard range test method (without learning) and with an end-to-end fully connected deep neural network, a purely data-driven method., Comment: 25 pages, 12 figures

We remove the dependence on the `hot-spots' conjecture in two of the main theorems of the recent paper of Nickl (2024, Annals of Statistics). Specifically, we characterise the minimax convergence rates for estimation of the transition operator $P_{f}$ arising from the Neumann Laplacian with diffusion coefficient $f$ on arbitrary convex domains with smooth boundary, and further show that a general Lipschitz stability estimate holds for the inverse map $P_f\mapsto f$ from $H^2\to H^2$ to $L^1$.

Stability is a key property of both forward models and inverse problems, and depends on the norms considered in the relevant function spaces. For instance, stability estimates for hyperbolic partial differential equations are often based on energy conservation principles, and are therefore expressed in terms of $L^2$ norms. The focus of this paper is on stability with respect to the $L^\infty$ norm, which is more relevant to detect localized phenomena. The linear wave equation is not stable in $L^\infty$, and we design an alternative solution method based on the regularization of Fourier multipliers, which is stable in $L^\infty$. Furthermore, we show how these ideas can be extended to inverse problems, and design a regularization method for the inversion of compact operators that is stable in $L^\infty$. We also discuss the connection with the stability of deep neural networks modeled by hyperbolic PDEs., Comment: 29 pages, 8 figures

In trapped-atom quantum computers, high-fidelity control of optical qubits is challenging due to the motion of atoms in the trap. If not corrected, the atom motion gets entangled with the qubit degrees of freedom through two fundamental mechanisms, (i) photon recoil and (ii) thermal motion, both leading to a reduction of the gate fidelity. We develop motion-insensitive pulses that suppress both sources of infidelity by modulating the phase of the driving laser field in time. To eliminate photon recoil, we use bang-bang pulses$-$derived using time-optimal control$-$which shorten the gate duration by about 20 times compared to conventional pulses. However, even when photon recoil is eliminated, we find that the gate error does not vanish, but is rather limited by a bound arising from thermal motion-induced entanglement. Remarkably, this bound is independent of the Rabi frequency, meaning that, unlike for photon recoil, operating in the resolved sideband regime does not mitigate this source of infidelity. To overcome this bound, we derive smooth-phase pulses, which allow for a further reduction of the gate error by more than an order of magnitude for typical thermal atoms. Motion-insensitive pulses can be refined to compensate for laser inhomogeneities, enhancing the gate performance in practical situations. Our results are validated through simulations of one-qubit gates operating on the optical clock transition of ${}^{88}$Sr atoms trapped in an optical tweezers array., Comment: 22 pages, 10 figures

In neutral atom quantum computers, readout and preparation of the atomic qubits are usually based on fluorescence imaging and subsequent analysis of the acquired image. For each atom site, the brightness or some comparable metric is estimated and used to predict the presence or absence of an atom. Across different setups, we can see a vast number of different approaches used to analyze these images. Often, the choice of detection algorithm is either not mentioned at all or it is not justified. We investigate several different algorithms and compare their performance in terms of both precision and execution run time. To do so, we rely on a set of synthetic images across different simulated exposure times with known occupancy states. Since the use of simulation provides us with the ground truth of atom site occupancy, we can easily state precise error rates and variances of the reconstructed property. To also rule out the possibility of better algorithms existing, we calculated the Cram\'er-Rao bound in order to establish an upper limit that even a perfect estimator cannot outperform. As the metric of choice, we used the number of photonelectrons that can be contributed to a specific atom site. Since the bound depends on the occupancy of neighboring sites, we provide the best and worst cases, as well as a half filled one. Our comparison shows that of our tested algorithms, a global non-linear least-squares solver that uses the optical system's PSF to return a each sites' number of photoelectrons performed the best, on average crossing the worst-case bound for longer exposure times. Its main drawback is its huge computational complexity and, thus, required calculation time. We manage to somewhat reduce this problem, suggesting that its use may be viable. However, our study also shows that for cases where utmost speed is required, simple algorithms may be preferable., Comment: 10 pages, 11 figures

In recent years, computational improvements have allowed for more nuanced, data-driven and geographically explicit agent-based simulations. So far, simulations have struggled to adequately represent the attributes that motivate the actions of the agents. In fact, existing population synthesis frameworks generate agent profiles limited to socio-demographic attributes. In this paper, we introduce a novel value-enriched population synthesis framework that integrates a motivational layer with the traditional individual and household socio-demographic layers. Our research highlights the significance of extending the profile of agents in synthetic populations by incorporating data on values, ideologies, opinions and vital priorities, which motivate the agents' behaviour. This motivational layer can help us develop a more nuanced decision-making mechanism for the agents in social simulation settings. Our methodology integrates microdata and macrodata within different Bayesian network structures. This contribution allows to generate synthetic populations with integrated value systems that preserve the inherent socio-demographic distributions of the real population in any specific region.

We propose a scheme to perform optical pulses that suppress the effect of photon recoil by three orders of magnitude compared to ordinary pulses in the Lamb-Dicke regime. We derive analytical insight about the fundamental limits to the fidelity of optical qubits for trapped atoms and ions. This paves the way towards applications in quantum computing for realizing $>1000$ of gates with an overall fidelity above 99\%., Comment: 5 pages plus appendices and bibliography, 6 figures

The first generation of ELT instruments includes an optical-infrared high-resolution spectrograph, indicated as ELT-HIRES and recently christened ANDES (ArmazoNes high Dispersion Echelle Spectrograph). ANDES consists of three fibre-fed spectrographs ([U]BV, RIZ, YJH) providing a spectral resolution of $\sim$100,000 with a minimum simultaneous wavelength coverage of 0.4-1.8 $\mu$m with the goal of extending it to 0.35-2.4 $\mu$m with the addition of a U arm to the BV spectrograph and a separate K band spectrograph. It operates both in seeing- and diffraction-limited conditions and the fibre feeding allows several, interchangeable observing modes including a single conjugated adaptive optics module and a small diffraction-limited integral field unit in the NIR. Modularity and fibre-feeding allow ANDES to be placed partly on the ELT Nasmyth platform and partly in the Coud\'e room. ANDES has a wide range of groundbreaking science cases spanning nearly all areas of research in astrophysics and even fundamental physics. Among the top science cases, there are the detection of biosignatures from exoplanet atmospheres, finding the fingerprints of the first generation of stars, tests on the stability of Nature's fundamental couplings, and the direct detection of the cosmic acceleration. The ANDES project is carried forward by a large international consortium, composed of 35 Institutes from 13 countries, forming a team of almost 300 scientists and engineers which include the majority of the scientific and technical expertise in the field that can be found in ESO member states., Comment: SPIE astronomical telescope and instrumentation 2024, in press

Learning how to safely handle Plasma-Material Interaction (PMI) is a key challenge towards the commercialisation of energy from nuclear fusion. In this respect, linear plasma devices are ideal experimental testbeds, and numerical codes play a crucial complementary role. In this paper, a numerical investigation of PMI-relevant helium plasma experimental discharges in GyM linear device is presented, in which SOLPS-ITER and ERO2.0 codes are coupled for plasma background generation and material erosion investigation respectively, with the aim to support the interpretation and complement the available experimental dataset. On the plasma side, simulated profiles are validated against experimental data to provide a realistic plasma background, and the role of He metastable states is assessed for the first time in SOLPS simulations. On the material side, the erosion and deposition effects due to the introduction of the sample-holder in the simulation volume are investigated, now considering also the real stainless steel composition as wall material.

Gyrotrons are essential for electron cyclotron resonance heating (ECRH) in fusion reactors, making efficient operation crucial for advancing fusion energy. Past experiments revealed instability issues due to trapped electrons in the magnetron injection gun (MIG) region, causing undesired currents and operational failures. To address this, tight manufacturing tolerances are required for the MIG geometry [1]. We present initial findings of the TRapped Electrons eXperiment (T-REX) developed at the Swiss Plasma Center, designed to understand the physics of electron clouds in gyrotron MIGs. T-REX replicates MIG geometries, as well as their typical electric and magnetic fields, and it is supported by 2D Particle-in-Cell (PIC) simulations with the FENNECS code [2, 3]. The setup includes two coaxial electrodes in a vacuum chamber atop a superconducting magnet, with a central electrode biased to negative DC voltages and an outer one at ground, creating a radial electric field (1 to 2 MV/M) and an axial magnetic field (B < 0.4 T). This setup mimics Penning-Malmberg traps. We present the experimental device and first findings on current distribution and also qualitative comparison with FENNECS simulations [4]. Planned diagnostics include optical emission spectroscopy, phosphor screen imaging, Streak camera imaging, and potentially electric field distribution via the Stark effect. This research aims to enhance gyrotron performance and reliability in fusion energy systems.

In this work we introduce the dynamic $\Lambda$-random graph and the associated multiplicative $\Lambda$-coalescent. The dynamic $\Lambda$-random graph is an exchangeable and consistent random graph process, inspired by the classical $\Lambda$-coalescent studied in mathematical population genetics. Our main concern is the multiplicative $\Lambda$-coalescent, which accounts for the connected components (blocks) of the graph. This process is exchangeable but not consistent. We focus on the case where $\Lambda$ is the beta measure and prove a dynamic law of large numbers for the block size spectrum, which tracks the numbers of blocks containing $1,...,d$ elements. On top of that, we provide a functional limit theorem for the fluctuations around its deterministic trajectory. The limit process satisfies a stochastic differential equation of Ornstein-Uhlenbeck type., Comment: 20 pages, 2 figures

Experts in various fields routinely perform methodical writing tasks to plan, organize, and report their work. From a clinician writing a differential diagnosis for a patient, to a teacher writing a lesson plan for students, these tasks are pervasive, requiring to methodically generate structured long-form output for a given input. We develop a typology of methodical tasks structured in the form of a task objective, procedure, input, and output, and introduce DoLoMiTes, a novel benchmark with specifications for 519 such tasks elicited from hundreds of experts from across 25 fields. Our benchmark further contains specific instantiations of methodical tasks with concrete input and output examples (1,857 in total) which we obtain by collecting expert revisions of up to 10 model-generated examples of each task. We use these examples to evaluate contemporary language models highlighting that automating methodical tasks is a challenging long-form generation problem, as it requires performing complex inferences, while drawing upon the given context as well as domain knowledge., Comment: Accepted to TACL; to be presented at EMNLP 2024. Dataset available at https://dolomites-benchmark.github.io

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.

Recent work has shown that attention-based language models excel at recall, the ability to ground generations in tokens previously seen in context. However, the efficiency of attention-based models is bottle-necked during inference by the KV-cache's aggressive memory consumption. In this work, we explore whether we can improve language model efficiency (e.g. by reducing memory consumption) without compromising on recall. By applying experiments and theory to a broad set of architectures, we identify a key tradeoff between a model's state size and recall ability. We show that efficient alternatives to attention (e.g. H3, Mamba, RWKV) maintain a fixed-size recurrent state, but struggle at recall. We propose BASED a simple architecture combining linear and sliding window attention. By varying BASED window size and linear attention feature dimension, we can dial the state size and traverse the pareto frontier of the recall-memory tradeoff curve, recovering the full quality of attention on one end and the small state size of attention-alternatives on the other. We train language models up to 1.3b parameters and show that BASED matches the strongest sub-quadratic models (e.g. Mamba) in perplexity and outperforms them on real-world recall-intensive tasks by 6.22 accuracy points. Implementations of linear attention are often less efficient than optimized standard attention implementations. To make BASED competitive, we develop IO-aware algorithms that enable 24x higher throughput on language generation than FlashAttention-2, when generating 1024 tokens using 1.3b parameter models. Code for this work is provided at: https://github.com/HazyResearch/based.

Cultural gems is a web application conceived by the European Commission's Joint Research Centre (DG JRC), which aims at engaging people and organisations across Europe to create a unique repository of cultural and creative places. The main goal is to provide a vision of European culture in order to strengthen a sense of identity within a single European cultural realm. Cultural gems maps more than 130,000 physical places in over 300 European cities and towns, and since 2020 it also lists online cultural initiatives. The new release aims, among other, to increase the interoperability of the application. At this purpose, we provide an overview on the current development of an ontology for Cultural gems used to map cultural heritage in European cities by using Linked Open Data (LOD) standards, and making the data FAIR, that is Findable, Accessible, Interoperable, and Reusable. We provide an overview of the methodology, presenting the structure of the ontology, and the services and tools we are currently building on top., Comment: 11 pages, 3 figures, Eighth International Congress on Information and Communication Technology (ICICT 2023)

The recovery and resilience of the cultural and creative sectors after the COVID-19 pandemic is a current topic with priority for the European Commission. Cultural gems is a crowdsourced web platform managed by the Joint Research Centre of the European Commission aimed at creating community-led maps as well as a common repository for cultural and creative places across European cities and towns. More than 130,000 physical locations and online cultural activities in more than 300 European cities and towns are currently tracked by the application. The main objective of Cultural gems consists in raising a holistic vision of European culture, reinforcing a sense of belonging to a common European cultural space. This data article describes the ontology developed for Cultural gems, adopted to represent the domain of knowledge of the application by means of FAIR (Findable, Accessible, Interoperable, Reusable) principles and following the paradigms of Linked Open Data (LOD). We provide an overview of this dataset, and describe the ontology model, along with the services used to access and consume the data., Comment: 10 pages, 3 figures

We consider the inverse scattering problem for time-harmonic acoustic waves in a medium with pointwise inhomogeneities. In the Foldy-Lax model, the estimation of the scatterers' locations and intensities from far field measurements can be recast as the recovery of a discrete measure from nonlinear observations. We propose a "linearize and locally optimize" approach to perform this reconstruction. We first solve a convex program in the space of measures (known as the Beurling LASSO), which involves a linearization of the forward operator (the far field pattern in the Born approximation). Then, we locally minimize a second functional involving the nonlinear forward map, using the output of the first step as initialization. We provide guarantees that the output of the first step is close to the sought-after measure when the scatterers have small intensities and are sufficiently separated. We also provide numerical evidence that the second step still allows for accurate recovery in settings that are more involved.

In inverse problems, it is widely recognized that the incorporation of a sparsity prior yields a regularization effect on the solution. This approach is grounded on the a priori assumption that the unknown can be appropriately represented in a basis with a limited number of significant components, while most coefficients are close to zero. This occurrence is frequently observed in real-world scenarios, such as with piecewise smooth signals. In this study, we propose a probabilistic sparsity prior formulated as a mixture of degenerate Gaussians, capable of modeling sparsity with respect to a generic basis. Under this premise, we design a neural network that can be interpreted as the Bayes estimator for linear inverse problems. Additionally, we put forth both a supervised and an unsupervised training strategy to estimate the parameters of this network. To evaluate the effectiveness of our approach, we conduct a numerical comparison with commonly employed sparsity-promoting regularization techniques, namely LASSO, group LASSO, iterative hard thresholding, and sparse coding/dictionary learning. Notably, our reconstructions consistently exhibit lower mean square error values across all $1$D datasets utilized for the comparisons, even in cases where the datasets significantly deviate from a Gaussian mixture model.

External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depends on the degree of access granted to auditors. Recent audits of state-of-the-art AI systems have primarily relied on black-box access, in which auditors can only query the system and observe its outputs. However, white-box access to the system's inner workings (e.g., weights, activations, gradients) allows an auditor to perform stronger attacks, more thoroughly interpret models, and conduct fine-tuning. Meanwhile, outside-the-box access to training and deployment information (e.g., methodology, code, documentation, data, deployment details, findings from internal evaluations) allows auditors to scrutinize the development process and design more targeted evaluations. In this paper, we examine the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks. Given that different forms of access can lead to very different levels of evaluation, we conclude that (1) transparency regarding the access and methods used by auditors is necessary to properly interpret audit results, and (2) white- and outside-the-box access allow for substantially more scrutiny than black-box access alone., Comment: FAccT 2024

L-mode negative triangularity (NT) operation is a promising alternative to the positive triangularity (PT) H-mode as a high-confinement ELM-free operational regime. In this work, two TCV L-mode lower single null Ohmic discharges with opposite triangularity $\delta \simeq \pm 0.3$ are investigated using SOLPS-ITER modelling. The main focus is the exploration of the reasons behind the experimentally observed feature of NT plasmas being more difficult to detach than similar PT experiments. SOLPS-ITER simulations are performed assuming the same anomalous diffusivity for particles $D_n^{AN}$ and energy $\kappa_{e/i}^{AN}$ in PT and NT. Nonetheless, the results clearly show dissimilar transport and accumulation of neutral particles in the scrape-off layer (SOL) of the two configurations, which consequently gives rise to different ionization sources for the plasma and produces different poloidal and cross-field fluxes. Simulations also recover the experimental feature of the outer target being hotter in the NT scenario (with $T_{e, NT} \gtrsim 5 \, \mathrm{eV}$) than in the PT counterpart., Comment: 18 pages, 8 figures

We consider a *continuous* solution $u$ of the balance law \[ \partial_{\mathit t} u + \partial_{\mathit x} (f(u)) = g\] in one space dimension, where the flux function $f$ is of class $C^2$ and the source term $g$ is bounded. This equation admits an Eulerian intepretation (namely the distributional one) and a Lagrangian intepretation (which can be further specified). Since $u$ is only continuous, these interpretations do not necessessarily agree; moreover each interpretation naturally entails a different equivalence class for the source term $g$. In this paper we complete the comparison between these notions of solutions started in the companion paper [Alberti-Bianchini-Caravenna I], and analize in detail the relations between the corresponding notions of source term.

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.

We demonstrate a method for determining the three-dimensional location of single atoms in a quantum gas microscopy system using a phase-only spatial light modulator to modify the point-spread function of the high-resolution imaging system. Here, the typical diffracted spot generated by a single atom as a point source is modified to a double spot that rotates as a function of the atom's distance from the focal plane of the imaging system. We present and numerically validate a simple model linking the rotation angle of the point-spread function with the distance to the focal plane. We show that, when aberrations in the system are carefully calibrated and compensated for, this method can be used to determine an atom's position to within a single lattice site in a single experimental image, extending quantum simulation with microscopy systems further into the regime of three dimensions., Comment: Accepted version

The variability in the magnetic activity of the Sun is the main source of the observed changes in the plasma and electromagnetic environments within the heliosphere. The primary way in which solar activity affects the Earth's environment is via the solar wind and its transients. However, the relationship between solar activity and solar wind is not the same at the Space Weather and Space Climate time scales. In this work, we investigate this relationship exploiting five solar cycles data of Ca II K index and solar wind parameters, by taking advantage of the Hilbert-Huang Transform, which allows to separate the contribution at the different time scales. By filtering out the high frequency components and looking at decennial time scales, we confirm the presence of a delayed response of solar wind to Ca II K index variations, with a time lag of ~ 3.1-year for the speed and ~ 3.4-year for the dynamic pressure. To assess the results in a stronger framework, we make use of a Transfer Entropy approach to investigate the information flow between the quantities and to test the causality of the relation. The time lag results from the latter are consistent with the cross-correlation ones, pointing out the presence of a statistical significant information flow from Ca II K index to solar wind dynamic pressure that peaks at time lag of 3.6-year. Such a result could be of relevance to build up a predictive model in a Space Climate context.

We discuss the closability of directional derivative operators with respect to a general Radon measure $\mu$ on $\mathbb{R}^d$; our main theorem completely characterizes the vectorfields for which the corresponding operator is closable from the space of Lipschitz functions ${\mathrm{Lip}}(\mathbb{R^d})$ to $L^p(\mu)$, for $1\leq p\leq\infty$. We also consider certain classes of multilinear differential operators. We then discuss the closability of the same operators from $L^q(\mu)$ to $L^p(\mu)$; we give necessary conditions and sufficient conditions for closability, but we do not have an exact characterization. As a corollary we obtain that classical differential operators such as gradient, divergence and jacobian determinant are closable from $L^q(\mu)$ to $L^p(\mu)$ only if $\mu$ is absolutely continuous with respect to the Lebesgue measure. Lastly, we rephrase our results on multilinear operators in terms of metric currents.

We prove that every flat chain with finite mass in $\mathbb{R}^d$ with coefficients in a normed abelian group $G$ is the restriction of a normal $G$-current to a Borel set. We deduce a characterization of real flat chains with finite mass in terms of a pointwise relation between the associated measure and vector field. We also deduce that any codimension-one real flat chain with finite mass can be written as an integral of multiplicity-one rectifiable currents, without loss of mass. Given a Lipschitz homomorphism $\phi:\tilde G\to G$ between two groups, we then study the associated map $\pi$ between flat chains in $\mathbb{R}^d$ with coefficients in $\tilde G$ and $G$ respectively. In the case $\tilde G=\mathbb{R}$ and $G=\mathbb{S}^1$, we prove that if $\phi$ is surjective, so is the restriction of $\pi$ to the set of flat chains with finite mass of dimension $0$, $1$, $d-1$, $d$.

While large language models demonstrate remarkable capabilities, they often present challenges in terms of safety, alignment with human values, and stability during training. Here, we focus on two prevalent methods used to align these models, Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF). SFT is simple and robust, powering a host of open-source models, while RLHF is a more sophisticated method used in top-tier models like ChatGPT but also suffers from instability and susceptibility to reward hacking. We propose a novel approach, Supervised Iterative Learning from Human Feedback (SuperHF), which seeks to leverage the strengths of both methods. Our hypothesis is two-fold: that the reward model used in RLHF is critical for efficient data use and model generalization and that the use of Proximal Policy Optimization (PPO) in RLHF may not be necessary and could contribute to instability issues. SuperHF replaces PPO with a simple supervised loss and a Kullback-Leibler (KL) divergence prior. It creates its own training data by repeatedly sampling a batch of model outputs and filtering them through the reward model in an online learning regime. We then break down the reward optimization problem into three components: robustly optimizing the training rewards themselves, preventing reward hacking-exploitation of the reward model that degrades model performance-as measured by a novel METEOR similarity metric, and maintaining good performance on downstream evaluations. Our experimental results show SuperHF exceeds PPO-based RLHF on the training objective, easily and favorably trades off high reward with low reward hacking, improves downstream calibration, and performs the same on our GPT-4 based qualitative evaluation scheme all the while being significantly simpler to implement, highlighting SuperHF's potential as a competitive language model alignment technique., Comment: Accepted to the Socially Responsible Language Modelling Research (SoLaR) workshop at NeurIPS 2023

Statistical physics and dynamical systems theory are key tools to study high-impact geophysical events such as temperature extremes, cyclones, thunderstorms, geomagnetic storms and many more. Despite the intrinsic differences between these events, they all originate as temporary deviations from the typical trajectories of a geophysical system, resulting in well-organised, coherent structures at characteristic spatial and temporal scales. While statistical extreme value analysis techniques are capable to provide return times and probabilities of occurrence of certain geophysical events, they are not apt to account for their underlying physics. Their focus is to compute the probability of occurrence of events that are large or small with respect to some specific observable (e.g. temperature, precipitation, solar wind), rather than to relate rare or extreme phenomena to the underlying anomalous geophysical regimes. This paper outlines this knowledge gap, presenting some related challenges, new formalisms and briefly commenting on how stochastic approaches tailored to the study of extreme geophysical events can help to advance their understanding., Comment: This perspective paper has issued from discussions held during the UNDERPIN symposium held in Erice, Italy in 2022

While one can unequivocally identify past climate transitions, we lack comprehensive knowledge about their underlying mechanisms and timescales. Our study employs a dimensional analysis of benthic stable isotope records to uncover, across different timescales, how the climatic fluctuation of the Cenozoic are associated with changes in the number of effective degrees of freedom. Precession timescales dominate the Hothouse and Warmhouse states, while the Icehouse climate is primarily influenced by obliquity and eccentricity timescales. Notably, the Coolhouse state lacks dominant timescales. Our analysis proves effective in objectively identifying abrupt climate shifts and extremes. This is also demonstrated using high-resolution data from the last glacial cycle, revealing abrupt climate shifts within a single climate state. These findings significantly impact our understanding of the inherent stability of each climate state and the evaluation of (paleo-)climate models' ability to replicate key features of past/future climate states and transitions.

The possibility of modeling and therefore predicting the trend of demographic mortality is of great scientific and social interest. The article presents and discusses the hypothesis that the demographic distribution of mortality in advanced ages converges asymptotically to an S-system distribution as lifespan increases. The statistical distribution of the S-system was introduced by the author in a 2022 paper and was derived by applying the methods of Fermi statistics to a cellular automaton acting as an "arbitrary oscillator". This distribution is here recalled and formalized analytically and its characteristics are described. The conjecture is based on two case studies: mortality in the United States from 1900 to 2017 and mortality in Italy from 1974 to 2019. The conjecture, applied to both case studies, appears reasonable. Tables and comparison figures are provided to support this. Finally, an attempt to predict demographic mortality behavior and limitations for the years to come is provided., Comment: 20 pages, 13 figures, 5 tables, 1 appendix, note added on age offset due to interpolation

Planet-scale image geolocalization remains a challenging problem due to the diversity of images originating from anywhere in the world. Although approaches based on vision transformers have made significant progress in geolocalization accuracy, success in prior literature is constrained to narrow distributions of images of landmarks, and performance has not generalized to unseen places. We present a new geolocalization system that combines semantic geocell creation, multi-task contrastive pretraining, and a novel loss function. Additionally, our work is the first to perform retrieval over location clusters for guess refinements. We train two models for evaluations on street-level data and general-purpose image geolocalization; the first model, PIGEON, is trained on data from the game of Geoguessr and is capable of placing over 40% of its guesses within 25 kilometers of the target location globally. We also develop a bot and deploy PIGEON in a blind experiment against humans, ranking in the top 0.01% of players. We further challenge one of the world's foremost professional Geoguessr players to a series of six matches with millions of viewers, winning all six games. Our second model, PIGEOTTO, differs in that it is trained on a dataset of images from Flickr and Wikipedia, achieving state-of-the-art results on a wide range of image geolocalization benchmarks, outperforming the previous SOTA by up to 7.7 percentage points on the city accuracy level and up to 38.8 percentage points on the country level. Our findings suggest that PIGEOTTO is the first image geolocalization model that effectively generalizes to unseen places and that our approach can pave the way for highly accurate, planet-scale image geolocalization systems. Our code is available on GitHub., Comment: Accepted at CVPR 2024

Natural language processing (NLP) made an impressive jump with the introduction of Transformers. ChatGPT is one of the most famous examples, changing the perception of the possibilities of AI even outside the research community. However, besides the impressive performance, the quadratic time and space complexity of Transformers with respect to sequence length pose significant limitations for handling long sequences. While efficient Transformer architectures like Linformer and Performer with linear complexity have emerged as promising solutions, their theoretical understanding remains limited. In this paper, we introduce Sumformer, a novel and simple architecture capable of universally approximating equivariant sequence-to-sequence functions. We use Sumformer to give the first universal approximation results for Linformer and Performer. Moreover, we derive a new proof for Transformers, showing that just one attention layer is sufficient for universal approximation.

Current understanding of the kinetic-scale turbulence in weakly-collisional plasmas still remains elusive. We employ a general framework in which the turbulent energy transfer is envisioned as a scale-to-scale Langevin process. Fluctuations in the sub-ion range show a global scale invariance, thus suggesting a homogeneous energy repartition. In this Letter, we interpret such a feature by linking the drift term of the Langevin equation to scaling properties of fluctuations. Theoretical expectations are verified on solar wind observations and numerical simulations thus giving relevance to the proposed framework for understanding kinetic-scale turbulence in space plasmas., Comment: Accepted for publication in Physical Review Research

Cross-lingual summarization consists of generating a summary in one language given an input document in a different language, allowing for the dissemination of relevant content across speakers of other languages. The task is challenging mainly due to the paucity of cross-lingual datasets and the compounded difficulty of summarizing and translating. This work presents $\mu$PLAN, an approach to cross-lingual summarization that uses an intermediate planning step as a cross-lingual bridge. We formulate the plan as a sequence of entities capturing the summary's content and the order in which it should be communicated. Importantly, our plans abstract from surface form: using a multilingual knowledge base, we align entities to their canonical designation across languages and generate the summary conditioned on this cross-lingual bridge and the input. Automatic and human evaluation on the XWikis dataset (across four language pairs) demonstrates that our planning objective achieves state-of-the-art performance in terms of informativeness and faithfulness. Moreover, $\mu$PLAN models improve the zero-shot transfer to new cross-lingual language pairs compared to baselines without a planning component., Comment: EACL 2024

Representing a manifold of very high-dimensional data with generative models has been shown to be computationally efficient in practice. However, this requires that the data manifold admits a global parameterization. In order to represent manifolds of arbitrary topology, we propose to learn a mixture model of variational autoencoders. Here, every encoder-decoder pair represents one chart of a manifold. We propose a loss function for maximum likelihood estimation of the model weights and choose an architecture that provides us the analytical expression of the charts and of their inverses. Once the manifold is learned, we use it for solving inverse problems by minimizing a data fidelity term restricted to the learned manifold. To solve the arising minimization problem we propose a Riemannian gradient descent algorithm on the learned manifold. We demonstrate the performance of our method for low-dimensional toy examples as well as for deblurring and electrical impedance tomography on certain image manifolds.

Compressed sensing allows for the recovery of sparse signals from few measurements, whose number is proportional to the sparsity of the unknown signal, up to logarithmic factors. The classical theory typically considers either random linear measurements or subsampled isometries and has found many applications, including accelerated magnetic resonance imaging, which is modeled by the subsampled Fourier transform. In this work, we develop a general theory of infinite-dimensional compressed sensing for abstract inverse problems, possibly ill-posed, involving an arbitrary forward operator. This is achieved by considering a generalized restricted isometry property, and a quasi-diagonalization property of the forward map. As a notable application, for the first time, we obtain rigorous recovery estimates for the sparse Radon transform (i.e., with a finite number of angles $\theta_1,\dots,\theta_m$), which models computed tomography, in both the parallel-beam and the fan-beam settings. In the case when the unknown signal is $s$-sparse with respect to an orthonormal basis of compactly supported wavelets, we prove stable recovery under the condition \[ m\gtrsim s, \] up to logarithmic factors., Comment: 57 pages

Image geolocalization is the challenging task of predicting the geographic coordinates of origin for a given photo. It is an unsolved problem relying on the ability to combine visual clues with general knowledge about the world to make accurate predictions across geographies. We present $\href{https://huggingface.co/geolocal/StreetCLIP}{\text{StreetCLIP}}$, a robust, publicly available foundation model not only achieving state-of-the-art performance on multiple open-domain image geolocalization benchmarks but also doing so in a zero-shot setting, outperforming supervised models trained on more than 4 million images. Our method introduces a meta-learning approach for generalized zero-shot learning by pretraining CLIP from synthetic captions, grounding CLIP in a domain of choice. We show that our method effectively transfers CLIP's generalized zero-shot capabilities to the domain of image geolocalization, improving in-domain generalized zero-shot performance without finetuning StreetCLIP on a fixed set of classes.

Understanding the interplay between recombination and resampling is a significant challenge in mathematical population genetics and of great practical relevance. Asymptotic results about the distribution of samples when recombination is strong compared to resampling are often based on the approximate solution of certain recursions, which is technically hard and offers little conceptual insight. This work generalises an elegant probabilistic argument, based on the coupling of ancestral processes but so far only available in the case of two sites, to the multilocus setting. This offers an alternative route to, and slightly generalises, a classical result of Bhaskar and Song.

Most recent coreference resolution systems use search algorithms over possible spans to identify mentions and resolve coreference. We instead present a coreference resolution system that uses a text-to-text (seq2seq) paradigm to predict mentions and links jointly. We implement the coreference system as a transition system and use multilingual T5 as an underlying language model. We obtain state-of-the-art accuracy on the CoNLL-2012 datasets with 83.3 F1-score for English (a 2.3 higher F1-score than previous work (Dobrovolskii, 2021)) using only CoNLL data for training, 68.5 F1-score for Arabic (+4.1 higher than previous work) and 74.3 F1-score for Chinese (+5.3). In addition we use the SemEval-2010 data sets for experiments in the zero-shot setting, a few-shot setting, and supervised setting using all available training data. We get substantially higher zero-shot F1-scores for 3 out of 4 languages than previous approaches and significantly exceed previous supervised state-of-the-art results for all five tested languages.

The paper presents an approach to semantic grounding of language models (LMs) that conceptualizes the LM as a conditional model generating text given a desired semantic message formalized as a set of entity-relationship triples. It embeds the LM in an auto-encoder by feeding its output to a semantic parser whose output is in the same representation domain as the input message. Compared to a baseline that generates text using greedy search, we demonstrate two techniques that improve the fluency and semantic accuracy of the generated text: The first technique samples multiple candidate text sequences from which the semantic parser chooses. The second trains the language model while keeping the semantic parser frozen to improve the semantic accuracy of the auto-encoder. We carry out experiments on the English WebNLG 3.0 data set, using BLEU to measure the fluency of generated text and standard parsing metrics to measure semantic accuracy. We show that our proposed approaches significantly improve on the greedy search baseline. Human evaluation corroborates the results of the automatic evaluation experiments.

The availability of large, high-quality datasets has been one of the main drivers of recent progress in question answering (QA). Such annotated datasets however are difficult and costly to collect, and rarely exist in languages other than English, rendering QA technology inaccessible to underrepresented languages. An alternative to building large monolingual training datasets is to leverage pre-trained language models (PLMs) under a few-shot learning setting. Our approach, QAmeleon, uses a PLM to automatically generate multilingual data upon which QA models are trained, thus avoiding costly annotation. Prompt tuning the PLM for data synthesis with only five examples per language delivers accuracy superior to translation-based baselines, bridges nearly 60% of the gap between an English-only baseline and a fully supervised upper bound trained on almost 50,000 hand labeled examples, and always leads to substantial improvements compared to fine-tuning a QA model directly on labeled examples in low resource settings. Experiments on the TyDiQA-GoldP and MLQA benchmarks show that few-shot prompt tuning for data synthesis scales across languages and is a viable alternative to large-scale annotation., Comment: To Appear at Transactions of Association for Computational Linguistics (TACL)

We report on novel investigations into training models that make sentences concise. We define the task and show that it is different from related tasks such as summarization and simplification. For evaluation, we release two test sets, consisting of 2000 sentences each, that were annotated by two and five human annotators, respectively. We demonstrate that conciseness is a difficult task for which zero-shot setups with large neural language models often do not perform well. Given the limitations of these approaches, we propose a synthetic data generation method based on round-trip translations. Using this data to either train Transformers from scratch or fine-tune T5 models yields our strongest baselines that can be further improved by fine-tuning on an artificial conciseness dataset that we derived from multi-annotator machine translation test sets., Comment: EMNLP 2022 Workshop on Text Simplification, Accessibility, and Readability (TSAR)

The solar activity displays variability and periodic behaviours over a wide range of timescales, with the presence of a most prominent cycle with a mean length of 11 years. Such variability is transported within the heliosphere by solar wind, radiation and other processes, affecting the properties of the interplanetary medium. The presence of solar activity-related periodicities is well visible in different solar wind and geomagnetic indices, although with time lags with respect to the solar one, leading to hysteresis cycles. Here, we investigate the time lag behaviour between a physical proxy of the solar activity, the Ca II K index, and two solar wind parameters (speed and dynamic pressure), studying how their pairwise relative lags vary over almost five solar cycles. We find that the lag between Ca II K index and solar wind speed is not constant over the whole time interval investigated, with values ranging from 6 years to 1 year (average 3.2 years). A similar behaviour is found also for the solar wind dynamic pressure. Then, by using a Lomb-Scargle periodogram analysis we obtain a 10.21-year mean periodicity for the speed and 10.30-year for the dynamic pressure. We speculate that the different periodicities of the solar wind parameters with respect to the solar 11-year cycle may be related to the overall observed temporal evolution of the time lags. Finally, by accounting for them, we obtain empirical relations that link the amplitude of the Ca II K index to the two solar wind parameters., Comment: 10 pages, 6 figures