Search

Recent work in language modeling has raised the possibility of self-improvement, where a language models evaluates and refines its own generations to achieve higher performance without external feedback. It is impossible for this self-improvement to create information that is not already in the model, so why should we expect that this will lead to improved capabilities? We offer a new perspective on the capabilities of self-improvement through a lens we refer to as sharpening. Motivated by the observation that language models are often better at verifying response quality than they are at generating correct responses, we formalize self-improvement as using the model itself as a verifier during post-training in order to ``sharpen'' the model to one placing large mass on high-quality sequences, thereby amortizing the expensive inference-time computation of generating good sequences. We begin by introducing a new statistical framework for sharpening in which the learner aims to sharpen a pre-trained base policy via sample access, and establish fundamental limits. Then we analyze two natural families of self-improvement algorithms based on SFT and RLHF. We find that (i) the SFT-based approach is minimax optimal whenever the initial model has sufficient coverage, but (ii) the RLHF-based approach can improve over SFT-based self-improvement by leveraging online exploration, bypassing the need for coverage. Finally, we empirically validate the sharpening mechanism via inference-time and amortization experiments. We view these findings as a starting point toward a foundational understanding that can guide the design and evaluation of self-improvement algorithms.

We introduce a novel protocol, which enables Heisenberg-limited quantum-enhanced sensing using the dynamics of any interacting many-body Hamiltonian. Our approach - dubbed butterfly metrology - utilizes a single application of forward and reverse time evolution to produce a coherent superposition of a "scrambled" and "unscrambled" quantum state. In this way, we create metrologically-useful long-range entanglement from generic local quantum interactions. The sensitivity of butterfly metrology is given by a sum of local out-of-time-order correlators (OTOCs) - the prototypical diagnostic of quantum information scrambling. Our approach broadens the landscape of platforms capable of performing quantum-enhanced metrology; as an example, we provide detailed blueprints and numerical studies demonstrating a route to scalable quantum-enhanced sensing in ensembles of solid-state spin defects.

Survival analysis is a classic problem in statistics with important applications in healthcare. Most machine learning models for survival analysis are black-box models, limiting their use in healthcare settings where interpretability is paramount. More recently, glass-box machine learning models have been introduced for survival analysis, with both strong predictive performance and interpretability. Still, several gaps remain, as no prior glass-box survival model can produce calibrated shape functions with enough flexibility to capture the complex patterns often found in real data. To fill this gap, we introduce a new glass-box machine learning model for survival analysis called DNAMite. DNAMite uses feature discretization and kernel smoothing in its embedding module, making it possible to learn shape functions with a flexible balance of smoothness and jaggedness. Further, DNAMite produces calibrated shape functions that can be directly interpreted as contributions to the cumulative incidence function. Our experiments show that DNAMite generates shape functions closer to true shape functions on synthetic data, while making predictions with comparable predictive performance and better calibration than previous glass-box and black-box models.

The power of foundation models (FMs) lies in their capacity to learn highly expressive representations that can be adapted to a broad spectrum of tasks. However, these pretrained models require multiple stages of fine-tuning to become effective for downstream applications. Conventionally, the model is first retrained on the aggregate of a diverse set of tasks of interest and then adapted to specific low-resource downstream tasks by utilizing a parameter-efficient fine-tuning (PEFT) scheme. While this two-phase procedure seems reasonable, the independence of the retraining and fine-tuning phases causes a major issue, as there is no guarantee the retrained model will achieve good performance post-fine-tuning. To explicitly address this issue, we introduce a meta-learning framework infused with PEFT in this intermediate retraining stage to learn a model that can be easily adapted to unseen tasks. For our theoretical results, we focus on linear models using low-rank adaptations. In this setting, we demonstrate the suboptimality of standard retraining for finding an adaptable set of parameters. Further, we prove that our method recovers the optimally adaptable parameters. We then apply these theoretical insights to retraining the RoBERTa model to predict the continuation of conversations between different personas within the ConvAI2 dataset. Empirically, we observe significant performance benefits using our proposed meta-learning scheme during retraining relative to the conventional approach., Comment: Preprint

Image-based diagnostic decision support systems (DDSS) utilizing deep learning have the potential to optimize clinical workflows. However, developing DDSS requires extensive datasets with expert annotations and is therefore costly. Leveraging report contents from radiological data bases with Natural Language Processing to annotate the corresponding image data promises to replace labor-intensive manual annotation. As mining "real world" databases can introduce label noise, noise-robust training losses are of great interest. However, current noise-robust losses do not consider noise estimations that can for example be derived based on the performance of the automatic label generator used. In this study, we expand the noise-robust Deep Abstaining Classifier (DAC) loss to an Informed Deep Abstaining Classifier (IDAC) loss by incorporating noise level estimations during training. Our findings demonstrate that IDAC enhances the noise robustness compared to DAC and several state-of-the-art loss functions. The results are obtained on various simulated noise levels using a public chest X-ray data set. These findings are reproduced on an in-house noisy data set, where labels were extracted from the clinical systems of the University Hospital Bonn by a text-based transformer. The IDAC can therefore be a valuable tool for researchers, companies or clinics aiming to develop accurate and reliable DDSS from routine clinical data., Comment: This preprint has no post-submission improvements or corrections. The Version of Record of this contribution is published in the Neural Information Processing, ICONIP 2024 Proceedings

When analyzing the individual positional dynamics of an ensemble of moving objects, the extracted parameters that characterize the motion of individual objects, such as the mean-squared instantaneous velocity or the diffusivity, exhibit a spread that is due to the convolution of three different effects: i) Motion stochasticity, caused by the fluctuating environment and enhanced by limited observation time, ii) measurement errors that depend on details of the detection technique, and iii) the intrinsic parameter variance that characterizes differences between individual objects, the quantity of ultimate interest. We develop the theoretical framework to separate these effects using the generalized Langevin equation (GLE), which constitutes the most general description of active and passive dynamics, as it derives from the general underlying many-body Hamiltonian for the studied system without approximations. We apply our methodology to determine intrinsic cell-to-cell differences of living human breast-cancer cells, algae cells and, as a benchmark, size differences of passively moving polystyrene beads in water. We find algae and human breast-cancer cells to exhibit significant individual differences, reflected by the spreading of the intrinsic mean-squared instantaneous velocity over two orders of magnitude, which is remarkable in light of the genetic homogeneity of the investigated breast-cancer cells and highlights their phenotypical diversity. Quantification of the intrinsic variance of single-cell properties is relevant for infection biology, ecology and medicine and opens up new possibilities to estimate population heterogeneity on the single-organism level in a non-destructive manner. Our framework is not limited to motility properties but can be readily applied to general experimental time-series data., Comment: 10 pages main text including 6 figures , 3pages methods, 15 pages supplementary information including 8 more figures

The precision measurement of the tritium $\beta$-decay spectrum performed by the KATRIN experiment provides a unique way to search for general neutrino interactions (GNI). All theoretical allowed GNI terms involving neutrinos are incorporated into a low-energy effective field theory, and can be identified by specific signatures in the measured tritium $\beta$-spectrum. In this paper an effective description of the impact of GNI on the $\beta$-spectrum is formulated and the first constraints on the effective GNI parameters are derived based on the 4 million electrons collected in the second measurement campaign of KATRIN in 2019. In addition, constraints on selected types of interactions are investigated, thereby exploring the potential of KATRIN to search for more specific new physics cases, including a right-handed W boson, a charged Higgs or leptoquarks.

Single-ion mass identification is important for atomic and nuclear physics experiments on ions produced with low yields. Cooling the ion to ultra-low temperatures by interacting with a laser-cooled ion will enhance the precision of the measurements. In this paper we present axial-common-mode frequency measurements of balanced and unbalanced Coulomb crystals from the Fourier transform of the fluorescence photons from a Doppler-cooling transition in calcium ions, after probing the ion/crystal with a 5-radiofrequency comb. A single ion non-destructively detected can be used for identification yielding a mass resolving power $m/\Delta m_\mathrm{FWHM}\approx 310$ from the axial common mode. This identification can be performed from a single measurement within times below one second., Comment: 6 pages, 4 figures

Entanglement in a many-particle system can enable measurement sensitivities beyond that achievable by only classical correlations. For an ensemble of spins, all-to-all interactions are known to reshape the quantum projection noise, leading to a form of entanglement known as spin squeezing. Here, we demonstrate spin squeezing with strictly short-range contact interactions. In particular, working with ultracold lithium atoms in optical lattices, we utilize superexchange interactions to realize a nearest-neighbor anisotropic Heisenberg model. We investigate the resulting quench dynamics from an initial product state in both one and three dimensions. In 1D, we observe $1.9^{+0.7}_{-0.5}$ dB of spin squeezing in quantitative agreement with theory. However, in 3D, we observe a maximum of $2.0^{+0.7}_{-0.7}$ dB of squeezing, over an order of magnitude smaller than that expected. We demonstrate that this discrepancy arises from the presence of a finite density of holes; both the motion of the holes as well as direct coupling between spin and density qualitatively alter the spin dynamics. Our observations point to the importance of understanding the complex interplay between motional and spin degrees of freedom in quantum simulators.

Communicating one's sensemaking during a complex analysis session to explain thought processes is hard, yet most intelligence occurs in collaborative settings. Team members require a deeper understanding of the work being completed by their peers and subordinates, but little research has fully articulated best practices for analytic provenance consumers. This work proposes an automatic summarization technique that separates an analysis session and summarizes interaction provenance as textual blurbs to allow for meta-analysis of work done. Focusing on the domain of intelligence analysis, we demonstrate our segmentation technique using five datasets, including both publicly available and classified interaction logs. We shared our demonstration with a notoriously inaccessible population of expert reviewers with experience as United States Department of Defense analysts. Our findings indicate that the proposed pipeline effectively generates cards that display key events from interaction logs, facilitating the sharing of analysis progress. Yet, we also hear that there is a need for more prominent justifications and pattern elicitation controls to communicate analysis summaries more effectively. The expert review highlights the potential of automated approaches in addressing the challenges of provenance information in complex domains. We'd like to emphasize the need for further research into provenance communication in other domains. A free copy of this paper and all supplemental materials are available at https://osf.io/j4bxt

We consider a sparse matrix-matrix multiplication (SpGEMM) setting where one matrix is square and the other is tall and skinny. This special variant, called TS-SpGEMM, has important applications in multi-source breadth-first search, influence maximization, sparse graph embedding, and algebraic multigrid solvers. Unfortunately, popular distributed algorithms like sparse SUMMA deliver suboptimal performance for TS-SpGEMM. To address this limitation, we develop a novel distributed-memory algorithm tailored for TS-SpGEMM. Our approach employs customized 1D partitioning for all matrices involved and leverages sparsity-aware tiling for efficient data transfers. In addition, it minimizes communication overhead by incorporating both local and remote computations. On average, our TS-SpGEMM algorithm attains 5x performance gains over 2D and 3D SUMMA. Furthermore, we use our algorithm to implement multi-source breadth-first search and sparse graph embedding algorithms and demonstrate their scalability up to 512 Nodes (or 65,536 cores) on NERSC Perlmutter.

The projected sensitivity of the effective electron neutrino-mass measurement with the KATRIN experiment is below 0.3 eV (90 % CL) after five years of data acquisition. The sensitivity is affected by the increased rate of the background electrons from KATRIN's main spectrometer. A special shifted-analysing-plane (SAP) configuration was developed to reduce this background by a factor of two. The complex layout of electromagnetic fields in the SAP configuration requires a robust method of estimating these fields. We present in this paper a dedicated calibration measurement of the fields using conversion electrons of gaseous $^\mathrm{83m}$Kr, which enables the neutrino-mass measurements in the SAP configuration., Comment: 19 pages, 11 figures

Neutral atoms become strongly interacting when their electrons are excited to loosely bound Rydberg states. We investigate the strongly correlated quantum phases of matter that emerge in two-dimensional atom arrays where three Rydberg levels are used to encode an effective spin-1 degree of freedom. Dipolar exchange between such spin-1 Rydberg atoms naturally yields two distinct models: (i) a two-species hardcore boson model, and (ii) upon tuning near a F\"orster resonance, a dipolar spin-1 XY model. Through extensive, large-scale infinite density matrix renormalization group calculations, we provide a broad roadmap predicting the quantum phases that emerge from these models on a variety of lattice geometries: square, triangular, kagome, and ruby. We identify a wealth of correlated states, including lattice supersolids and simplex phases, all of which can be naturally realized in near-term experiments., Comment: 5 pages, 4 figures

Imitation learning (IL) aims to mimic the behavior of an expert in a sequential decision making task by learning from demonstrations, and has been widely applied to robotics, autonomous driving, and autoregressive text generation. The simplest approach to IL, behavior cloning (BC), is thought to incur sample complexity with unfavorable quadratic dependence on the problem horizon, motivating a variety of different online algorithms that attain improved linear horizon dependence under stronger assumptions on the data and the learner's access to the expert. We revisit the apparent gap between offline and online IL from a learning-theoretic perspective, with a focus on the realizable/well-specified setting with general policy classes up to and including deep neural networks. Through a new analysis of behavior cloning with the logarithmic loss, we show that it is possible to achieve horizon-independent sample complexity in offline IL whenever (i) the range of the cumulative payoffs is controlled, and (ii) an appropriate notion of supervised learning complexity for the policy class is controlled. Specializing our results to deterministic, stationary policies, we show that the gap between offline and online IL is smaller than previously thought: (i) it is possible to achieve linear dependence on horizon in offline IL under dense rewards (matching what was previously only known to be achievable in online IL); and (ii) without further assumptions on the policy class, online IL cannot improve over offline IL with the logarithmic loss, even in benign MDPs. We complement our theoretical results with experiments on standard RL tasks and autoregressive language generation to validate the practical relevance of our findings., Comment: NeurIPS 2024

Relativistic effects strongly influence the electronic structures of the heaviest elements, thereby shaping their chemical and physical properties. Studying ion mobility within a noble gas environment reveals how the ion-neutral interactions depend on the ion's electronic configurations, thus providing an avenue for exploring these effects. An ion mobility spectrometer with a cryogenic drift tube was developed to precisely measure the low-field reduced mobility of heavy lanthanide and actinide cations. The apparatus was characterized by optimizing the bunching operation of ions with a miniature RF coulomb buncher and evaluating the chromatography performance of the drift tube operated with helium buffer gas at a temperature of 298K. Systematic ion mobility measurements of lutetium cations (Lu$^{+}$) drifting in helium gas were carried out as a case study. The electronic state chromatography of Lu$^{+}$ has been demonstrated. The low-field reduced ion mobility for the ground and lowest meta-stable state of Lu$^{+}$ have been examined. In addition, the variation of both states' reduced mobility and the quenching of meta-stable population has been investigated under different reduced electric fields ($E/n_0$), the ratio of an electric field to neutral gas number density., Comment: 9 pages, 10 figures

Quantum computing is one of the most enticing computational paradigms with the potential to revolutionize diverse areas of future-generation computational systems. While quantum computing hardware has advanced rapidly, from tiny laboratory experiments to quantum chips that can outperform even the largest supercomputers on specialized computational tasks, these noisy-intermediate scale quantum (NISQ) processors are still too small and non-robust to be directly useful for any real-world applications. In this paper, we describe NASA's work in assessing and advancing the potential of quantum computing. We discuss advances in algorithms, both near- and longer-term, and the results of our explorations on current hardware as well as with simulations, including illustrating the benefits of algorithm-hardware co-design in the NISQ era. This work also includes physics-inspired classical algorithms that can be used at application scale today. We discuss innovative tools supporting the assessment and advancement of quantum computing and describe improved methods for simulating quantum systems of various types on high-performance computing systems that incorporate realistic error models. We provide an overview of recent methods for benchmarking, evaluating, and characterizing quantum hardware for error mitigation, as well as insights into fundamental quantum physics that can be harnessed for computational purposes., Comment: 27 pages, 0 figures

The fact that neutrinos carry a non-vanishing rest mass is evidence of physics beyond the Standard Model of elementary particles. Their absolute mass bears important relevance from particle physics to cosmology. In this work, we report on the search for the effective electron antineutrino mass with the KATRIN experiment. KATRIN performs precision spectroscopy of the tritium $\beta$-decay close to the kinematic endpoint. Based on the first five neutrino-mass measurement campaigns, we derive a best-fit value of $m_\nu^{2} = {-0.14^{+0.13}_{-0.15}}~\mathrm{eV^2}$, resulting in an upper limit of $m_\nu < {0.45}~\mathrm{eV}$ at 90 % confidence level. With six times the statistics of previous data sets, amounting to 36 million electrons collected in 259 measurement days, a substantial reduction of the background level and improved systematic uncertainties, this result tightens KATRIN's previous bound by a factor of almost two., Comment: 61 pages, 20 figures, 2 tables

We present an extensive quantum Monte Carlo study of a nearest-neighbor, singlet-projection model on the pyrochlore lattice that exhibits SO($N$) symmetry and is sign-problem-free. We find that in contrast to the previously studied two-dimensional variations of this model that harbor critical points between their ground state phases, the non-bipartite pyrochlore lattice in three spatial dimensions appears to exhibit a first-order transition between a magnetically-ordered phase and some, as yet uncharacterized, paramagnetic phase. We also observe that the magnetically-ordered phase survives to a relatively large value of $N=8$, and that it is gone for $N=9$., Comment: 6 pages, 5 figures

Although $S_\omega$ (the group of all permutations of $\mathbb{N}$) is size continuum, both it and its closed subgroups can be presented as the set of paths through a countable tree. The subgroups of $S_\omega$ that can be presented this way with finite branching trees are exactly the profinite ones. We use these tree presentations to find upper bounds on the complexity of the existential theories of profinite subgroups of $S_\omega$, as well as to prove sharpness for these bounds. These complexity results enable us to distinguish a simple subclass of profinite groups, those with \emph{orbit independence}, for which we find an upper bound on the complexity of the entire first order theory., Comment: Accepted to appear in the 2024 Computability in Europe proceedings volume in the Lecture Notes in Computer Science (LNCS) series from Springer-Verlag

We report first results obtained during the commissioning of the Laser Resonance Chromatography (LRC) apparatus, which is conceived to enable atomic structure investigations in the region of the heaviest elements beyond nobelium. In our studies we first established optimum conditions for the operation of the different components of the setup, including the radio-frequency quadrupole ion buncher and the cryogenic drift tube, which was operated with helium buffer gas at relatively low electric fields. We used laser ablated hafnium, lutetium, and ytterbium cations to assess the chromatography performance of the drift tube at a gas temperature of $295$K. Arrival time distributions of singly charged lutetium revealed two distinct ion mobilities of this ion in the ground and metastable state in helium with a relative difference of about $19$%. By using $^{219}$Rn ions from a $^{223}$Ra recoil source the overall efficiency of the apparatus is found to be $(0.6\pm0.1)$%. The findings help to establish LRC on lutetium, which is the lighter chemical homolog of lawrencium., Comment: 12 pages, 9 figures

Research funding systems are not isolated systems - they are embedded in a larger scientific system with an enormous influence on the system. This paper aims to analyze the allocation of competitive research funding from different perspectives: How reliable are decision processes for funding? What are the economic costs of competitive funding? How does competition for funds affect doing risky research? How do competitive funding environments affect scientists themselves, and which ethical issues must be considered? We attempt to identify gaps in our knowledge of research funding systems; we propose recommendations for policymakers and funding agencies, including empirical experiments of decision processes and the collection of data on these processes. With our recommendations we hope to contribute to developing improved ways of organizing research funding., Comment: 29 pages

Creating a standardized introduction course becomes more critical as the field of human-robot interaction (HRI) becomes more established. This paper outlines the key components necessary to provide an undergraduate with a sufficient foundational understanding of the interdisciplinary nature of this field and provides proposed course content. It emphasizes the importance of creating a course with theoretical and experimental components to accommodate all different learning preferences. This manuscript also advocates creating or adopting a universal platform to standardize the hands-on component of introductory HRI courses, regardless of university funding or size. Next, it recommends formal training in how to read scientific articles and staying up-to-date with the latest relevant papers. Finally, it provides detailed lecture content and project milestones for a 15-week semester. By creating a standardized course, researchers can ensure consistency and quality are maintained across institutions, which will help students as well as industrial and academic employers understand what foundational knowledge is expected., Comment: Presented at the Designing an Intro to HRI Course Workshop at HRI 2024 (arXiv:2403.05588) Report-no: HRI101/2024/4

This paper develops a systematic framework for the evidence neuroscientists use to establish whether a neural response represents a feature. Researchers try to establish that the neural response is (1) sensitive and (2) specific to the feature, (3) invariant to other features, and (4) functional, which means that it is used downstream in the brain. We formalize these desiderata in information-theoretic terms. This formalism allows us to precisely state the desiderata while unifying the different analysis methods used in neuroscience under one framework. We discuss how common methods such as correlational analyses, decoding and encoding models, representational similarity analysis, and tests of statistical dependence are used to evaluate the desiderata. In doing so, we provide a common terminology to researchers that helps to clarify disagreements, to compare and integrate results across studies and research groups, and to identify when evidence might be missing and when evidence for some representational conclusion is strong. We illustrate the framework with several canonical examples, including the representation of orientation, numerosity, faces, and spatial location. We end by discussing how the framework can be extended to cover models of the neural code, multi-stage models, and other domains., Comment: 50 pages, 11 figures

In this paper, an extension of a linear control design for hyperbolic linear partial differential equations is presented for a first-order traffic flow model. Starting from the Lighthill-Whitham-Richards (LWR) model, variable speed limit control (VSL) is applied through a modification of Greenshield's equilibrium flow model. Then, an optimal linear quadratic (LQ) controller is designed on the linear LWR model. The LQ state feedback function is found via the solution of a Riccati differential equation. Unlike previous studies, the control input is the rate of change of the input, not the input itself. The proposed controller is then verified on both the linear and nonlinear models. In both cases, the controller is able to drive the system to a desired density profile. In the nonlinear application, a higher control gain is needed to achieve similar results as in the linear case., Comment: Accepted to 2024 American Control Conference

In order to circumvent statistical and computational hardness results in sequential decision-making, recent work has considered smoothed online learning, where the distribution of data at each time is assumed to have bounded likeliehood ratio with respect to a base measure when conditioned on the history. While previous works have demonstrated the benefits of smoothness, they have either assumed that the base measure is known to the learner or have presented computationally inefficient algorithms applying only in special cases. This work investigates the more general setting where the base measure is \emph{unknown} to the learner, focusing in particular on the performance of Empirical Risk Minimization (ERM) with square loss when the data are well-specified and smooth. We show that in this setting, ERM is able to achieve sublinear error whenever a class is learnable with iid data; in particular, ERM achieves error scaling as $\tilde O( \sqrt{\mathrm{comp}(\mathcal F)\cdot T} )$, where $\mathrm{comp}(\mathcal F)$ is the statistical complexity of learning $\mathcal F$ with iid data. In so doing, we prove a novel norm comparison bound for smoothed data that comprises the first sharp norm comparison for dependent data applying to arbitrary, nonlinear function classes. We complement these results with a lower bound indicating that our analysis of ERM is essentially tight, establishing a separation in the performance of ERM between smoothed and iid data.

Due to statistical lower bounds on the learnability of many function classes under privacy constraints, there has been recent interest in leveraging public data to improve the performance of private learning algorithms. In this model, algorithms must always guarantee differential privacy with respect to the private samples while also ensuring learning guarantees when the private data distribution is sufficiently close to that of the public data. Previous work has demonstrated that when sufficient public, unlabelled data is available, private learning can be made statistically tractable, but the resulting algorithms have all been computationally inefficient. In this work, we present the first computationally efficient, algorithms to provably leverage public data to learn privately whenever a function class is learnable non-privately, where our notion of computational efficiency is with respect to the number of calls to an optimization oracle for the function class. In addition to this general result, we provide specialized algorithms with improved sample complexities in the special cases when the function class is convex or when the task is binary classification.