Search

Many data sets cannot be accurately described by standard probability distributions due to the excess number of zero values present. For example, zero-inflation is prevalent in microbiome data and single-cell RNA sequencing data, which serve as our real data examples. Several models have been proposed to address zero-inflated datasets including the zero-inflated negative binomial, hurdle negative binomial model, and the truncated latent Gaussian copula model. This study aims to compare various models and determine which one performs optimally under different conditions using both simulation studies and real data analyses. We are particularly interested in investigating how dependence among the variables, level of zero-inflation or deflation, and variance of the data affects model selection.

Inflammation of the umbilical cord can be seen as a result of ascending intrauterine infection or other inflammatory stimuli. Acute fetal inflammatory response (FIR) is characterized by infiltration of the umbilical cord by fetal neutrophils, and can be associated with neonatal sepsis or fetal inflammatory response syndrome. Recent advances in deep learning in digital pathology have demonstrated favorable performance across a wide range of clinical tasks, such as diagnosis and prognosis. In this study we classified FIR from whole slide images (WSI). We digitized 4100 histological slides of umbilical cord stained with hematoxylin and eosin(H&E) and extracted placental diagnoses from the electronic health record. We build models using attention-based whole slide learning models. We compared strategies between features extracted by a model (ConvNeXtXLarge) pretrained on non-medical images (ImageNet), and one pretrained using histopathology images (UNI). We trained multiple iterations of each model and combined them into an ensemble. The predictions from the ensemble of models trained using UNI achieved an overall balanced accuracy of 0.836 on the test dataset. In comparison, the ensembled predictions using ConvNeXtXLarge had a lower balanced accuracy of 0.7209. Heatmaps generated from top accuracy model appropriately highlighted arteritis in cases of FIR 2. In FIR 1, the highest performing model assigned high attention to areas of activated-appearing stroma in Wharton's Jelly. However, other high-performing models assigned attention to umbilical vessels. We developed models for diagnosis of FIR from placental histology images, helping reduce interobserver variability among pathologists. Future work may examine the utility of these models for identifying infants at risk of systemic inflammatory response or early onset neonatal sepsis.

A detailed description of the asymptotic behaviour in the Belinski-Khalatnikov-Lifshitz (BKL) scenario is presented through a simple geometric picture. The Lagrangian version of the dynamics governed by the BKL equations is described in terms of trajectories inside a conical subset of the corresponding space of the generalised velocities. The calculations confirm that the initial conditions of decreasing volume inevitably result in total collapse, while oscillations along paths reflecting from a hyperboloid, similar to Kasner's solutions, occur on the way. The exact solution, found in our previous work, proves to be the only one that shrinks to a point along a differentiable path. Therefore, its instability means that the collapse is always chaotic. The collapse of the universe along asymptotics of exact Kasner's solutions is proved to be impossible for solutions of the BKL equations., Comment: 18 pages, 3 figures

Neural networks has recently attracted much interest as useful representations of quantum many body ground states, which might help address the infamous sign problem. Most attention was directed at their representability properties, while possible limitations on finding the desired optimal state have not been suitably explored. By leveraging well-established results applicable in the context of infinite width, specifically regarding the renowned neural tangent kernel and conjugate kernel, a comprehensive analysis of the convergence and initialization characteristics of the method is conducted. We reveal the dependence of these characteristics on the interplay among these kernels, the Hamiltonian, and the basis used for its representation. We introduce and motivate novel performance metrics and explore the condition for their optimization. By leveraging these findings, we elucidate a substantial dependence of the effectiveness of this approach on the selected basis, demonstrating that so-called stoquastic Hamiltonians are more amenable to solution through neural networks than those suffering from a sign problem.

Many human interactions, such as political debates, are carried out in group settings, where there are arbitrarily many participants, each with different views and agendas. To explore such complex social settings, we present SAUCE: a customizable Python platform, allowing researchers to plug-and-play various LLMs participating in discussions on any topic chosen by the user. Our platform takes care of instantiating the models, scheduling their responses, managing the discussion history, and producing a comprehensive output log, all customizable through configuration files, requiring little to no coding skills. A novel feature of SAUCE is our asynchronous communication feature, where models decide when to speak in addition to what to say, thus modeling an important facet of human communication. We show SAUCE's attractiveness in two initial experiments, and invite the community to use it in simulating various group simulations., Comment: https://github.com/Deep-Cognition-Lab/SAUCE

The placenta forms a critical barrier to infection through pregnancy, labor and, delivery. Inflammatory processes in the placenta have short-term, and long-term consequences for offspring health. Digital pathology and machine learning can play an important role in understanding placental inflammation, and there have been very few investigations into methods for predicting and understanding Maternal Inflammatory Response (MIR). This work intends to investigate the potential of using machine learning to understand MIR based on whole slide images (WSI), and establish early benchmarks. To that end, we use Multiple Instance Learning framework with 3 feature extractors: ImageNet-based EfficientNet-v2s, and 2 histopathology foundation models, UNI and Phikon to investigate predictability of MIR stage from histopathology WSIs. We also interpret predictions from these models using the learned attention maps from these models. We also use the MIL framework for predicting white blood cells count (WBC) and maximum fever temperature ($T_{max}$). Attention-based MIL models are able to classify MIR with a balanced accuracy of up to 88.5% with a Cohen's Kappa ($\kappa$) of up to 0.772. Furthermore, we found that the pathology foundation models (UNI and Phikon) are both able to achieve higher performance with balanced accuracy and $\kappa$, compared to ImageNet-based feature extractor (EfficientNet-v2s). For WBC and $T_{max}$ prediction, we found mild correlation between actual values and those predicted from histopathology WSIs. We used MIL framework for predicting MIR stage from WSIs, and compared effectiveness of foundation models as feature extractors, with that of an ImageNet-based model. We further investigated model failure cases and found them to be either edge cases prone to interobserver variability, examples of pathologist's overreach, or mislabeled due to processing errors.

Contrastive learning methods, such as CLIP, leverage naturally paired data-for example, images and their corresponding text captions-to learn general representations that transfer efficiently to downstream tasks. While such approaches are generally applied to two modalities, domains such as robotics, healthcare, and video need to support many types of data at once. We show that the pairwise application of CLIP fails to capture joint information between modalities, thereby limiting the quality of the learned representations. To address this issue, we present Symile, a simple contrastive learning approach that captures higher-order information between any number of modalities. Symile provides a flexible, architecture-agnostic objective for learning modality-specific representations. To develop Symile's objective, we derive a lower bound on total correlation, and show that Symile representations for any set of modalities form a sufficient statistic for predicting the remaining modalities. Symile outperforms pairwise CLIP, even with modalities missing in the data, on cross-modal classification and retrieval across several experiments including on an original multilingual dataset of 33M image, text and audio samples and a clinical dataset of chest X-rays, electrocardiograms, and laboratory measurements. All datasets and code used in this work are publicly available at https://github.com/rajesh-lab/symile., Comment: NeurIPS 2024

Artificial intelligence may assist healthcare systems in meeting increasing demand for pathology services while maintaining diagnostic quality and reducing turnaround time and costs. We aimed to investigate the performance of an institutionally developed system for prostate cancer detection, grading, and workflow optimization and to contrast this with commercial alternatives. From August 2021 to March 2023, we scanned 21,396 slides from 1,147 patients with positive biopsies. We developed models for cancer detection, grading, and screening of equivocal cases for IHC ordering. We compared a task-specific model trained using the PANDA dataset of prostate cancer biopsies with one built using features extracted by the general-purpose histology foundation model, UNI and compare their performance in an unfiltered prospectively collected dataset that reflects our patient population (1737 slides,95 patients). We evaluated the contributions of a bespoke model designed to improve sensitivity in detecting small cancer foci and scoring of broader patterns observed at lower resolution. We found high concordance between the developed systems and pathologist reference in detection (AUC 98.5, sensitivity 95.0, and specificity 97.8), ISUP grading (quadratic Cohen's kappa 0.869), grade group 3 or higher (AUC 97.5, sensitivity 94.9, specificity 96.6) and comparable to published data from commercial systems. Screening could reduce IHC ordering for equivocal cases by 44.5% with an overall error rate of 1.8% (1.4% false positive, 0.4% false negative rates). Institutions like academic medical centers that have high scanning volumes and report abstraction capabilities can develop accurate computational pathology models for internal use. These models have the potential to aid in quality control role and to improve workflow in the pathology lab to help meet future challenges in prostate cancer diagnosis.

A likelihood analysis of the observables in deeply virtual exclusive photoproduction off a proton target, $ep \rightarrow e' p' \gamma'$, is presented. Two processes contribute to the reaction: deeply virtual Compton scattering, where the photon is produced at the proton vertex, and the Bether-Heitler process, where the photon is radiated from the electron. We consider the unpolarized process for which the largest amount of data with all the kinematic dependences are available from corresponding datasets with unpolarized beams and unpolarized targets from Jefferson Lab. We provide and use a method which derives a joint likelihood of the Compton form factors, which parametrize the deeply virtual Compton scattering amplitude in QCD, for each observed combination of the kinematic variables defining the reaction. The unpolarized twist-two cross section likelihood fully constrains only three of the Compton form factors (CFFs). The impact of the twist-three corrections to the analysis is also explored. The derived likelihoods are explored using Markov chain Monte Carlo (MCMC) methods. Using our proposed method we derive CFF error bars and covariances. Additionally, we explore methods which may reduce the magnitude of error bars/contours in the future., Comment: 22 pages, 11 figures

Instantons, semi-classical trajectories of quantum tunneling in imaginary time, have long been used to study thermodynamic and transport properties in a myriad of condensed matter and high energy systems. A recent experiment in superconducting circuits [Phys. Rev. Lett. 126, 197701, (2021)] provided first evidence for direct dynamical signatures of instantons (phase slips), manifested by order-unity inelastic decay probabilities for photons with which they interact, motivating the development of a scattering theory of instantons [Phys. Rev. Lett. 126, 137701, (2021)]. While this framework successfully predicted the measured inelastic decay rates of the photons for several experimental devices, it is valid only if the tunneling time of the instantons is much shorter than the relaxation time of the environment in which they are embedded, and requires a closed analytical expression for the instanton trajectory. Here, we amend these issues by incorporating numerical methods that lift some of the previously applied approximations. Our results agree with the experimental measurements, also for devices with shorter relaxation times, without fitting parameters. This framework should be useful in many other quantum field theoretical contexts., Comment: 15 pages, 4 figures. The first two authors contributed equally

Despite being a pillar of quantum mechanics, little attention has been paid to the onset of Fermi's golden rule as a discrete microscopic bath of modes approaches the macroscopic thermodynamic limit and forms a continuum. Motivated by recent experiments in circuit quantum electrodynamics, we tackle this question through the lens of single-photon decay in a finite transmission line coupled to a qubit ("quantum impurity"). We consider a single-photon state, coupled via the nonlinear impurity to several baths formed by multi-photon states with different number of photons, which are inherently discrete due to the finite size of the line. We focus on the late-time dynamics of the single-photon, and uncover the conditions under which the photon's decoherence rate approaches the decay rate predicted by Fermi's golden rule. We show that it is necessary to keep a small but finite escape rate (unrelated to the impurity) for each single-photon mode to obtain a finite long-time decay rate. We analyze the contribution of the baths formed by many-body states with different number of photons, and illustrate how the decay rate induced by some bath of $n$ photon states is enhanced by the presence of other baths of $m \neq n$ photon states, highlighting the contribution of cascade photon decay processes. Our formalism could be used to analyze recent experiments in superconducting circuits., Comment: 14 pages, 5 figures

Understanding the inner workings of Transformers is crucial for achieving more accurate and efficient predictions. In this work, we analyze the computation performed by Transformers in the layers after the top-1 prediction has become fixed, which has been previously referred to as the "saturation event". We expand the concept of saturation events for top-k tokens, demonstrating that similar saturation events occur across language, vision, and speech models. We find that these saturation events happen in order of the corresponding tokens' ranking, i.e., the model first decides on the top ranking token, then the second highest ranking token, and so on. This phenomenon seems intrinsic to the Transformer architecture, occurring across different architectural variants (decoder-only, encoder-only, and to a lesser extent full-Transformer), and even in untrained Transformers. We propose an underlying mechanism of task transition for this sequential saturation, where task k corresponds to predicting the k-th most probable token, and the saturation events are in fact discrete transitions between the tasks. In support of this we show that it is possible to predict the current task from hidden layer embedding. Furthermore, using an intervention method we demonstrate that we can cause the model to switch from one task to the next. Finally, leveraging our findings, we introduce a novel token-level early-exit strategy, which surpasses existing methods in balancing performance and efficiency.

Background/Objectives: Cataract surgery, a very common and critical procedure for restoring vision, has outcomes that can vary based on patient demographics. This study aimed to elucidate the effects of age and sex on the risk factors, intraoperative complications, and postoperative outcomes of cataract surgery. Subjects/Methods: Conducted as a single-center retrospective cohort study, it analyzed 691 eyes from 589 individuals who underwent surgery at a tertiary referral center, utilizing data from electronic medical records to assess preoperative risk factors, intraoperative complications, and pre- and post-operative best corrected visual acuity (BCVA) along with demographic data. Results: The main results highlighted that males aged 65-75 years exhibited significantly higher rates of functional postoperative BCVA (91% for males vs. 79% for females, p=0.007), a disparity that is not explained by differences in surgical complications or risk factor prevalence. Furthermore, the study identified age-specific thresholds where BCVA improvements significantly declined beyond 65 years for females and 75 years for males. The likelihood of worsened BCVA post-surgery increased with age for both sexes, with a significant decline in BCVA improvement transitioning from 55-65 years to 65-75 years age groups. Conclusions: The findings underscore the critical influence of both sex and age on cataract surgery outcomes, revealing significant sex-specific age thresholds that signal lesser improvements in postoperative BCVA. These insights advocate for the integration of patient age and sex into preoperative evaluations to better tailor the timing and planning of cataract surgery, ultimately aiming to optimize clinical outcomes.

It is generally assumed that existing artificial systems are not phenomenally conscious, and that the construction of phenomenally conscious artificial systems would require significant technological progress if it is possible at all. We challenge this assumption by arguing that if Global Workspace Theory (GWT) - a leading scientific theory of phenomenal consciousness - is correct, then instances of one widely implemented AI architecture, the artificial language agent, might easily be made phenomenally conscious if they are not already. Along the way, we articulate an explicit methodology for thinking about how to apply scientific theories of consciousness to artificial systems and employ this methodology to arrive at a set of necessary and sufficient conditions for phenomenal consciousness according to GWT.

Many real-world applications of tabular data involve using historic events to predict properties of new ones, for example whether a credit card transaction is fraudulent or what rating a customer will assign a product on a retail platform. Existing approaches to event prediction include costly, brittle, and application-dependent techniques such as time-aware positional embeddings, learned row and field encodings, and oversampling methods for addressing class imbalance. Moreover, these approaches often assume specific use-cases, for example that we know the labels of all historic events or that we only predict a pre-specified label and not the data's features themselves. In this work, we propose a simple but flexible baseline using standard autoregressive LLM-style transformers with elementary positional embeddings and a causal language modeling objective. Our baseline outperforms existing approaches across popular datasets and can be employed for various use-cases. We demonstrate that the same model can predict labels, impute missing values, or model event sequences., Comment: 10 pages, 6 pages of references+appendix

We give an overview of the science objectives and mission design of the Spectroscopic Time-Resolving Observatory for Broadband Energy X-rays (STROBE-X) observatory, which has been proposed as a NASA probe-class (~$1.5B) mission in response to the Astro2020 recommendation for an X-ray probe., Comment: 11 pages, 5 figures, accepted for publication in JATIS

In this work we study the Kondo impurity problem - at a finite concentration of impurities. We identify two parameter regimes for the Kondo impurity problem. 1) The single impurity limit, where the concentration of Kondo impurities is so low that the background scattering mechanisms (non-magnetic impurities, Umklapp scattering, etc.) of the metal considered are the dominant conduction electron scattering mechanisms at zero temperature. 2) The dilute impurity system limit where the concentration of magnetic impurities is such that they form the dominant mechanism of conduction electron scattering at zero temperature of the metal in question (this is accompanied by a variety of easily detectable Kondo signatures (resistance minimum, specific heat measurements, magnetization as a function of external magnetic field, conduction electron dephasing rates as well as ARPES, RIXS and NMR spectroscopies)) while still being very dilute. Most theoretical efforts are currently in regime where a single isolated impurity is considered - regime 1) while most experimental efforts are in regime 2). We present analytical evidence that this explains the well known discrepancy between experiment and theory as to the value of the Kondo temperature. We find that the ratio between the two Kondo temperatures in regime 1) and regime 2) is given by: $\mathcal{R}=\exp\left[\frac{\pi^{2}\rho v_{F}}{2k_{F}^{2}Vol}\right]$ where $\rho$ is the density of states, $v_{F}$ is the fermi velocity, and $k_{F}$ is the Fermi wavevector and $Vol$ is the volume of a unit cell. We note that there is no dependence on the impurity concentration in this ratio so it is possible to define a single Kondo temperature for limit 2) for the dilute Kondo impurity system. In this work we present results within the Reed-Newns Kondo meanfield approximation and to leading order of the linked cluster expansion., Comment: Comments welcome

The prompt spectra of gamma-ray bursts are known to follow broadband continuum behavior over decades in energy. GRB 221009A, given the moniker the brightest of all time (BOAT), is the brightest gamma-ray burst identified in half a century of observations, and was first identified by the Fermi Gamma-ray Burst Monitor (GBM). On behalf of the Fermi-GBM Team, Lesage et al. (2023) described the initial GBM analysis. Ravasio et al. (2024) report the identification of a spectral line in part of the prompt emission of this burst, which they describe as evolving over 80 s from $\sim$12 MeV to 6 MeV. We report a GBM Team analysis on the Ravasio Line: 1) We cannot identify an instrumental effect that could have produced this signal, and 2) our method of calculating the statistical significance of the line shows it easily exceeds the 5$\sigma$ discovery threshold. We additionally comment on the claim of the line beginning at earlier time intervals, up to 37 MeV, as reported in Zhang et al. (2024). We find that it is reasonable to utilize these measurements for characterization of the line evolution, with caution. We encourage theoretical studies exploring this newly discovered gamma-ray burst spectral feature, unless any rigorous alternative explanation unrelated to the emission from GRB 221009A is identified.

We present a quantum alternating operator ansatz (QAOA$^+$) that solves a class of linearly constrained optimization problems by evolving a quantum state within a Hilbert subspace of feasible problem solutions. Our main focus is on a class of problems with a linear constraint containing sequential integer coefficients. For problems in this class, we devise QAOA$^+$ circuits that provably converge to the optimal solution as the number of circuit layers increases, generalizing previous guarantees for solving unconstrained problems or problems with symmetric constraints. Our approach includes asymmetric ``mixing" Hamiltonians that drive transitions between feasible states, as well as a method to incorporate an arbitrary known feasible solution as the initial state, each of which can be applied beyond the specific linear constraints considered here. This analysis extends QAOA$^+$ performance guarantees to a more general set of linearly-constrained problems and provides tools for future generalizations., Comment: 15+9 pages, 4+6 figures

While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of complexity are still blank. Aiming to address this limitation, we present Easy2Hard-Bench, a consistently formatted collection of 6 benchmark datasets spanning various domains, such as mathematics and programming problems, chess puzzles, and reasoning questions. Each problem within these datasets is annotated with numerical difficulty scores. To systematically estimate problem difficulties, we collect abundant performance data on attempts to each problem by humans in the real world or LLMs on the prominent leaderboard. Leveraging the rich performance data, we apply well-established difficulty ranking systems, such as Item Response Theory (IRT) and Glicko-2 models, to uniformly assign numerical difficulty scores to problems. Moreover, datasets in Easy2Hard-Bench distinguish themselves from previous collections by a higher proportion of challenging problems. Through extensive experiments with six state-of-the-art LLMs, we provide a comprehensive analysis of their performance and generalization capabilities across varying levels of difficulty, with the aim of inspiring future research in LLM generalization. The datasets are available at https://huggingface.co/datasets/furonghuang-lab/Easy2Hard-Bench., Comment: NeurIPS 2024 Datasets and Benchmarks Track

The resonant conversion of cosmic microwave background (CMB) photons into axions within large-scale structure induces an anisotropic spectral distortion in CMB temperature maps. Applying state-of-the-art foreground cleaning techniques to $\textit{Planck}$ CMB observations, we construct maps of axion-induced "patchy screening" of the CMB. We cross-correlate these maps with data from the $\textit{unWISE}$ galaxy survey and find no evidence of axions. We constrain the axion-photon coupling, $g_{a\gamma\gamma} \lesssim 2 \times 10^{-12}~{\rm GeV}^{-1}$, at the 95% confidence level for axion masses in the range $10^{-13}~{\rm eV} \lesssim m_a \lesssim 10^{-12}~{\rm eV}$. These constraints are competitive with the tightest astrophysical axion limits in this mass range and are inferred from robust population-level statistics, which makes them complementary to existing searches that rely on modeling of individual systems., Comment: 5+15 pages; 3+15 figures

We present the detection and analysis of GRB 231115A, a candidate extragalactic magnetar giant flare (MGF) observed by Fermi/GBM and localized by INTEGRAL to the starburst galaxy M82. This burst exhibits distinctive temporal and spectral characteristics that align with known MGFs, including a short duration and a high peak energy. Gamma-ray analyses reveal significant insights into this burst, supporting conclusions already established in the literature: our time-resolved spectral studies provide further evidence that GRB 231115A is indeed a MGF. Significance calculations also suggest a robust association with M82, further supported by a high Bayes factor that minimizes the probability of chance alignment with a neutron star merger. Despite extensive follow-up efforts, no contemporaneous gravitational wave or radio emissions were detected. The lack of radio emission sets stringent upper limits on possible radio luminosity. Constraints from our analysis show no fast radio bursts (FRBs) associated with two MGFs. X-ray observations conducted post-burst by Swift/XRT and XMM/Newton provided additional data, though no persistent counterparts were identified. Our study underscores the importance of coordinated multi-wavelength follow-up and highlights the potential of MGFs to enhance our understanding of short GRBs and magnetar activities in the cosmos. Current MGF identification and follow-up implementation are insufficient for detecting expected counterparts; however, improvements in these areas may allow for the recovery of follow-up signals with existing instruments. Future advancements in observational technologies and methodologies will be crucial in furthering these studies.

Using Parker Solar Probe data from orbits 8 through 17, we examine fluctuation amplitudes throughout the critical region where the solar wind flow speed approaches and then exceeds the Alfv\'en wave speed, taking account of various exigencies of the plasma data. In contrast to WKB theory for non-interacting Alfv\'en waves streaming away from the Sun, the magnetic and kinetic fluctuation energies per unit volume are not monotonically decreasing. Instead, there is clear violation of conservation of standard WKB wave action, which is consistent with previous indications of strong in-situ fluctuation energy input in the solar wind near the Alfv\'en critical region. This points to strong violations of WKB theory due to nonlinearity (turbulence) and major energy input near the critical region, which we interpret as likely due to driving by large-scale coronal shear flows.

The IceCube neutrino observatory detects the diffuse astrophysical neutrino background with high significance, but the contribution of different classes of sources is not established. Because of their non-thermal spectrum, gamma-ray bursts (GRBs) are prime particle acceleration sites and one of the candidate classes for significant neutrino production. Exhaustive searches, based on stacking analysis of GRBs however could not establish the link between neutrinos and GRBs. Gamma-ray burst GRB 221009A had the highest time integrated gamma-ray flux of any detected GRB so far. The total fluence exceeds the sum of all Fermi Gamma-ray Burst Monitor (GBM) detected GRBs by a factor of two. Because it happened relatively nearby, it is one of the most favorable events for neutrino production from GRBs yet no neutrinos were detected. We calculate neutrino fluxes for this GRB in the TeV-PeV range using the most accurate, time-resolved spectral data covering the brightest intervals. We place limits on the physical parameters (Lorentz factor, baryon loading or emission radius) of the burst that are better by a factor of 2 compared to previous limits. The neutrino non-detection indicates a bulk Lorentz factor greater than 500 and possibly even 1000, consistent with other observations., Comment: 10 pages, 3 figures, 1 table. Submitted to AAS journals

The study of transmission and progression of human papillomavirus (HPV) is crucial for understanding the incidence of cervical cancers, and has been identified as a priority worldwide. The complexity of the disease necessitates a detailed model of HPV transmission and its progression to cancer; to infer properties of the above we require a careful process that can match to imperfect or incomplete observational data. In this paper, we describe the HPVsim simulator to satisfy the former requirement; to satisfy the latter we couple this stochastic simulator to a process of emulation and history matching using the R package hmer. With these tools, we are able to obtain a comprehensive collection of parameter combinations that could give rise to observed cancer data, and explore the implications of the variability of these parameter sets as it relates to future health interventions., Comment: 21 pages, 15 figures; submitted to Epidemics

Suppose we are given two OCaml modules implementing the same signature. How do we check that they are observationally equivalent -- that is, that they behave the same on all inputs? One established technique is to use a property-based testing (PBT) tool such as QuickCheck. Currently, however, this can require significant amounts of boilerplate code and ad-hoc test harnesses. To address this issue, we present Mica, an automated tool for testing observational equivalence of OCaml modules. Mica is implemented as a PPX compiler extension, allowing users to supply minimal annotations to a module signature. These annotations guide Mica to automatically derive specialized PBT code that checks observational equivalence. We discuss the design of Mica and demonstrate its efficacy as a testing tool on various modules taken from real-world OCaml libraries., Comment: OCaml Workshop 2024

To achieve confident non-zero polarization measurements for gamma-ray bursts (GRBs) we need sensitive polarimeters and bright GRBs. Here we report on the polarimetric analysis of the bright GRB 180720B using the \Fermi Gamma-ray Burst Monitor (GBM). We rely on the detection of photons that scattered off Earth's atmosphere and into GBM from this burst. Polarized gamma-rays will exhibit a characteristic pattern when scattering off the atmosphere that differs from an unpolarized beam. We compare the measured photon counts in the GBM detectors with extensive simulations of polarized beams to derive the most probable polarization degree (PD) and angle (PA). For the entire GRB, we find PD$=72^{+24}_{-30}\% ~(1\sigma)$ and PA$=91^{+11}_{-9}$ deg ($1\sigma$, equatorial frame). Interestingly, the PA value is broadly consistent with an early optical PA measurement by the Kanata telescope, starting shortly after the end of the prompt emission. The consistency of PAs lends support for this method. The relatively high polarization degree (albeit with large uncertainties) agrees with similar past measurements suggesting that some GRBs might be highly polarized. This will be confirmed or refuted by the upcoming dedicated GRB polarimeters., Comment: 8 pages, 5 figures, submitted to AAS journals

Deeply virtual exclusive scattering processes (DVES) serve as precise probes of nucleon quark and gluon distributions in coordinate space. These distributions are derived from generalized parton distributions (GPDs) via Fourier transform relative to proton momentum transfer. QCD factorization theorems enable DVES to be parameterized by Compton form factors (CFFs), which are convolutions of GPDs with perturbatively calculable kernels. Accurate extraction of CFFs from DVCS, benefiting from interference with the Bethe-Heitler (BH) process and a simpler final state structure, is essential for inferring GPDs. This paper focuses on extracting CFFs from DVCS data using a variational autoencoder inverse mapper (VAIM) and its constrained variant (C-VAIM). VAIM is shown to be consistent with Markov Chain Monte Carlo (MCMC) methods in extracting multiple CFF solutions for given kinematics, while C-VAIM effectively captures correlations among CFFs across different kinematic values, providing more constrained solutions. This study represents a crucial first step towards a comprehensive analysis pipeline towards the extraction of GPDs., Comment: 12 pages, 9 figures

In overview of the recent activity of the newly funded EXCLusives with AI and Machine learning (EXCLAIM) collaboration is presented. The main goal of the collaboration is to develop a framework to implement AI and machine learning techniques in problems emerging from the phenomenology of high energy exclusive scattering processes from nucleons and nuclei, maximizing the information that can be extracted from various sets of experimental data, while implementing theoretical constraints from lattice QCD. A specific perspective embraced by EXCLAIM is to use the methods of theoretical physics to understand the working of ML, beyond its standardized applications to physics analyses which most often rely on industrially provided tools, in an automated way., Comment: 9 pages, 3 figures

Large Language Models (LLMs) have demonstrated impressive capabilities in generating diverse and contextually rich text. However, concerns regarding copyright infringement arise as LLMs may inadvertently produce copyrighted material. In this paper, we first investigate the effectiveness of watermarking LLMs as a deterrent against the generation of copyrighted texts. Through theoretical analysis and empirical evaluation, we demonstrate that incorporating watermarks into LLMs significantly reduces the likelihood of generating copyrighted content, thereby addressing a critical concern in the deployment of LLMs. Additionally, we explore the impact of watermarking on Membership Inference Attacks (MIAs), which aim to discern whether a sample was part of the pretraining dataset and may be used to detect copyright violations. Surprisingly, we find that watermarking adversely affects the success rate of MIAs, complicating the task of detecting copyrighted text in the pretraining dataset. Finally, we propose an adaptive technique to improve the success rate of a recent MIA under watermarking. Our findings underscore the importance of developing adaptive methods to study critical problems in LLMs with potential legal implications., Comment: 21 pages, 6 figures

In this work we propose a new efficient basis for the electronic structure problem. The basis is based on the Muffin Tin Orbital (MTO) idea that the eigenstates of the Khon Sham (KS) Hamiltonian may we be expanded in terms of eigenstates of the spherically averaged KS Hamiltonian inside the so called Muffin Tin (MT) spheres and Bessel functions in the interstitial multiplied by appropriate spherical Harmonics. Here we use the fact that the solution to problem of finding the ground state electron density is most often done through an iterative process, where generically on the order of over 20 iterations are taken till the ground state electron density and energy converges to the lowest values allowed by the correlation and exchange functional for the fixed form of the external potential. We use eigenstate information from the previous convergence iteration to choose the energies of the eigenstates of the spherically averaged KS Hamiltonian. Furthermore within the Atomic Sphere Approximation (ASA) the energies of the Bessel functions do not matter as they are cancelled out. This is an efficient method aimed at studying the electronic structure of materials with large unit cells especially if they are of close packed form where ASA is particularly accurate., Comment: Comments welcome

A Kondo model with symplectic symmetry was recently put forward as the effective low-energy theory of a superconducting-island device coupled to multiple leads. This model, which possesses non-Fermi liquid physics and effective anyons, was argued to belong to the class of topological Kondo effects. Here, we clarify the extent of stability of its exotic fixed point using perturbative and numerical renormalization group in conjunction with bosonization and conformal field theory. In contrast to previous claims, we show that asymmetry in the coupling to the leads destabilizes the non-Fermi liquid. Other destabilizing perturbations include asymmetry in the superconducting pairing or internal energy of the individual quantum dots in the island. Nevertheless, these perturbations all generate the same relevant operators. Thus, only a small number of couplings need to be tuned individually, and these can be selected according to experimental convenience. Our results highlight a common misconception that anisotropy in the Kondo coupling is always irrelevant. As demonstrated, relevant terms will emerge whenever the group generators do not span the full space of impurity operators. This calls for a more detailed inspection of models that exhibit this property, such as large-spin impurities and SO(M) Kondo models, Comment: 26 pages, 7 figures

We introduce GoldFinch, a hybrid Linear Attention/Transformer sequence model that uses a new technique to efficiently generate a highly compressed and reusable KV-Cache in linear time and space with respect to sequence length. GoldFinch stacks our new GOLD transformer on top of an enhanced version of the Finch (RWKV-6) architecture. We train up to 1.5B parameter class models of the Finch, Llama, and GoldFinch architectures, and find dramatically improved modeling performance relative to both Finch and Llama. Our cache size savings increase linearly with model layer count, ranging from 756-2550 times smaller than the traditional transformer cache for common sizes, enabling inference of extremely large context lengths even on limited hardware. Although autoregressive generation has O(n) time complexity per token because of attention, pre-fill computation of the entire initial cache state for a submitted context costs only O(1) time per token due to the use of a recurrent neural network (RNN) to generate this cache. We release our trained weights and training code under the Apache 2.0 license for community use.

A junction of two 2/3 fractional quantum Hall (FQH) edges, with no charge tunneling between them, may exhibit Anderson localization of neutral modes. Manifestations of such localization in transport properties of the junction are explored. There are two competing localization channels, ``neutral-mode superconductivity'' and ``neutral-mode backscattering''. Localization in any of these channels leads to an effective theory of the junction that is characteristic for FQH effect of bosons, with a minimal integer excitation charge equal to two, and with elementary quasiparticle charge equal to 2/3. These values can be measured by studying shot noise in tunneling experiments. Under the assumption of ballistic transport in the arms connecting the junction to contacts, the two-terminal conductance of the junction is found to be 4/3 for the former localization channel and 1/3 for the latter. The four-terminal conductance matrix reveals in this regime a strong quantized drag between the edges induced by neutral-mode localization. The two localization channels lead to opposite signs of the drag conductance, equal to $\pm 1/4$, which can also be interpreted as a special type of Andreev scattering. Coherent random tunneling in arms of the device (which are segments of 2/3 edges) leads to strong mesoscopic fluctuations of the conductance matrix. In the case of fully equilibrated arms, transport via the junction is insensitive to neutral-mode localization: The two-terminal conductance is quantized to 2/3 and the drag is absent., Comment: 12 pages, 3 figures

Massive particles produced during inflation impact soft limits of primordial correlators. Searching for these signatures presents an exciting opportunity to uncover the particle spectrum in the inflationary epoch. We present non-perturbative methods to constrain intermediate-mass scalars ($0\leq m/H<3/2$, where $H$ is the inflationary Hubble scale) produced during inflation, which give rise to a power-law scaling in the squeezed primordial bispectrum. Exploiting the large-scale structure consistency relations and the separate universe approach, we derive models for the late-time squeezed matter bispectrum and collapsed matter trispectrum sourced by these fields. To validate our models, we run $N$-body simulations with the "Cosmological Collider" squeezed bispectrum for two different particle masses. Our models yield unbiased constraints on the amplitude of non-Gaussianity, $f_{\rm NL}^{\Delta}$, from the squeezed bispectrum and collapsed trispectrum deep into the non-linear regime ($k_{\rm max}\approx 2~h/{\rm Mpc}$ at $z=0$). We assess the information content of these summary statistics, emphasizing the importance of sample variance cancellation in the matter sector. We also study the scale-dependent halo bias in our simulations. For mass-selected halos, the non-Gaussian bias estimated from our simulations agrees with predictions based on (i) separate universe simulations and (ii) universal mass functions. With further work, these results can be used to search for inflationary massive particle production with upcoming galaxy surveys., Comment: 25 pages, 11 figures; comments welcome

Reversing a diffusion process by learning its score forms the heart of diffusion-based generative modeling and for estimating properties of scientific systems. The diffusion processes that are tractable center on linear processes with a Gaussian stationary distribution. This limits the kinds of models that can be built to those that target a Gaussian prior or more generally limits the kinds of problems that can be generically solved to those that have conditionally linear score functions. In this work, we introduce a family of tractable denoising score matching objectives, called local-DSM, built using local increments of the diffusion process. We show how local-DSM melded with Taylor expansions enables automated training and score estimation with nonlinear diffusion processes. To demonstrate these ideas, we use automated-DSM to train generative models using non-Gaussian priors on challenging low dimensional distributions and the CIFAR10 image dataset. Additionally, we use the automated-DSM to learn the scores for nonlinear processes studied in statistical physics.

For the one-dimensional Facilitated Exclusion Process with initial state a product measure of density $\rho=1/2-\delta$, $\delta\ge0$, there exists an infinite-time limiting state $\nu_\rho$ in which all particles are isolated and hence cannot move. We study the variance $V(L)$, under $\nu_\rho$, of the number of particles in an interval of $L$ sites. Under $\nu_{1/2}$ either all odd or all even sites are occupied, so that $V(L)=0$ for $L$ even and $V(L)=1/4$ for $L$ odd: the state is hyperuniform, since $V(L)$ grows more slowly than $L$. We prove that for densities approaching 1/2 from below there exist three regimes in $L$, in which the variance grows at different rates: for $L\gg\delta^{-2}$, $V(L)\simeq\rho(1-\rho)L$, just as in the initial state; for $A(\delta)\ll L\ll\delta^{-2}$, with $A(\delta)=\delta^{-2/3}$ for $L$ odd and $A(\delta)=1$ for $L$ even, $V(L)\simeq CL^{3/2}$ with $C=2\sqrt{2/\pi}/3$; and for $L\ll\delta^{-2/3}$ with $L$ odd, $V(L)\simeq1/4$. The analysis is based on a careful study of a renewal process with a long tail. Our study is motivated by simulation results showing similar behavior in higher dimensions; we discuss this background briefly., Comment: 21 pages, no figures

Within biological fluid dynamics, it is conventional to distinguish between "puller" and "pusher" microswimmers on the basis of the forward or aft location of the flagella relative to the cell body: typically, bacteria are pushers and algae are pullers. Here we note that since many pullers have "outboard" cilia or flagella displaced laterally from the cell centerline on both sides of the organism, there are two important subclasses whose far-field is that of a stresslet, but whose near field is qualitatively more complex. The ciliary beat creates not only a propulsive force but also swirling flows that can be represented by paired rotlets with two possible senses of rotation, either "feeders" that sweep fluid toward the cell apex, or "expellers" that push fluid away. Experimental studies of the rotifer $Brachionus~plicatilis$ in combination with earlier work on the green algae $Chlamydomonas~reinhardtii$ show that the two classes have markedly different interactions with surfaces. When swimming near a surface, expellers such as $C.~reinhardtii$ scatter from the wall, whereas a feeder like $B.~plicatilis$ stably attaches. This results in a stochastic "run-and-stick" locomotion, with periods of ballistic motion parallel to the surface interrupted by trapping at the surface., Comment: 16 pages, 6 figures, supplementary videos available at website of REG

Test set contamination, wherein test data from a benchmark ends up in a newer model's training set, is a well-documented obstacle for fair LLM evaluation and can quickly render benchmarks obsolete. To mitigate this, many recent benchmarks crowdsource new prompts and evaluations from human or LLM judges; however, these can introduce significant biases, and break down when scoring hard questions. In this work, we introduce a new benchmark for LLMs designed to be immune to both test set contamination and the pitfalls of LLM judging and human crowdsourcing. We release LiveBench, the first benchmark that (1) contains frequently-updated questions from recent information sources, (2) scores answers automatically according to objective ground-truth values, and (3) contains a wide variety of challenging tasks, spanning math, coding, reasoning, language, instruction following, and data analysis. To achieve this, LiveBench contains questions that are based on recently-released math competitions, arXiv papers, news articles, and datasets, and it contains harder, contamination-free versions of tasks from previous benchmarks such as Big-Bench Hard, AMPS, and IFEval. We evaluate many prominent closed-source models, as well as dozens of open-source models ranging from 0.5B to 110B in size. LiveBench is difficult, with top models achieving below 65% accuracy. We release all questions, code, and model answers. Questions will be added and updated on a monthly basis, and we will release new tasks and harder versions of tasks over time so that LiveBench can distinguish between the capabilities of LLMs as they improve in the future. We welcome community engagement and collaboration for expanding the benchmark tasks and models.

This paper examines the question of whether Large Language Models (LLMs) like ChatGPT possess minds, focusing specifically on whether they have a genuine folk psychology encompassing beliefs, desires, and intentions. We approach this question by investigating two key aspects: internal representations and dispositions to act. First, we survey various philosophical theories of representation, including informational, causal, structural, and teleosemantic accounts, arguing that LLMs satisfy key conditions proposed by each. We draw on recent interpretability research in machine learning to support these claims. Second, we explore whether LLMs exhibit robust dispositions to perform actions, a necessary component of folk psychology. We consider two prominent philosophical traditions, interpretationism and representationalism, to assess LLM action dispositions. While we find evidence suggesting LLMs may satisfy some criteria for having a mind, particularly in game-theoretic environments, we conclude that the data remains inconclusive. Additionally, we reply to several skeptical challenges to LLM folk psychology, including issues of sensory grounding, the "stochastic parrots" argument, and concerns about memorization. Our paper has three main upshots. First, LLMs do have robust internal representations. Second, there is an open question to answer about whether LLMs have robust action dispositions. Third, existing skeptical challenges to LLM representation do not survive philosophical scrutiny.

Large language models (LLMs) have shown an impressive ability to perform tasks believed to require thought processes. When the model does not document an explicit thought process, it becomes difficult to understand the processes occurring within its hidden layers and to determine if these processes can be referred to as reasoning. We introduce a novel and interpretable analysis of internal multi-hop reasoning processes in LLMs. We demonstrate that the prediction process for compositional reasoning questions can be modeled using a simple linear transformation between two semantic category spaces. We show that during inference, the middle layers of the network generate highly interpretable embeddings that represent a set of potential intermediate answers for the multi-hop question. We use statistical analyses to show that a corresponding subset of tokens is activated in the model's output, implying the existence of parallel reasoning paths. These observations hold true even when the model lacks the necessary knowledge to solve the task. Our findings can help uncover the strategies that LLMs use to solve reasoning tasks, offering insights into the types of thought processes that can emerge from artificial intelligence. Finally, we also discuss the implication of cognitive modeling of these results.

This study empirically tests the $\textit{Narrative Economics}$ hypothesis, which posits that narratives (ideas that are spread virally and affect public beliefs) can influence economic fluctuations. We introduce two curated datasets containing posts from X (formerly Twitter) which capture economy-related narratives (Data will be shared upon paper acceptance). Employing Natural Language Processing (NLP) methods, we extract and summarize narratives from the tweets. We test their predictive power for $\textit{macroeconomic}$ forecasting by incorporating the tweets' or the extracted narratives' representations in downstream financial prediction tasks. Our work highlights the challenges in improving macroeconomic models with narrative data, paving the way for the research community to realistically address this important challenge. From a scientific perspective, our investigation offers valuable insights and NLP tools for narrative extraction and summarization using Large Language Models (LLMs), contributing to future research on the role of narratives in economics.

Training large vision-language models requires extensive, high-quality image-text pairs. Existing web-scraped datasets, however, are noisy and lack detailed image descriptions. To bridge this gap, we introduce PixelProse, a comprehensive dataset of over 16M (million) synthetically generated captions, leveraging cutting-edge vision-language models for detailed and accurate descriptions. To ensure data integrity, we rigorously analyze our dataset for problematic content, including child sexual abuse material (CSAM), personally identifiable information (PII), and toxicity. We also provide valuable metadata such as watermark presence and aesthetic scores, aiding in further dataset filtering. We hope PixelProse will be a valuable resource for future vision-language research. PixelProse is available at https://huggingface.co/datasets/tomg-group-umd/pixelprose, Comment: pixelprose 16M dataset

Most public instruction finetuning datasets are relatively small compared to the closed source datasets used to train industry models. To study questions about finetuning at scale, such as curricula and learning rate cooldown schedules, there is a need for industrial-scale datasets. However, this scale necessitates a data generation process that is almost entirely automated. In this work, we study methods for generating large instruction datasets from a single prompt. With little human oversight, we get LLMs to write diverse sets of instruction examples ranging from simple completion tasks to complex multi-turn dialogs across a variety of subject areas. When finetuning a Llama-3 8B base model, our dataset meets or exceeds both WizardLM and Ultrachat on both knowledge-intensive leaderboard tasks as well as conversational evaluations. We release our dataset, the "generator" prompts that created it, and our finetuned model checkpoints., Comment: 9.5 pages, 6 Figures, and 3 tables in the main body. Dataset available at https://huggingface.co/datasets/tomg-group-umd/GenQA

Recent advancements in novel view synthesis have enabled real-time rendering speeds and high reconstruction accuracy. 3D Gaussian Splatting (3D-GS), a foundational point-based parametric 3D scene representation, models scenes as large sets of 3D Gaussians. Complex scenes can comprise of millions of Gaussians, amounting to large storage and memory requirements that limit the viability of 3D-GS on devices with limited resources. Current techniques for compressing these pretrained models by pruning Gaussians rely on combining heuristics to determine which ones to remove. In this paper, we propose a principled spatial sensitivity pruning score that outperforms these approaches. It is computed as a second-order approximation of the reconstruction error on the training views with respect to the spatial parameters of each Gaussian. Additionally, we propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing the training pipeline. After pruning 88.44% of the Gaussians, we observe that our PUP 3D-GS pipeline increases the average rendering speed of 3D-GS by 2.65$\times$ while retaining more salient foreground information and achieving higher image quality metrics than previous pruning techniques on scenes from the Mip-NeRF 360, Tanks & Temples, and Deep Blending datasets.

Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, randomly sampled subsets of tokens are excluded from the loss computation. These dropped tokens are not memorized by the model, which prevents verbatim reproduction of a complete chain of tokens from the training set. We run extensive experiments training billion-scale Llama-2 models, both pre-trained and trained from scratch, and demonstrate significant reductions in extractable memorization with little to no impact on downstream benchmarks., Comment: 10 pages, 8 figures, and 1 table in the main body. Code available at https://github.com/ahans30/goldfish-loss and checkpoints at https://huggingface.co/collections/tomg-group-umd/goldfish-loss-mitigating-memorization-in-llms-66c175becb6aab07744f7272

Reinforcement learning with human feedback~(RLHF) is critical for aligning Large Language Models (LLMs) with human preference. Compared to the widely studied offline version of RLHF, \emph{e.g.} direct preference optimization (DPO), recent works have shown that the online variants achieve even better alignment. However, online alignment requires on-the-fly generation of new training data, which is costly, hard to parallelize, and suffers from varying quality and utility. In this paper, we propose a more efficient data exploration strategy for online preference tuning (OPTune), which does not rely on human-curated or pre-collected teacher responses but dynamically samples informative responses for on-policy preference alignment. During data generation, OPTune only selects prompts whose (re)generated responses can potentially provide more informative and higher-quality training signals than the existing responses. In the training objective, OPTune reweights each generated response (pair) by its utility in improving the alignment so that learning can be focused on the most helpful samples. Throughout our evaluations, OPTune'd LLMs maintain the instruction-following benefits provided by standard preference tuning whilst enjoying 1.27-1.56x faster training speed due to the efficient data exploration strategy., Comment: 16 pages, 7 figures

Eliciting reasoning capabilities from language models (LMs) is a critical direction on the path towards building intelligent systems. Most recent studies dedicated to reasoning focus on out-of-distribution performance on procedurally-generated synthetic benchmarks, bespoke-built to evaluate specific skills only. This trend makes results hard to transfer across publications, slowing down progress. Three years ago, a similar issue was identified and rectified in the field of neural algorithmic reasoning, with the advent of the CLRS benchmark. CLRS is a dataset generator comprising graph execution traces of classical algorithms from the Introduction to Algorithms textbook. Inspired by this, we propose CLRS-Text -- a textual version of these algorithmic traces. Out of the box, CLRS-Text is capable of procedurally generating trace data for thirty diverse, challenging algorithmic tasks across any desirable input distribution, while offering a standard pipeline in which any additional algorithmic tasks may be created in the benchmark. We fine-tune and evaluate various LMs as generalist executors on this benchmark, validating prior work and revealing a novel, interesting challenge for the LM reasoning community. Our code is available at https://github.com/google-deepmind/clrs/tree/master/clrs/_src/clrs_text., Comment: Preprint, under review. Comments welcome