Search

How much cosmological information can we reliably extract from existing and upcoming large-scale structure observations? Many summary statistics fall short in describing the non-Gaussian nature of the late-time Universe in comparison to existing and upcoming measurements. In this article we demonstrate that we can identify optimal summary statistics and that we can link them with existing summary statistics. Using simulation based inference (SBI) with automatic data-compression, we learn summary statistics for galaxy catalogs in the context of cosmological parameter estimation. By construction these summary statistics do not require the ability to write down an explicit likelihood. We demonstrate that they can be used for efficient parameter inference. These summary statistics offer a new avenue for analyzing different simulation models for baryonic physics with respect to their relevance for the resulting cosmological features. The learned summary statistics are low-dimensional, feature the underlying simulation parameters, and are similar across different network architectures. To link our models, we identify the relevant scales associated to our summary statistics (e.g. in the range of modes between $k= 5 - 30 h/\mathrm{Mpc}$) and we are able to match the summary statistics to underlying simulation parameters across various simulation models., Comment: 44 pages, 14 figures

Natural-language assistants are designed to provide users with helpful responses while avoiding harmful outputs, largely achieved through alignment to human preferences. Yet there is limited understanding of whether alignment techniques may inadvertently perpetuate or even amplify harmful biases inherited from their pre-aligned base models. This issue is compounded by the choice of bias evaluation benchmarks in popular preference-finetuned models, which predominantly focus on dominant social categories, such as binary gender, thereby limiting insights into biases affecting underrepresented groups. Towards addressing this gap, we center transgender, nonbinary, and other gender-diverse identities to investigate how alignment procedures interact with pre-existing gender-diverse bias in LLMs. Our key contributions include: 1) a comprehensive survey of bias evaluation modalities across leading preference-finetuned LLMs, highlighting critical gaps in gender-diverse representation, 2) systematic evaluation of gender-diverse biases across 12 models spanning Direct Preference Optimization (DPO) stages, uncovering harms popular bias benchmarks fail to detect, and 3) a flexible framework for measuring harmful biases in implicit reward signals applicable to other social contexts. Our findings reveal that DPO-aligned models are particularly sensitive to supervised finetuning (SFT), and can amplify two forms of real-world gender-diverse harms from their base models: stigmatization and gender non-affirmative language. We conclude with recommendations tailored to DPO and broader alignment practices, advocating for the adoption of community-informed bias evaluation frameworks to more effectively identify and address underrepresented harms in LLMs., Comment: Accepted to 2024 Neurips Queer in AI Workshop

Social media platforms have evolved rapidly in modernity without strong regulation. One clear obstacle faced by current users is that of toxicity. Toxicity on social media manifests through a number of forms, including harassment, negativity, misinformation or other means of divisiveness. In this paper, we characterize literature surrounding toxicity, formalize a definition of toxicity, propose a novel cycle of internet extremism, list current approaches to toxicity detection, outline future directions to minimize toxicity in future social media endeavors, and identify current gaps in research space. We present a novel perspective of the negative impacts of social media platforms and fill a gap in literature to help improve the future of social media platforms., Comment: 12 pages, 2 figures, 2 tables, Cycle of Internet Extremism

We present nifty-ls, a software package for fast and accurate evaluation of the Lomb-Scargle periodogram. nifty-ls leverages the fact that Lomb-Scargle can be computed using a non-uniform FFT (NUFFT), which we evaluate with the Flatiron Institute NUFFT package (finufft). This approach achieves a many-fold speedup over the Press & Rybicki (1989) method as implemented in Astropy and is simultaneously many orders of magnitude more accurate. nifty-ls also supports fast evaluation on GPUs via CUDA and integrates with the Astropy Lomb-Scargle interface. nifty-ls is publicly available as open-source software., Comment: 4 pages, 1 figure, submitted to RNAAS. Software available at https://github.com/flatironinstitute/nifty-ls/

Recognizing errors in assembly and maintenance procedures is valuable for industrial applications, since it can increase worker efficiency and prevent unplanned down-time. Although assembly state recognition is gaining attention, none of the current works investigate assembly error localization. Therefore, we propose StateDiffNet, which localizes assembly errors based on detecting the differences between a (correct) intended assembly state and a test image from a similar viewpoint. StateDiffNet is trained on synthetically generated image pairs, providing full control over the type of meaningful change that should be detected. The proposed approach is the first to correctly localize assembly errors taken from real ego-centric video data for both states and error types that are never presented during training. Furthermore, the deployment of change detection to this industrial application provides valuable insights and considerations into the mechanisms of state-of-the-art change detection algorithms. The code and data generation pipeline are publicly available at: https://timschoonbeek.github.io/error_seg., Comment: 23 pages (14 main paper, 2 references, 7 supplementary), 15 figures (8 main paper, 7 supplementary). Accepted at ECCV Vision-based InduStrial InspectiON (VISION) workshop

In the context of medical decision making, counterfactual prediction enables clinicians to predict treatment outcomes of interest under alternative courses of therapeutic actions given observed patient history. In this work, we present G-Transformer for counterfactual outcome prediction under dynamic and time-varying treatment strategies. Our approach leverages a Transformer architecture to capture complex, long-range dependencies in time-varying covariates while enabling g-computation, a causal inference method for estimating the effects of dynamic treatment regimes. Specifically, we use a Transformer-based encoder architecture to estimate the conditional distribution of relevant covariates given covariate and treatment history at each time point, then produces Monte Carlo estimates of counterfactual outcomes by simulating forward patient trajectories under treatment strategies of interest. We evaluate G-Transformer extensively using two simulated longitudinal datasets from mechanistic models, and a real-world sepsis ICU dataset from MIMIC-IV. G-Transformer outperforms both classical and state-of-the-art counterfactual prediction models in these settings. To the best of our knowledge, this is the first Transformer-based architecture that supports g-computation for counterfactual outcome prediction under dynamic and time-varying treatment strategies.

Empirical evidence for a gap between the computational powers of classical and quantum computers has been provided by experiments that sample the output distributions of two-dimensional quantum circuits. Many attempts to close this gap have utilized classical simulations based on tensor network techniques, and their limitations shed light on the improvements to quantum hardware required to frustrate classical simulability. In particular, quantum computers having in excess of $\sim 50$ qubits are primarily vulnerable to classical simulation due to restrictions on their gate fidelity and their connectivity, the latter determining how many gates are required (and therefore how much infidelity is suffered) in generating highly-entangled states. Here, we describe recent hardware upgrades to Quantinuum's H2 quantum computer enabling it to operate on up to $56$ qubits with arbitrary connectivity and $99.843(5)\%$ two-qubit gate fidelity. Utilizing the flexible connectivity of H2, we present data from random circuit sampling in highly connected geometries, doing so at unprecedented fidelities and a scale that appears to be beyond the capabilities of state-of-the-art classical algorithms. The considerable difficulty of classically simulating H2 is likely limited only by qubit number, demonstrating the promise and scalability of the QCCD architecture as continued progress is made towards building larger machines., Comment: Includes minor updates to the text and an updated author list to include researchers who made technical contributions in upgrading the machine to 56 qubits but were left off the original version by mistake

We measure the clustering of Lyman Alpha Emitting galaxies (LAEs) selected from the One-hundred-square-degree DECam Imaging in Narrowbands (ODIN) survey, with spectroscopic follow-up from Dark Energy Spectroscopic Instrument (DESI). We use DESI spectroscopy to optimize our selection and to constrain the interloper fraction and redshift distribution of our narrow-band selected sources. We select samples of 4000 LAEs at z=2.45 and 3.1 in 9 sq.deg. centered on the COSMOS field with median LyA fluxes of 10^{-16}erg/s/cm2. Covariances and cosmological inferences are obtained from a series of mock catalogs built upon high-resolution N-body simulations that match the footprint, number density, redshift distribution and observed clustering of the sample. We find that both samples have a correlation length of r_0=(3.0\pm 0.2)Mpc/h. Within our fiducial cosmology these correspond to 3D number densities of 10^{-3} h^3/Mpc^3 and, from our mock catalogs, biases of 1.7 and 2.0 at z=2.45 and 3.1, respectively. We discuss the implications of these measurements for the use of LAEs as large-scale structure tracers for high-redshift cosmology., Comment: 26 pages, 13 figures; minor revisions to match version accepted by JCAP

We present the development of a new astrochemical research tool HILTRAC, the Highly Instrumented Low Temperature ReAction Chamber. The instrument is based on a pulsed form of the CRESU (Cin\'etique de R\'eaction en \'Ecoulement Supersonique Uniforme, meaning reaction kinetics in a uniform supersonic flow) apparatus, with the aim of collecting kinetics and spectroscopic information on gas phase chemical reactions important in interstellar space or planetary atmospheres. We discuss the apparatus design and its flexibility, the implementation of pulsed laser photolysis followed by laser induced fluorescence (PLP-LIF), and the first implementation of direct infrared frequency comb spectroscopy (DFCS) coupled to the uniform supersonic flow. Achievable flow temperatures range from 32(3) - 111(9) K, characterising a total of five Laval nozzles for use with N2 and Ar buffer gases by pressure impact measurements. These results were further validated using LIF and DFCS measurements of the CH radical and OCS, respectively. Spectroscopic constants and linelists for OCS are reported for the 1001 band near $2890 - 2940 cm^{-1}$ for both $OC^{32}S$ and $OC^{34}S$, measured using DFCS. Additional peaks in the spectrum are tentatively assigned to the OCS-Ar complex. The first reaction rate coefficients for the CH + OCS reaction measured between 32(3) K and 58(5) K are reported. The reaction rate coefficient at 32(3) K was measured to be $3.9(4) \times 10^{10} cm^3 molecule^{-1} s^{-1}$ and the reaction was found to exhibit no observable temperature dependence over this low temperature range., Comment: The following article has been submitted to The Journal of Chemical Physics. After it is published, it will be found at https://publishing.aip.org/resources/librarians/products/journals/ Copyright 2024 Daniel I. Lucas, Th\'eo Guillaume, Dwayne E. Heard, and Julia H. Lehman. This article is distributed under a Creative Commons Attribution (CC BY) License

Using a scale-free $N$-body simulation generated with the ABACUS $N$-body code, we test the robustness of halo mass accretion histories via their convergence to self-similarity. We compare two halo finders, ROCKSTAR and COMPASO. We find superior self-similarity in halo mass accretion histories determined using ROCKSTAR, with convergence to 5% or better between $\sim10^2$ to $10^5$ particles. For COMPASO we find weaker convergence over a similar region, with at least 10% between $\sim10^2$ to $10^4$ particles. Furthermore, we find the convergence to self-similarity improves as the simulation evolves, with the largest and deepest regions of convergence appearing after the scale factor quadrupled from the time at which non-linear structures begin to form. With sufficient time evolution, halo mass accretion histories are converged to self-similarity within 5% with as few as $\sim70$ particles for COMPASO and within 2% for as few as $\sim30$ particles for ROCKSTAR., Comment: 19 pages, 12 figures

This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, providing an overview of AI assistants, their technical foundations and potential range of applications. It then explores questions around AI value alignment, well-being, safety and malicious uses. Extending the circle of inquiry further, we next consider the relationship between advanced AI assistants and individual users in more detail, exploring topics such as manipulation and persuasion, anthropomorphism, appropriate relationships, trust and privacy. With this analysis in place, we consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants. Finally, we conclude by providing a range of recommendations for researchers, developers, policymakers and public stakeholders.

A measurement of a primordial non-Gaussianity (PNG) signal through late- or early-Universe probes has the potential to transform our understanding of the physics of the primordial Universe. While large-scale structure observables in principle contain vital information, interpreting these measurements is challenging due to poorly understood astrophysical effects. Luckily, $N$-body simulations, such as the public \textsc{AbacusPNG} set presented in this study, consisting of 9 boxes, each of size $L_{\rm box} = 2~{\rm Gpc}/h$ and particle mass of $1.01 \times 10^{10} \ M_\odot/h$, provide a viable path forward. As validation, we find good agreement between the simulations and our expectations from one-loop perturbation theory and the `separate universe' method for the matter bispectrum, matter power spectrum and the halo bias parameter associated with PNG, $b_\phi$. As a science application, we investigate the link between halo assembly bias and $b_\phi$ for halo properties known to play a vital role in accurately predicting galaxy clustering: concentration, shear (environment), and accretion rate. We find a strong response for all three parameters, suggesting that the connection between $b_\phi$ and the assembly history of halos needs to be taken into account by future PNG analyses. We further perform the first study of the $b_\phi$ parameter from fits to early DESI data of the luminous red galaxy (LRG) and quasi-stellar object (QSO) samples and comment on the effect on $f_{\rm NL}$ constraints for the allowed galaxy-halo models (note that $\sigma [f_{\rm NL}] \propto \frac{\sigma [b_\phi]}{b_\phi}$). We find that the error on $f_{\rm NL}$ is 21, 6, 22 for the LRGs at $z = 0.5$ and $z = 0.8$ and QSOs at $z = 1.4$, respectively, suggesting that a thorough understanding of galaxy assembly bias is warranted so as to perform robust high-precision analysis of local-type PNG with future surveys., Comment: 21 pages, 7 figures, 2 tables

Gamma-Ray Bursts (GRBs), due to their high luminosities are detected up to redshift 10, and thus have the potential to be vital cosmological probes of early processes in the universe. Fulfilling this potential requires a large sample of GRBs with known redshifts, but due to observational limitations, only 11\% have known redshifts ($z$). There have been numerous attempts to estimate redshifts via correlation studies, most of which have led to inaccurate predictions. To overcome this, we estimated GRB redshift via an ensemble supervised machine learning model that uses X-ray afterglows of long-duration GRBs observed by the Neil Gehrels Swift Observatory. The estimated redshifts are strongly correlated (a Pearson coefficient of 0.93) and have a root mean square error, namely the square root of the average squared error $\langle\Delta z^2\rangle$, of 0.46 with the observed redshifts showing the reliability of this method. The addition of GRB afterglow parameters improves the predictions considerably by 63\% compared to previous results in peer-reviewed literature. Finally, we use our machine learning model to infer the redshifts of 154 GRBs, which increase the known redshifts of long GRBs with plateaus by 94\%, a significant milestone for enhancing GRB population studies that require large samples with redshift., Comment: 12 Figures, 24 pages. Accepted for publication at The Astrophysical Journal Supplement Series

Understanding the physical mechanism behind the formation of a co-rotating thin plane of satellite galaxies, like the one observed around the Milky Way (MW), has been challenging. The perturbations induced by a massive satellite galaxy, like the Large Magellanic Cloud (LMC) provide valuable insight into this problem. The LMC induces an apparent co-rotating motion in the outer halo by displacing the inner regions of the halo with respect to the outer halo. Using the Latte suite of FIRE-2 cosmological simulations of MW-mass galaxies, we confirm that the apparent motion of the outer halo induced by the infall of a massive satellite changes the observed distribution of orbital poles of outer-halo tracers, including satellites. We quantify the changes in the distribution of orbital poles using the two-point angular correlation function and find that all satellites induce changes. However, the most massive satellites with pericentric passages between 30-100kpc induce the largest changes. The best LMC-like satellite analog shows the largest change in orbital pole distribution. The dispersion of orbital poles decreases by 20{\deg} during the first two pericentric passages. Even when excluding the satellites brought in with the LMC-like satellite, there is clustering of orbital poles. These results suggest that in the MW, the recent pericentric passage of the LMC should have changed the observed distribution of orbital poles of all other satellites. Therefore, studies of kinematically-coherent planes of satellites that seek to place the MW in a cosmological context should account for the existence of a massive satellite like the LMC., Comment: 20 pages, 10 figures. ApJ submitted, Comments are welcome

Sepsis is a life-threatening condition triggered by an extreme infection response. Our objective is to forecast sepsis patient outcomes using their medical history and treatments, while learning interpretable state representations to assess patients' risks in developing various adverse outcomes. While neural networks excel in outcome prediction, their limited interpretability remains a key issue. In this work, we use knowledge distillation via constrained variational inference to distill the knowledge of a powerful "teacher" neural network model with high predictive power to train a "student" latent variable model to learn interpretable hidden state representations to achieve high predictive performance for sepsis outcome prediction. Using real-world data from the MIMIC-IV database, we trained an LSTM as the "teacher" model to predict mortality for sepsis patients, given information about their recent history of vital signs, lab values and treatments. For our student model, we use an autoregressive hidden Markov model (AR-HMM) to learn interpretable hidden states from patients' clinical time series, and use the posterior distribution of the learned state representations to predict various downstream outcomes, including hospital mortality, pulmonary edema, need for diuretics, dialysis, and mechanical ventilation. Our results show that our approach successfully incorporates the constraint to achieve high predictive power similar to the teacher model, while maintaining the generative performance., Comment: Extended Abstract presented at Machine Learning for Health (ML4H) symposium 2023, December 10th, 2023, New Orleans, United States, 12 pages

Automated classification of cancer pathology reports can extract information from unstructured reports and categorize each report into structured diagnosis and severity categories. Thus, such system can reduce the burden for populating tumor registries, help registration for clinical trial as well as developing large dataset for deep learning model development using true pathologic ground truth. However, the content of breast pathology reports can be difficult for categorize due to the high linguistic variability in content and wide variety of potential diagnoses >50. Existing NLP models are primarily focused on developing classifier for primary breast cancer types (e.g. IDC, DCIS, ILC) and tumor characteristics, and ignore the rare diagnosis of cancer subtypes. We then developed a hierarchical hybrid transformer-based pipeline (59 labels) - Hierarchical Classification System for Breast Cancer Specimen Report (HCSBC), which utilizes the potential of the transformer context-preserving NLP technique and compared our model to several state of the art ML and DL models. We trained the model on the EUH data and evaluated our model's performance on two external datasets - MGH and Mayo Clinic. We publicly release the code and a live application under Huggingface spaces repository

A recent line of empirical studies has demonstrated that SGD might exhibit a heavy-tailed behavior in practical settings, and the heaviness of the tails might correlate with the overall performance. In this paper, we investigate the emergence of such heavy tails. Previous works on this problem only considered, up to our knowledge, online (also called single-pass) SGD, in which the emergence of heavy tails in theoretical findings is contingent upon access to an infinite amount of data. Hence, the underlying mechanism generating the reported heavy-tailed behavior in practical settings, where the amount of training data is finite, is still not well-understood. Our contribution aims to fill this gap. In particular, we show that the stationary distribution of offline (also called multi-pass) SGD exhibits 'approximate' power-law tails and the approximation error is controlled by how fast the empirical distribution of the training data converges to the true underlying data distribution in the Wasserstein metric. Our main takeaway is that, as the number of data points increases, offline SGD will behave increasingly 'power-law-like'. To achieve this result, we first prove nonasymptotic Wasserstein convergence bounds for offline SGD to online SGD as the number of data points increases, which can be interesting on their own. Finally, we illustrate our theory on various experiments conducted on synthetic data and neural networks., Comment: In Neural Information Processing Systems (NeurIPS), Spotlight Presentation, 2023

In many text-generation problems, users may prefer not only a single response, but a diverse range of high-quality outputs from which to choose. Quality-diversity (QD) search algorithms aim at such outcomes, by continually improving and diversifying a population of candidates. However, the applicability of QD to qualitative domains, like creative writing, has been limited by the difficulty of algorithmically specifying measures of quality and diversity. Interestingly, recent developments in language models (LMs) have enabled guiding search through AI feedback, wherein LMs are prompted in natural language to evaluate qualitative aspects of text. Leveraging this development, we introduce Quality-Diversity through AI Feedback (QDAIF), wherein an evolutionary algorithm applies LMs to both generate variation and evaluate the quality and diversity of candidate text. When assessed on creative writing domains, QDAIF covers more of a specified search space with high-quality samples than do non-QD controls. Further, human evaluation of QDAIF-generated creative texts validates reasonable agreement between AI and human evaluation. Our results thus highlight the potential of AI feedback to guide open-ended search for creative and original solutions, providing a recipe that seemingly generalizes to many domains and modalities. In this way, QDAIF is a step towards AI systems that can independently search, diversify, evaluate, and improve, which are among the core skills underlying human society's capacity for innovation., Comment: minor additions to supplementary results

Reinforcement Learning from Human Feedback (RLHF) has shown potential in qualitative tasks where easily defined performance measures are lacking. However, there are drawbacks when RLHF is commonly used to optimize for average human preferences, especially in generative tasks that demand diverse model responses. Meanwhile, Quality Diversity (QD) algorithms excel at identifying diverse and high-quality solutions but often rely on manually crafted diversity metrics. This paper introduces Quality Diversity through Human Feedback (QDHF), a novel approach that progressively infers diversity metrics from human judgments of similarity among solutions, thereby enhancing the applicability and effectiveness of QD algorithms in complex and open-ended domains. Empirical studies show that QDHF significantly outperforms state-of-the-art methods in automatic diversity discovery and matches the efficacy of QD with manually crafted diversity metrics on standard benchmarks in robotics and reinforcement learning. Notably, in open-ended generative tasks, QDHF substantially enhances the diversity of text-to-image generation from a diffusion model and is more favorably received in user studies. We conclude by analyzing QDHF's scalability, robustness, and quality of derived diversity metrics, emphasizing its strength in open-ended optimization tasks. Code and tutorials are available at https://liding.info/qdhf., Comment: ICML 2024

Selecting the ``right'' amount of information to include in a summary is a difficult task. A good summary should be detailed and entity-centric without being overly dense and hard to follow. To better understand this tradeoff, we solicit increasingly dense GPT-4 summaries with what we refer to as a ``Chain of Density'' (CoD) prompt. Specifically, GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities without increasing the length. Summaries generated by CoD are more abstractive, exhibit more fusion, and have less of a lead bias than GPT-4 summaries generated by a vanilla prompt. We conduct a human preference study on 100 CNN DailyMail articles and find that that humans prefer GPT-4 summaries that are more dense than those generated by a vanilla prompt and almost as dense as human written summaries. Qualitative analysis supports the notion that there exists a tradeoff between informativeness and readability. 500 annotated CoD summaries, as well as an extra 5,000 unannotated summaries, are freely available on HuggingFace (https://huggingface.co/datasets/griffin/chain_of_density)., Comment: preprint

This paper presents a new symbolic corpus of musical themes from the complete Star Wars trilogies (Episodes I-IX) by John Williams. The corpus files are made available in multiple formats (.krn, .sib, and .musicxml) and include melodic, harmonic, and formal information. The Star Wars Thematic Corpus (SWTC) contains a total of 64 distinctive, recurring, and symbolically meaningful themes and motifs, commonly referred to as leitmotifs. Through this corpus we also introduce a new humdrum standard for non-functional harmony encodings, **harte, based on Harte (2005, 2010). This report details the motivation, describes the transcription and encoding processes, and provides some brief summary statistics. While relatively small in scale, the SWTC represents a unified collection from one of the most prolific and influential composers of the 20th century, and the under-studied subset of film and multimedia musical material in general. We hope the SWTC will provide insights into John Williams' compositional style, as well as prove useful in comparisons against other thematic corpora from film and beyond., Comment: Corpus report (5000 words)