Search

The fall 2023 and spring 2024 undergraduate enrollment increases, marking the first growth since the COVID-19 pandemic, show signs of a post-pandemic turnaround for higher education. However, a significant share of current undergraduates will eventually disengage from college before earning a degree or other credential. They will join tens of millions of other adult Americans who are Some College, No Credential (SCNC). The SCNC population has been consistently rising over time. Re-engaging those who stop out remains a persistent challenge and a priority for the forty states that have set ambitious postsecondary attainment goals. This report aims to provide timely insights into the SCNC population, offering state leaders and policymakers accurate data on its current status, along with tracking progress and outcome measures for SCNC students. The first section of this report describes who makes up the SCNC population and how it has changed since the last report. In this section, the authors pay particular attention to Recent Stopouts, who joined the SCNC population after being stopped out between January 2021 and July 2022. In the second section, the authors report on SCNC re-enrollment in the 2022-23 academic year as well as first-year credential earning for re-enrollees. The authors also provide new updates on continued enrollment and second-year credential earning for SCNC re-enrollees in the 2021-22 academic year, whom were first reported on last year.

While coreference resolution is traditionally used as a component in individual document understanding, in this work we take a more global view and explore what can we learn about a domain from the set of all document-level coreference relations that are present in a large corpus. We derive coreference chains from a corpus of 30 million biomedical abstracts and construct a graph based on the string phrases within these chains, establishing connections between phrases if they co-occur within the same coreference chain. We then use the graph structure and the betweeness centrality measure to distinguish between edges denoting hierarchy, identity and noise, assign directionality to edges denoting hierarchy, and split nodes (strings) that correspond to multiple distinct concepts. The result is a rich, data-driven ontology over concepts in the biomedical domain, parts of which overlaps significantly with human-authored ontologies. We release the coreference chains and resulting ontology under a creative-commons license, along with the code.

Structured state space models (SSMs), the core engine behind prominent neural networks such as S4 and Mamba, are linear dynamical systems adhering to a specified structure, most notably diagonal. In contrast to typical neural network modules, whose parameterizations are real, SSMs often use complex parameterizations. Theoretically explaining the benefits of complex parameterizations for SSMs is an open problem. The current paper takes a step towards its resolution, by establishing formal gaps between real and complex diagonal SSMs. Firstly, we prove that while a moderate dimension suffices in order for a complex SSM to express all mappings of a real SSM, a much higher dimension is needed for a real SSM to express mappings of a complex SSM. Secondly, we prove that even if the dimension of a real SSM is high enough to express a given mapping, typically, doing so requires the parameters of the real SSM to hold exponentially large values, which cannot be learned in practice. In contrast, a complex SSM can express any given mapping with moderate parameter values. Experiments corroborate our theory, and suggest a potential extension of the theory that accounts for selectivity, a new architectural feature yielding state of the art performance., Comment: 12 pages. Accepted to NeurIPS 2024

Human-like personality traits have recently been discovered in large language models, raising the hypothesis that their (known and as yet undiscovered) biases conform with human latent psychological constructs. While large conversational models may be tricked into answering psychometric questionnaires, the latent psychological constructs of thousands of simpler transformers, trained for other tasks, cannot be assessed because appropriate psychometric methods are currently lacking. Here, we show how standard psychological questionnaires can be reformulated into natural language inference prompts, and we provide a code library to support the psychometric assessment of arbitrary models. We demonstrate, using a sample of 88 publicly available models, the existence of human-like mental health-related constructs (including anxiety, depression, and Sense of Coherence) which conform with standard theories in human psychology and show similar correlations and mitigation strategies. The ability to interpret and rectify the performance of language models by using psychological tools can boost the development of more explainable, controllable, and trustworthy models.

We propose a variant of chain of thought (CoT) prompting called Program Trace Prompting that makes explanations more observable while preserving the power, generality and flexibility of CoT. In our approach, few-shot CoT demonstrations are wrapped in a formal syntax based on Python, and each prompt: identifies and names steps; defines the input/output behavior of steps; and replaces CoT explanations of in-context examples with chains of these formalized steps on the same examples. Program Trace Prompting is applicable to many tasks, achieving strong results on the 23 diverse tasks in the BIG-Bench Hard benchmark. More importantly, by instrumenting explanations in this way, we enable new types of analysis. In particular, we identify "non-local errors" (which correspond to incorrectly learning the reasoning method illustrated in the demonstrations) as an unaddressed issue in CoT learning, and we present methods for verifying the modularity of steps in a CoT explanation.

Assessing instruction quality is a fundamental component of any improvement efforts in the education system. However, traditional manual assessments are expensive, subjective, and heavily dependent on observers' expertise and idiosyncratic factors, preventing teachers from getting timely and frequent feedback. Different from prior research that focuses on low-inference instructional practices, this paper presents the first study that leverages Natural Language Processing (NLP) techniques to assess multiple high-inference instructional practices in two distinct educational settings: in-person K-12 classrooms and simulated performance tasks for pre-service teachers. This is also the first study that applies NLP to measure a teaching practice that has been demonstrated to be particularly effective for students with special needs. We confront two challenges inherent in NLP-based instructional analysis, including noisy and long input data and highly skewed distributions of human ratings. Our results suggest that pretrained Language Models (PLMs) demonstrate performances comparable to the agreement level of human raters for variables that are more discrete and require lower inference, but their efficacy diminishes with more complex teaching practices. Interestingly, using only teachers' utterances as input yields strong results for student-centered variables, alleviating common concerns over the difficulty of collecting and transcribing high-quality student speech data in in-person teaching settings. Our findings highlight both the potential and the limitations of current NLP techniques in the education domain, opening avenues for further exploration.

This First Look report provides selected findings from the High School Longitudinal Study of 2009 (HSLS:09) Postsecondary Education Administrative Records Collection (PEAR). HSLS:09 follows a nationally representative sample of students who were ninth-graders in fall 2009 from high school into postsecondary education and the workforce. The PEAR data collection was conducted in 2021, approximately 8 years after high school graduation for most of the cohort. These data provide information on whether fall 2009 ninth-graders enrolled in postsecondary education by June 2021, and allow researchers to examine enrollment characteristics, degree completion, and financial aid awards for the subset of fall 2009 ninth-graders who enrolled in postsecondary education.

Despite calls to modernize education preparation, the way we train, support, and grow educators has remained largely unchanged for decades. But some educator training and professional development organizations are taking a different approach by offering educators flexible and job-embedded learning opportunities that recognize and validate learning through demonstrations of competence. Educators earn "micro-credentials" in the form of digital badges, which capture both the skill the educator demonstrated and the evidence they used to prove their mastery of that skill. This case study offers a look at one micro-credential program, developed by UCLA's ExcEL Leadership Academy that has been approved for ESOL teacher certification in Rhode Island. The program offers a progression of 12 micro-credentials focused on the skills and competencies educators need to serve multilingual learners (MLLs) effectively. Additionally, the case study offers recommendations for other states that hope to offer their educators high-quality competency-based pathways to certification and/or professional growth.

Fokker-Planck equations (forward Kolmogorov equations) evolve probability densities in time from an initial condition. For distributions over the real line, these evolution equations can sometimes be transformed into dynamics over the incomplete zeroth and first moments. We call this perspective the Lorenz dynamics of the system after the Lorenz curve description of distributions of wealth. This offers the benefit of presenting the dynamics over a compact domain. The integral transformation is motivated and then stated for a general class of Fokker-Planck equations. Following this, the transformed equation is solved for the heat equation and some variants thereof. Finally, some equations arising from the application of kinetic theory to idealized economic systems are transformed and analyzed in this new light.

Recent work has documented striking heterogeneity in the performance of state-of-the-art vision language models (VLMs), including both multimodal language models and text-to-image models. These models are able to describe and generate a diverse array of complex, naturalistic images, yet they exhibit surprising failures on basic multi-object reasoning tasks -- such as counting, localization, and simple forms of visual analogy -- that humans perform with near perfect accuracy. To better understand this puzzling pattern of successes and failures, we turn to theoretical accounts of the binding problem in cognitive science and neuroscience, a fundamental problem that arises when a shared set of representational resources must be used to represent distinct entities (e.g., to represent multiple objects in an image), necessitating the use of serial processing to avoid interference. We find that many of the puzzling failures of state-of-the-art VLMs can be explained as arising due to the binding problem, and that these failure modes are strikingly similar to the limitations exhibited by rapid, feedforward processing in the human brain.

Cohen-Addad, Le, Pilipczuk, and Pilipczuk [CLPP23] recently constructed a stochastic embedding with expected $1+\varepsilon$ distortion of $n$-vertex planar graphs (with polynomial aspect ratio) into graphs of treewidth $O(\varepsilon^{-1}\log^{13} n)$. Their embedding is the first to achieve polylogarithmic treewidth. However, there remains a large gap between the treewidth of their embedding and the treewidth lower bound of $\Omega(\log n)$ shown by Carroll and Goel [CG04]. In this work, we substantially narrow the gap by constructing a stochastic embedding with treewidth $O(\varepsilon^{-1}\log^{3} n)$. We obtain our embedding by improving various steps in the CLPP construction. First, we streamline their embedding construction by showing that one can construct a low-treewidth embedding for any graph from (i) a stochastic hierarchy of clusters and (ii) a stochastic balanced cut. We shave off some logarithmic factors in this step by using a single hierarchy of clusters. Next, we construct a stochastic hierarchy of clusters with optimal separating probability and hop bound based on shortcut partition [CCLMST23, CCLMST24]. Finally, we construct a stochastic balanced cut with an improved trade-off between the cut size and the number of cuts. This is done by a new analysis of the contraction sequence introduced by [CLPP23]; our analysis gives an optimal treewidth bound for graphs admitting a contraction sequence., Comment: 39 pages, 6 figures

Optimization in deep learning remains poorly understood, even in the simple setting of deterministic (i.e. full-batch) training. A key difficulty is that much of an optimizer's behavior is implicitly determined by complex oscillatory dynamics, referred to as the "edge of stability." The main contribution of this paper is to show that an optimizer's implicit behavior can be explicitly captured by a "central flow:" a differential equation which models the time-averaged optimization trajectory. We show that these flows can empirically predict long-term optimization trajectories of generic neural networks with a high degree of numerical accuracy. By interpreting these flows, we reveal for the first time 1) the precise sense in which RMSProp adapts to the local loss landscape, and 2) an "acceleration via regularization" mechanism, wherein adaptive optimizers implicitly navigate towards low-curvature regions in which they can take larger steps. This mechanism is key to the efficacy of these adaptive optimizers. Overall, we believe that central flows constitute a promising tool for reasoning about optimization in deep learning., Comment: first two authors contributed equally; author order determined by coin flip

The measurement of type Ia supernovae magnitudes provides cosmological distances, which can be used to constrain dark energy parameters. Large photometric surveys require a substantial improvement in the calibration precision of their photometry to reduce systematic uncertainties in cosmological constraints. The StarDICE experiment is designed to establish accurate broadband flux references for these surveys, aiming for sub-percent precision in magnitude measurements. This requires a precise measurement of the filter bandpasses of both the StarDICE and survey instruments with sub-nanometer accuracy. To that end, we have developed the Collimated Beam Projector (CBP), an optical device capable of calibrating the throughput of an astronomical telescope and of its filters. The CBP is built from a tunable laser source and a reversed telescope to emit a parallel monochromatic light beam that is continuously monitored in flux and wavelength. The CBP output light flux is measured using a large area photodiode, previously calibrated relative to a NIST photodiode. We derive the StarDICE telescope throughput and filter transmissions from the CBP measurements, anchoring it to the absolute calibration provided by the NIST. After analyzing the systematic uncertainties, we achieved sub-nanometer accuracy in determining filter central wavelengths, measured each filter transmission with a precision of 0.5% per 1nm bin, and detected out-of-band leakages at 0.01%. Furthermore, we have synthesized the equivalent transmission for full pupil illumination from four sample positions in the StarDICE telescope mirror, with an accuracy of approximately 0.2nm for central wavelengths and 7mmag for broadband fluxes. We demonstrated our ability to characterize a telescope throughput down to the mmag, and paved the way for future developments, such as a portable CBP version for in-situ transmission monitoring., Comment: 29 pages, 50 figures

Given large data sets and sufficient compute, is it beneficial to design neural architectures for the structure and symmetries of each problem? Or is it more efficient to learn them from data? We study empirically how equivariant and non-equivariant networks scale with compute and training samples. Focusing on a benchmark problem of rigid-body interactions and on general-purpose transformer architectures, we perform a series of experiments, varying the model size, training steps, and dataset size. We find evidence for three conclusions. First, equivariance improves data efficiency, but training non-equivariant models with data augmentation can close this gap given sufficient epochs. Second, scaling with compute follows a power law, with equivariant models outperforming non-equivariant ones at each tested compute budget. Finally, the optimal allocation of a compute budget onto model size and training duration differs between equivariant and non-equivariant models.

Drug discovery typically consists of multiple steps, including identifying a target protein key to a disease's etiology, validating that interacting with this target could prevent symptoms or cure the disease, discovering a small molecule or biologic therapeutic to interact with it, and optimizing the candidate molecule through a complex landscape of required properties. Drug discovery related tasks often involve prediction and generation while considering multiple entities that potentially interact, which poses a challenge for typical AI models. For this purpose we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a method that we applied to create a versatile multi-task multi-align foundation model that learns from large-scale biological datasets (2 billion samples) across diverse modalities, including proteins, small molecules, and genes. We introduce a prompt syntax that supports a wide range of classification, regression, and generation tasks. It allows combining different modalities and entity types as inputs and/or outputs. Our model handles combinations of tokens and scalars and enables the generation of small molecules and proteins, property prediction, and transcriptomic lab test predictions. We evaluated the model on 11 diverse downstream tasks spanning different steps within a typical drug discovery pipeline, where it reaches new SOTA in 9 tasks and is comparable to SOTA in 2 tasks. This performance is achieved while using a unified architecture serving all tasks, in contrast to the original SOTA performance achieved using tailored architectures. The model code and pretrained weights are publicly available at https://github.com/BiomedSciAI/biomed-multi-alignment and https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m.

We elaborate on a recently proposed geometric framework for scalar effective field theories. Starting from the action, a metric can be identified that enables the construction of geometric quantities on the associated functional manifold. These objects transform covariantly under general field redefinitions that relate different operator bases, including those involving derivatives. We present a novel geometric formula for the amplitudes of the theory, where the vertices in Feynman diagrams are replaced by their geometrized counterparts. This makes the on-shell covariance of amplitudes manifest, providing the link between functional geometry and effective field theories., Comment: 7 pages + supplemental material

Fine-tuning pre-trained large language models (LLMs) on a diverse array of tasks has become a common approach for building models that can solve various natural language processing (NLP) tasks. However, where and to what extent these models retain task-specific knowledge remains largely unexplored. This study investigates the task-specific information encoded in pre-trained LLMs and the effects of instruction tuning on their representations across a diverse set of over 60 NLP tasks. We use a set of matrix analysis tools to examine the differences between the way pre-trained and instruction-tuned LLMs store task-specific information. Our findings reveal that while some tasks are already encoded within the pre-trained LLMs, others greatly benefit from instruction tuning. Additionally, we pinpointed the layers in which the model transitions from high-level general representations to more task-oriented representations. This finding extends our understanding of the governing mechanisms of LLMs and facilitates future research in the fields of parameter-efficient transfer learning and multi-task learning., Comment: Accepted to EMNLP 2024

By analyzing the spectral energy distributions (SEDs) of resolved stars in nearby galaxies, we can constrain their stellar properties and line-of-sight dust extinction. From the Scylla survey, we obtain ultraviolet to near-infrared photometry from Wide Field Camera 3 onboard the {\it Hubble Space Telescope} for more than 1.5 million stars in the SMC and LMC. We use the Bayesian Extinction and Stellar Tool (BEAST) to analyze the multi-band SEDs of these sources and characterize their initial masses, ages, metallicities, distances, and line-of-sight extinction properties (e.g.~$A_V$, $R_V$). We apply quality cuts and perform validation simulations to construct a catalog of over 550,000 stars with high-reliability SED fits, which we use to analyze the stellar content and extinction properties of the SMC and LMC. We detect stars with masses as low as 0.6 $M_{\odot}$. Observed stellar age distributions show a jump in stars around 6 Gyrs ago, which is in agreement with other star-formation histories. Extinctions ($A_V$) in both galaxies follow a log-normal distribution. We compare $A_V$ with ancillary gas and dust tracers like $HI$, $H_\alpha$, and far infrared (FIR) dust emission and find positive correlations on a field-by-field basis. We convert observed $A_V$ to predicted dust surface densities using the Draine et. al. (2014) model and find $A_V$-based dust surface densities are a factor of $\sim$2.5 lower than observed FIR-based dust surface densities, a correction factor similar to other studies., Comment: Submitted to ApJ, 31 pages

In this study, we employ the combined charged particle measurements from Integrated Science Investigation of the Sun (\ISOIS) onboard the Parker Solar Probe (PSP) and Energetic Particle Detector (EPD) onboard the Solar Orbiter (SolO) to study the composition variation of the solar energetic particle (SEP) event occurring on May 16, 2023. During the event, SolO and PSP were located at a similar radial distance of ~0.7 au and were separated by $\sim$60$^\circ$ in longitude. The footpoints of both PSP and SolO were west of the flare region but the former was much closer (18$^\circ$ vs 80$^\circ$). Such a distribution of observers is ideal for studying the longitudinal dependence of the ion composition with the minimum transport effects of particles along the radial direction. We focus on H, He, O, and Fe measured by both spacecraft in sunward and anti-sunward directions. Their spectra are in a double power-law shape, which is fitted best by the Band function. Notably, the event was Fe-rich at PSP, where the mean Fe/O ratio at energies of 0.1 - 10 Mev/nuc was 0.48, higher than the average Fe/O ratio in previous large SEP events. In contrast, the mean Fe/O ratio at SolO over the same energy range was considerable lower at 0.08. The Fe/O ratio between 0.5 and 10 MeV/nuc at both spacecraft is nearly constant. Although the He/H ratio shows energy dependence, decreasing with increasing energy, the He/H ratio at PSP is still about twice as high as that at SolO. Such a strong longitudinal dependence of element abundances and the Fe-rich component in the PSP data could be attributed to the direct flare contribution. Moreover, the temporal profiles indicate that differences in the Fe/O and He/H ratios between PSP and SolO persisted throughout the entire event rather than only at the start., Comment: 11 pages, 5 figures

This research presents an online path planner for Unmanned Aerial Vehicles (UAVs) that can handle dynamic obstacles and UAV motion constraints, including maximum curvature and desired orientations. Our proposed planner uses a NURBS path representation and a Differential Evolution algorithm, incorporating concepts from the Velocity Obstacle approach in a constraint function. Initial results show that our approach is feasible and provides a foundation for future extensions to three-dimensional (3D) environments., Comment: Accepted to the 40th Anniversary of the IEEE Conference on Robotics and Automation (ICRA@40)

Single-shot ptychography is a quantitative phase imaging method wherein overlapping beams of light arranged in a grid pattern simultaneously illuminate a sample, allowing a full ptychographic dataset to be collected in a single shot. It is primarily used at optical wavelengths, but there is interest in using it for X-ray imaging. However, the constraints imposed by X-ray optics have limited the resolution achievable to date. In this work, we reinterpret single-shot ptychography as a structured illumination method by viewing the grid of beams as a single, highly structured illumination function. Pre-calibrating this illumination and reconstructing single-shot data using the randomized probe imaging algorithm allows us to account for the overlap and coherent interference between the diffraction arising from each beam. We achieve a resolution 3.5 times finer than the numerical aperture-based limit imposed by traditional algorithms for single-shot ptychography. We argue that this reconstruction method will work better for most single-shot ptychography experiments and discuss the implications for the design of future single-shot X-ray microscopes., Comment: 4 pages, 3 figures

Weather and climate data are often available at limited temporal resolution, either due to storage limitations, or in the case of weather forecast models based on deep learning, their inherently long time steps. The coarse temporal resolution makes it difficult to capture rapidly evolving weather events. To address this limitation, we introduce an interpolation model that reconstructs the atmospheric state between two points in time for which the state is known. The model makes use of a novel network layer that modifies the adaptive Fourier neural operator (AFNO), which has been previously used in weather prediction and other applications of machine learning to physics problems. The modulated AFNO (ModAFNO) layer takes an embedding, here computed from the interpolation target time, as an additional input and applies a learned shift-scale operation inside the AFNO layers to adapt them to the target time. Thus, one model can be used to produce all intermediate time steps. Trained to interpolate between two time steps 6 h apart, the ModAFNO-based interpolation model produces 1 h resolution intermediate time steps that are visually nearly indistinguishable from the actual corresponding 1 h resolution data. The model reduces the RMSE loss of reconstructing the intermediate steps by approximately 50% compared to linear interpolation. We also demonstrate its ability to reproduce the statistics of extreme weather events such as hurricanes and heat waves better than 6 h resolution data. The ModAFNO layer is generic and is expected to be applicable to other problems, including weather forecasting with tunable lead time., Comment: 8 pages, 5 figures

Glioblastoma is a highly aggressive form of brain cancer characterized by rapid progression and poor prognosis. Despite advances in treatment, the underlying genetic mechanisms driving this aggressiveness remain poorly understood. In this study, we employed multimodal deep learning approaches to investigate glioblastoma heterogeneity using joint image/RNA-seq analysis. Our results reveal novel genes associated with glioblastoma. By leveraging a combination of whole-slide images and RNA-seq, as well as introducing novel methods to encode RNA-seq data, we identified specific genetic profiles that may explain different patterns of glioblastoma progression. These findings provide new insights into the genetic mechanisms underlying glioblastoma heterogeneity and highlight potential targets for therapeutic intervention.

Given a partition $\lambda$ of a number $k$, it is known that by adding a long line of length $n-k$, the dimension of the associated representation of $S_{n}$ is an integer-valued polynomial of degree $k$ in $n$. We show that its expansion in the binomial basis is bounded by the length of $\lambda$, and that the resulting coefficient of index $h$, with alternating signs, counts the standard Young tableaux of shape $\lambda$ in which a given collection of consecutive $h$ numbers lie in increasing rows. We also construct bijections in order to demonstare explicitly that this number is indeed independent of the set of consecutive $h$ numbers used., Comment: 19 pages, 1 figure

We propose a novel framework for exploring generalization errors of transfer learning through the lens of differential calculus on the space of probability measures. In particular, we consider two main transfer learning scenarios, $\alpha$-ERM and fine-tuning with the KL-regularized empirical risk minimization and establish generic conditions under which the generalization error and the population risk convergence rates for these scenarios are studied. Based on our theoretical results, we show the benefits of transfer learning with a one-hidden-layer neural network in the mean-field regime under some suitable integrability and regularity assumptions on the loss and activation functions., Comment: Under review

We report observations of direct evidence of energetic protons being accelerated above ~400 keV within the reconnection exhaust of a heliospheric current sheet (HCS) crossing by NASA's Parker Solar Probe (PSP) at a distance of ~16.25 solar radii (Rs) from the Sun. Inside the extended exhaust, both the reconnection-generated plasma jets and the accelerated protons propagated toward the Sun, unambiguously establishing their origin from HCS reconnection sites located beyond PSP. Within the core of the exhaust, PSP detected stably trapped energetic protons up to ~400 keV, which is approximately 1000 times greater than the available magnetic energy per particle. The differential energy spectrum of the accelerated protons behaved as a pure power-law with spectral index of about -5. Supporting simulations using the kglobal model suggest that the trapping and acceleration of protons up to ~400 keV in the reconnection exhaust is likely facilitated by merging magnetic islands with a guide field between ~0.2-0.3 of the reconnecting magnetic field, consistent with the observations. These new results, enabled by PSP's proximity to the Sun, demonstrate that magnetic reconnection in the HCS is a significant new source of energetic particles in the near-Sun solar wind. The discovery of in-situ particle acceleration via magnetic reconnection at the HCS provides valuable insights into this fundamental process which frequently converts the large magnetic field energy density in the near-Sun plasma environment and may be responsible for heating the sun's atmosphere, accelerating the solar wind, and energizing charged particles to extremely high energies in solar flares., Comment: 24 pages, 4 figures

Travel planning is a challenging and time-consuming task that aims to find an itinerary which satisfies multiple, interdependent constraints regarding flights, accommodations, attractions, and other travel arrangements. In this paper, we propose To the Globe (TTG), a real-time demo system that takes natural language requests from users, translates it to symbolic form via a fine-tuned Large Language Model, and produces optimal travel itineraries with Mixed Integer Linear Programming solvers. The overall system takes ~5 seconds to reply to the user request with guaranteed itineraries. To train TTG, we develop a synthetic data pipeline that generates user requests, flight and hotel information in symbolic form without human annotations, based on the statistics of real-world datasets, and fine-tune an LLM to translate NL user requests to their symbolic form, which is sent to the symbolic solver to compute optimal itineraries. Our NL-symbolic translation achieves ~91% exact match in a backtranslation metric (i.e., whether the estimated symbolic form of generated natural language matches the groundtruth), and its returned itineraries have a ratio of 0.979 compared to the optimal cost of the ground truth user request. When evaluated by users, TTG achieves consistently high Net Promoter Scores (NPS) of 35-40% on generated itinerary.

Earth's magnetotail, and the plasma sheet embedded in it, is a highly dynamic region that is coupled to both the solar wind and to the inner magnetosphere. As a consequence of this coupling, the plasma sheet undergoes explosive energy releases in the form of substorms. A substorm is initiated when reconnection is triggered within a thin current sheet, initiating a complex chain of phenomena. One consequence of these phenomena is heating of thermal electrons and acceleration of energetic (non-thermal) electrons. The upper-energy limit as well as the spatial scale size of the electron acceleration regions are ongoing mysteries in magnetotail physics because current missions can only offer us a glimpse into the numerous magnetotail phenomena ranging from electron- to global-scales occurring in this extensive system. Observational difficulties aside, these energetic electrons also provide a significant source of seed electrons for the Van Allen Radiation belts. Here we demonstrate a unique approach to study plasma sheet electron acceleration. We combine Magnetospheric Multiscale (MMS) Mission high-altitude observations with Electron Losses and Fields Investigation (ELFIN) low-altitude observations, to quantify the upper-energy extent and radial scale of a burst of plasma sheet electrons that mapped to 30 Earth radii. We find that the plasma sheet locally accelerated electrons to 2-3 MeV energies -- far higher than previously anticipated -- and scattered them into the atmospheric loss cone. Interestingly, high-altitude observations of the plasma sheet at 17 Earth radii showed only the usual substorm signatures: bursty bulk flows and dipolarizing flux bundles -- demonstrating that this burst was 1) intense, 2) localized to the far magnetotail, and 3) likely accelerated by a very efficient mechanism.

The UltraViolet imaging of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey Fields (UVCANDELS) program provides deep HST F275W and F435W imaging over four CANDELS fields (GOODS-N, GOODS-S, COSMOS, and EGS). We combine this newly acquired UV imaging with existing HST imaging from CANDELS as well as existing ancillary data to obtain robust photometric redshifts and reliable estimates for galaxy physical properties for over 150,000 galaxies in the $\sim$430 arcmin$^2$ UVCANDELS area. Here, we leverage the power of the new UV photometry to not only improve the photometric redshift measurements in these fields, but also constrain the full redshift probability distribution combining multiple redshift fitting tools. Furthermore, using the full UV-to-IR photometric dataset, we measure the galaxy physical properties by fitting templates from population synthesis models with two different parameterizations (flexible and fixed-form) of the star-formation histories (SFHs). Compared to the flexible SFH parametrization, we find that the fixed-form SFHs systematically underestimate the galaxy stellar masses, both at the low- ($\lesssim10^9 M_\odot$) and high- ($\gtrsim10^{10} M_\odot$) mass end, by as much as $\sim0.5$ dex. This underestimation is primarily due the limited ability of fixed-form SFH parameterization to simultaneously capture the chaotic nature of star-formation in these galaxies., Comment: 22 pages, 6 figures; accepted to ApJS; catalogs available via MAST

We present archival HST and JWST ultraviolet through near infrared time series photometric observations of a massive minimal-contact binary candidate in the metal-poor galaxy WLM ($Z = 0.14 Z_{\odot}$). This discovery marks the lowest metallicity contact binary candidate observed to date. We determine the nature of the two stars in the binary by using the eclipsing binary modeling software (PHysics Of Eclipsing BinariEs; PHOEBE) to train a neural network to fit our observed panchromatic multi-epoch photometry. The best fit model consists of two hot MS stars ($T_1=29800^{+2300}_{-1700}$ K, $M_1=16^{+2}_{-3}~M_{\odot}$, and $T_2=18000^{+5000}_{-5000}$ K, $M_2=7^{+5}_{-3}~M_{\odot}$). We discuss plausible evolutionary paths for the system, and suggest the system is likely to be currently in a contact phase before ultimately ending in a merger. Future spectroscopy will help to further narrow down evolutionary pathways. This work showcases a novel use of data of JWST and HST imaging originally taken to characterize RR Lyrae. We expect time series imaging from LSST, BlackGEM, etc. to uncover similar types of objects in nearby galaxies., Comment: comments welcome

In this study, we present the first-ever direct measurements of synchrotron-emitting heliospheric traveling shocks, intercepted by the Parker Solar Probe (PSP) during its close encounters. Given that much of our understanding of powerful astrophysical shocks is derived from synchrotron radiation, these observations by PSP provide an unprecedented opportunity to explore how shocks accelerate relativistic electrons and the conditions under which they emit radiation. The probe's unparalleled capabilities to measure both electromagnetic fields and energetic particles with high precision in the near-Sun environment has allowed us to directly correlate the distribution of relativistic electrons with the resulting photon emissions. Our findings reveal that strong quasi-parallel shocks emit radiation at significantly higher intensities than quasi-perpendicular shocks due to the efficient acceleration of ultra-relativistic electrons. These experimental results are consistent with theory and recent observations of supernova remnant shocks and advance our understanding of shock physics across diverse space environments., Comment: 10 Pages, and 4 figures (main text). 5 Pages, and 2 figures (Appendix). Astrophysical Journal Letters

Conditioning diffusion and flow models have proven effective for super-resolving small-scale details in natural images.However, in physical sciences such as weather, super-resolving small-scale details poses significant challenges due to: (i) misalignment between input and output distributions (i.e., solutions to distinct partial differential equations (PDEs) follow different trajectories), (ii) multi-scale dynamics, deterministic dynamics at large scales vs. stochastic at small scales, and (iii) limited data, increasing the risk of overfitting. To address these challenges, we propose encoding the inputs to a latent base distribution that is closer to the target distribution, followed by flow matching to generate small-scale physics. The encoder captures the deterministic components, while flow matching adds stochastic small-scale details. To account for uncertainty in the deterministic part, we inject noise into the encoder output using an adaptive noise scaling mechanism, which is dynamically adjusted based on maximum-likelihood estimates of the encoder predictions. We conduct extensive experiments on both the real-world CWA weather dataset and the PDE-based Kolmogorov dataset, with the CWA task involving super-resolving the weather variables for the region of Taiwan from 25 km to 2 km scales. Our results show that the proposed stochastic flow matching (SFM) framework significantly outperforms existing methods such as conditional diffusion and flows., Comment: 31 pages

This paper presents a comparative study of two Bayesian approaches - Markov Chain Monte Carlo (MCMC) and Approximate Bayesian Computation (ABC) - for estimating the parameters of autoregressive fractionally-integrated moving average (ARFIMA) models, which are widely used to capture long-memory in time series data. We propose a novel MCMC algorithm that filters the time series into distinct long-memory and ARMA components, and benchmarked it against standard approaches. Additionally, a new ABC method is proposed, using three different summary statistics used for posterior estimation. The methods are implemented and evaluated through an extensive simulation study, as well as applied to a real-world financial dataset, specifically the quarterly U.S. Gross National Product (GNP) series. The results demonstrate the effectiveness of the Bayesian methods in estimating long-memory and short-memory parameters, with the filtered MCMC showing superior performance in various metrics. This study enhances our understanding of Bayesian techniques in ARFIMA modeling, providing insights into their advantages and limitations when applied to complex time series data.

Origin of energetic upstream ions propagating towards the Sun from the Earth's bow shock is not understood clearly. In this letter, relationship between solar wind suprathermal and upstream ions has been investigated by analyzing fluxes of H, 4He, and CNO obtained from multidirectional in-situ measurements at the first Lagrange point of the Sun-Earth system during 2012-2014. 49 upstream events have been selected based on flux enhancements of the upstream ions in comparison with the solar wind suprathermal ions. An energy cut-off at less than 300 keV is observed for the upstream events. This is attributed to the efficacy of the particle acceleration process near the bow shock. Interestingly, spectra of upstream ions soften systematically as compared to the spectra of their solar wind counterpart with decreasing mass of elements. The degree of spectral softening increases with decreasing mass-to-charge ratio of the species. Since during most of the events the interplanetary magnetic field was radial, we argue that cross-field diffusion of upstream ions gives rise to the modulation (spectral softening) of upstream ions, which is dependent on the mass-to-charge ratio of species. Our work indicates towards a systematic change in solar wind suprathermal ions after interaction with the bow shock.

Remote control of robotic systems, also known as teleoperation, is crucial for the development of autonomous vehicle (AV) technology. It allows a remote operator to view live video from AVs and, in some cases, to make real-time decisions. The effectiveness of video-based teleoperation systems is heavily influenced by the quality of the cellular network and, in particular, its packet loss rate and latency. To optimize these parameters, an AV can be connected to multiple cellular networks and determine in real time over which cellular network each video packet will be transmitted. We present an algorithm, called Active Network Selector (ANS), which uses a time series machine learning approach for solving this problem. We compare ANS to a baseline non-learning algorithm, which is used today in commercial systems, and show that ANS performs much better, with respect to both packet loss and packet latency., Comment: IEEE Network of Future 2024

Measurements of the EMC effect in the tritium and helium-3 mirror nuclei are reported. The data were obtained by the MARATHON Jefferson Lab experiment, which performed deep inelastic electron scattering from deuterium and the three-body nuclei, using a cryogenic gas target system and the High Resolution Spectrometers of the Hall A Facility of the Lab. The data cover the Bjorken $x$ range from 0.20 to 0.83, corresponding to a squared four-momentum transfer $Q^2$ range from 2.7 to $11.9\gevsq$, and to an invariant mass $W$ of the final hadronic state greater than 1.84 GeV/${\it c}^2$. The tritium EMC effect measurement is the first of its kind. The MARATHON experimental results are compared to results from previous measurements by DESY-HERMES and JLab-Hall C experiments, as well as with few-body theoretical predictions., Comment: arXiv admin note: text overlap with arXiv:2104.05850

The proximity of the Large and Small Magellanic Clouds (LMC and SMC) provides the opportunity to study the impact of dwarf-dwarf interactions on their mass assembly with a unique level of detail. To this end, we analyze two-filter broadband imaging of 83 Hubble Space Telescope (HST) pointings covering 0.203 deg$^2$ towards the SMC, extending out to $\sim$3.5 kpc in projection from its optical center. Lifetime star formation histories (SFHs) fit to each pointing independently reveal an outside-in age gradient such that fields in the SMC outskirts are older on average. We measure radial gradients of the lookback time to form 90%, 75% and 50% of the cumulative stellar mass for the first time, finding $\delta$($\tau_{90}$, $\tau_{75}$, $\tau_{50}$)/$\delta$R = (0.61$^{+0.08}_{-0.07}$, 0.65$^{+0.09}_{-0.08}$, 0.82$^{+0.12}_{-0.16}$) Gyr/kpc assuming PARSEC evolutionary models and a commonly used elliptical geometry of the SMC, although our results are robust to these assumptions. The wing of the SMC deviates from this trend, forming 25\% of its cumulative mass over the most recent 3 Gyr due to a best-fit star formation rate that remains approximately constant. Our results are consistent with chemodynamical evidence of a tidally stripped SMC component in the foreground, and imply contributions to the observed SFH from multiple previous LMC-SMC interactions. We also compare our SMC SFH with results from a companion study of the LMC, finding that while the two galaxies present different internal, spatially resolved SFH trends, both the LMC and SMC have similar near-constant lifetime SFHs when viewed globally., Comment: ApJ in press. 40 pages, 18 figures, 5 tables including Appendices

The proximity of the Magellanic Clouds provides the opportunity to study interacting dwarf galaxies near a massive host, and spatial trends in their stellar population properties in particular, with a unique level of detail. The Scylla pure parallel program has obtained deep (80% complete to >1 mag below the ancient main sequence turnoff), homogeneous two-filter Hubble Space Telescope (HST) imaging sampling the inner star-forming disk of the Large Magellanic Cloud (LMC), the perfect complement to shallower, contiguous ground-based surveys. We harness this imaging together with extant archival data and fit lifetime star formation histories (SFHs) to resolved color-magnitude diagrams (CMDs) of 111 individual fields, using three different stellar evolutionary libraries. We validate per-field recovered distances and extinctions as well as the combined global LMC age-metallicity relation and SFH against independent estimates. We find that the present-day radial age gradient reverses from an inside-out gradient in the inner disk to an outside-in gradient beyond $\sim$2 disk scalelengths, supported by ground-based measurements. The gradients become relatively flatter at earlier lookback times, while the location of the inversion remains constant over an order of magnitude in lookback time, from $\sim$1$-$10 Gyr. This suggests at least one mechanism that predates the recent intense LMC-SMC interaction. We compare observed radial age trends to other late-type galaxies at fixed stellar mass and discuss similarities and differences in the context of potential drivers, implying strong radial migration in the LMC., Comment: ApJ in press. 45 pages, 17 figures, 9 tables including Appendices

Scylla is a deep Hubble Space Telescope survey of the stellar populations, interstellar medium and star formation in the LMC and SMC. As a pure-parallel complement to the Ultraviolet Legacy Library of Young Stars as Essential Standards (ULLYSES) survey, Scylla obtained 342 orbits of ultraviolet (UV) through near-infrared (IR) imaging of the LMC and SMC with Wide Field Camera 3. In this paper, we describe the science objectives, observing strategy, data reduction procedure, and initial results from our photometric analysis of 96 observed fields. Although our observations were constrained by ULYSSES primary exposures, we imaged all fields in at least two filters (F475W and F814W), and 64% of fields in at least three and as many as seven WFC3 filters spanning the UV to IR. Overall, we reach average 50% completeness of $m_{\rm F225W}=26.0$, $m_{\rm F275W}=26.2$, $m_{\rm F336W}=26.9$, $m_{\rm F475W}=27.8$, $m_{\rm F814W}=25.5$, $m_{\rm F110W}=24.7$, and $m_{\rm F160W}=24.0$ Vega magnitudes in our photometric catalogs, which is faintward of the ancient main sequence turnoff in all filters. The primary science goals of Scylla include characterizing the structure and properties of dust in the MCs, as well as their spatially-resolved star formation and chemical enrichment histories. Our images and photometric catalogs, which represent the widest-area coverage of MCs with HST photometry to date, are available as a high-level science product at the Barbara A. Mikulski Archive for Space Telescopes., Comment: 25 pages, 16 figures, 8 tables. Accepted for publication in ApJS

Neural networks are powered by an implicit bias: a tendency of gradient descent to fit training data in a way that generalizes to unseen data. A recent class of neural network models gaining increasing popularity is structured state space models (SSMs), regarded as an efficient alternative to transformers. Prior work argued that the implicit bias of SSMs leads to generalization in a setting where data is generated by a low dimensional teacher. In this paper, we revisit the latter setting, and formally establish a phenomenon entirely undetected by prior work on the implicit bias of SSMs. Namely, we prove that while implicit bias leads to generalization under many choices of training data, there exist special examples whose inclusion in training completely distorts the implicit bias, to a point where generalization fails. This failure occurs despite the special training examples being labeled by the teacher, i.e. having clean labels! We empirically demonstrate the phenomenon, with SSMs trained independently and as part of non-linear neural networks. In the area of adversarial machine learning, disrupting generalization with cleanly labeled training examples is known as clean-label poisoning. Given the proliferation of SSMs, particularly in large language models, we believe significant efforts should be invested in further delineating their susceptibility to clean-label poisoning, and in developing methods for overcoming this susceptibility.

Mathematical reasoning remains a significant challenge for large language models (LLMs), despite progress in prompting techniques such as Chain-of-Thought (CoT). We present Chain of Mathematically Annotated Thought (CoMAT), which enhances reasoning through two stages: Symbolic Conversion (converting natural language queries into symbolic form) and Reasoning Execution (deriving answers from symbolic representations). CoMAT operates entirely with a single LLM and without external solvers. Across four LLMs, CoMAT outperforms traditional CoT on six out of seven benchmarks, achieving gains of 4.48% on MMLU-Redux (MATH) and 4.58% on GaoKao MCQ. In addition to improved performance, CoMAT ensures faithfulness and verifiability, offering a transparent reasoning process for complex mathematical tasks, Comment: 8 pages, 12 figures

The emergence of neural representations has revolutionized our means for digitally viewing a wide range of 3D scenes, enabling the synthesis of photorealistic images rendered from novel views. Recently, several techniques have been proposed for connecting these low-level representations with the high-level semantics understanding embodied within the scene. These methods elevate the rich semantic understanding from 2D imagery to 3D representations, distilling high-dimensional spatial features onto 3D space. In our work, we are interested in connecting language with a dynamic modeling of the world. We show how to lift spatio-temporal features to a 4D representation based on 3D Gaussian Splatting. This enables an interactive interface where the user can spatiotemporally localize events in the video from text prompts. We demonstrate our system on public 3D video datasets of people and animals performing various actions., Comment: Project webpage: https://tau-vailab.github.io/4-LEGS/

The diffusion of financial news into market prices is a complex process, making it challenging to evaluate the connections between news events and market movements. This paper introduces FININ (Financial Interconnected News Influence Network), a novel market prediction model that captures not only the links between news and prices but also the interactions among news items themselves. FININ effectively integrates multi-modal information from both market data and news articles. We conduct extensive experiments on two datasets, encompassing the S&P 500 and NASDAQ 100 indices over a 15-year period and over 2.7 million news articles. The results demonstrate FININ's effectiveness, outperforming advanced market prediction models with an improvement of 0.429 and 0.341 in the daily Sharpe ratio for the two markets respectively. Moreover, our results reveal insights into the financial news, including the delayed market pricing of news, the long memory effect of news, and the limitations of financial sentiment analysis in fully extracting predictive power from news data., Comment: Accepted by EMNLP 2024

Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks. To address this concern, we introduce PoisonBench, a benchmark for evaluating large language models' susceptibility to data poisoning during preference learning. Data poisoning attacks can manipulate large language model responses to include hidden malicious content or biases, potentially causing the model to generate harmful or unintended outputs while appearing to function normally. We deploy two distinct attack types across eight realistic scenarios, assessing 21 widely-used models. Our findings reveal concerning trends: (1) Scaling up parameter size does not inherently enhance resilience against poisoning attacks; (2) There exists a log-linear relationship between the effects of the attack and the data poison ratio; (3) The effect of data poisoning can generalize to extrapolated triggers that are not included in the poisoned data. These results expose weaknesses in current preference learning techniques, highlighting the urgent need for more robust defenses against malicious models and data manipulation., Comment: Tingchen Fu and Fazl Barez are core research contributors

Prompting techniques such as chain-of-thought have established themselves as a popular vehicle for improving the outputs of large language models (LLMs). For code generation, however, their exact mechanics and efficacy are under-explored. We thus investigate the effects of a wide range of prompting strategies with a focus on automatic re-prompting over multiple turns and computational requirements. After systematically decomposing reasoning, instruction, and execution feedback prompts, we conduct an extensive grid search on the competitive programming benchmarks CodeContests and TACO for multiple LLM families and sizes (Llama 3.0 and 3.1, 8B, 70B, 405B, and GPT-4o). Our study reveals strategies that consistently improve performance across all models with small and large sampling budgets. We then show how finetuning with such an optimal configuration allows models to internalize the induced reasoning process and obtain improvements in performance and scalability for multi-turn code generation.

We present a quantum network that distributes entangled photons between the University of Illinois Urbana-Champaign and a public library in Urbana. The network allows members of the public to perform measurements on the photons. We describe its design and implementation and outreach based on the network. Over 400 instances of public interaction have been logged with the system since it was launched in November 2023., Comment: 7 Pages, 8 Figures

In reinforcement learning, if the agent's reward differs from the designers' true utility, even only rarely, the state distribution resulting from the agent's policy can be very bad, in theory and in practice. When RL policies would devolve into undesired behavior, a common countermeasure is KL regularization to a trusted policy ("Don't do anything I wouldn't do"). All current cutting-edge language models are RL agents that are KL-regularized to a "base policy" that is purely predictive. Unfortunately, we demonstrate that when this base policy is a Bayesian predictive model of a trusted policy, the KL constraint is no longer reliable for controlling the behavior of an advanced RL agent. We demonstrate this theoretically using algorithmic information theory, and while systems today are too weak to exhibit this theorized failure precisely, we RL-finetune a language model and find evidence that our formal results are plausibly relevant in practice. We also propose a theoretical alternative that avoids this problem by replacing the "Don't do anything I wouldn't do" principle with "Don't do anything I mightn't do"., Comment: 10 pages, 7 page appendix, 4 figures

QUIC, a new and increasingly used transport protocol, enhances TCP by providing better security, performance, and features like stream multiplexing. These features, however, also impose challenges for network middle-boxes that need to monitor and analyze web traffic. This paper proposes a novel solution for estimating the number of HTTP/3 responses in a given QUIC connection by an observer. This estimation reveals server behavior, client-server interactions, and data transmission efficiency, which is crucial for various applications such as designing a load balancing solution and detecting HTTP/3 flood attacks. The proposed scheme transforms QUIC connection traces into a sequence of images and trains machine learning (ML) models to predict the number of responses. Then, by aggregating images of a QUIC connection, an observer can estimate the total number of responses. As the problem is formulated as a discrete regression problem, we introduce a dedicated loss function. The proposed scheme is evaluated on a dataset of over seven million images, generated from $100,000$ traces collected from over $44,000$ websites over a four-month period, from various vantage points. The scheme achieves up to 97\% cumulative accuracy in both known and unknown web server settings and 92\% accuracy in estimating the total number of responses in unseen QUIC traces., Comment: arXiv admin note: substantial text overlap with arXiv:2410.03728

The yields of hadrons and light nuclei in relativistic collisions of heavy-nuclei at a center of mass energy of 2.6 TeV can be described remarkably well by a thermal distribution of an ideal gas of hadrons and light nuclei interacting only via the decay of resonances. Given the particularly small binding energy of hypertritons relative to the temperature describing the yields (about 156 MeV), one might naturally expect hypertrions to dissociate in medium, making the agreement of hypertriton yields with thermal predictions highly puzzling. The puzzle is compounded by the fact that small binding energy is associated with the large size of the hypertriton. This size is on a similar scale to the overall size of the fireball and much larger than the length scale over which temperatures in the fireball vary over phenomenologically relevant amounts. This paper quantifies the tension this effect causes and shows that it is sufficiently large to render the thermal model inconsistent: its natural assumptions are in conflict with its outputs. The possibility that hypertritons are formed at freeze out as compact objects, quark droplets, that subsequently evolve into hypertritons is considered as a way to resolve the puzzle. It is noted that beyond making the assumption that compact quark droplets form, additional detailed dynamical assumptions which have not been justified are needed to make the thermal model work. The issue of why, despite these issues, the hypertriton is well described by a simple statistical description at freeze out is unresolved. Resolving the hypertriton puzzle is important as it may clarify whether the phenomenological success of the simple thermal model for yields accurately reflects the simple picture of the underlying physics on which it is based., Comment: 21 pages

Dynamic models hold great potential for research and development in signal processing, machine learning, and digital twin algorithms for diagnosing rotating machinery. Various studies have suggested dynamic models of gears, employing many model approaches. However, there is currently a lack of a computationally efficient and publicly accessible model that accurately represents real-world data. In this study, we propose a novel hybrid model that integrates a realistic and efficiently validated dynamic model of spur gear vibrations with an enhancement process aimed at bridging the gap between simulation and reality. This process minimizes discrepancies between features extracted from simulated and measured data through fine-tuning of the model hyperparameters. The effectiveness of this hybrid model is demonstrated across numerous test apparatuses, encompassing several types of faults, severities, and speeds. The new hybrid model, inclusive of an upgraded dynamic model, generates data swiftly within seconds and is made publicly available with a user-friendly application programming interface and a detailed user manual. The novel suggested hybrid model has great potential to enhance future research in model-based studies, including machine learning, signal processing, and digital twin approaches. The user manual can be found in the following link: https://github.com/PHM-BGU/public_dynamic_model_for_gear_vibrations