Search

Existing automated essay scoring (AES) has solely relied on essay text without using explanatory rationales for the scores, thereby forgoing an opportunity to capture the specific aspects evaluated by rubric indicators in a fine-grained manner. This paper introduces Rationale-based Multiple Trait Scoring (RMTS), a novel approach for multi-trait essay scoring that integrates prompt-engineering-based large language models (LLMs) with a fine-tuning-based essay scoring model using a smaller large language model (S-LLM). RMTS uses an LLM-based trait-wise rationale generation system where a separate LLM agent generates trait-specific rationales based on rubric guidelines, which the scoring model uses to accurately predict multi-trait scores. Extensive experiments on benchmark datasets, including ASAP, ASAP++, and Feedback Prize, show that RMTS significantly outperforms state-of-the-art models and vanilla S-LLMs in trait-specific scoring. By assisting quantitative assessment with fine-grained qualitative rationales, RMTS enhances the trait-wise reliability, providing partial explanations about essays.

Learning effective visual representations is crucial in open-world environments where agents encounter diverse and unstructured observations. This ability enables agents to extract meaningful information from raw sensory inputs, like pixels, which is essential for generalization across different tasks. However, evaluating representation learning separately from policy learning remains a challenge in most reinforcement learning (RL) benchmarks. To address this, we introduce the Sliding Puzzles Gym (SPGym), a benchmark that extends the classic 15-tile puzzle with variable grid sizes and observation spaces, including large real-world image datasets. SPGym allows scaling the representation learning challenge while keeping the latent environment dynamics and algorithmic problem fixed, providing a targeted assessment of agents' ability to form compositional and generalizable state representations. Experiments with both model-free and model-based RL algorithms, with and without explicit representation learning components, show that as the representation challenge scales, SPGym effectively distinguishes agents based on their capabilities. Moreover, SPGym reaches difficulty levels where no tested algorithm consistently excels, highlighting key challenges and opportunities for advancing representation learning for decision-making research.

We deploy the new Arkenstone galactic wind model in cosmological simulations for the first time, allowing us to robustly resolve the evolution and impact of high specific energy winds. In a (25$\,h^{-1}\,$Mpc)$^3$ box we perform a set of numerical experiments that systematically vary the mass and energy loadings of such winds, finding that their energy content is the key parameter controlling the stellar to dark matter mass ratio. Increasing the mass loading, at fixed energy, actually results in mildly enhanced star formation, counter to prevailing wisdom but in agreement with recent analytic models. Of the simple parametrisations that we test, we find that an energy loading that scales inversely with halo mass best matches a wide range of observations, and can do so with mass loadings drastically lower than those in most previous cosmological simulations. In this scenario, much less material is ejected from the interstellar medium. Instead, winds both heat gas in the circumgalactic medium, slowing infall onto the galaxy, and also drive shocks beyond the virial radius, preventing accretion onto the halo in the first place. We have not yet tied the mass and energy loadings to high-resolution simulations (a key goal of the Learning the Universe collaboration); however, we can already report that a much lower fraction of the available supernova energy is needed in preventative galaxy regulation than required by ejective wind feedback models such as IllustrisTNG., Comment: Submitted to MNRAS. 19 pages, 14 figures. This is a Learning the Universe publication

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.

Exposure measurement error is a ubiquitous but often overlooked challenge in causal inference with observational data. Existing methods accounting for exposure measurement error largely rely on restrictive parametric assumptions, while emerging data-adaptive estimation approaches allow for less restrictive assumptions but at the cost of flexibility, as they are typically tailored towards rigidly-defined statistical quantities. There remains a critical need for assumption-lean estimation methods that are both flexible and possess desirable theoretical properties across a variety of study designs. In this paper, we introduce a general framework for estimation of causal quantities in the presence of exposure measurement error, adapted from the control variates approach of Yang and Ding (2019). Our method can be implemented in various two-phase sampling study designs, where one obtains gold-standard exposure measurements for a small subset of the full study sample, called the validation data. The control variates framework leverages both the error-prone and error-free exposure measurements by augmenting an initial consistent estimator from the validation data with a variance reduction term formed from the full data. We show that our method inherits double-robustness properties under standard causal assumptions. Simulation studies show that our approach performs favorably compared to leading methods under various two-phase sampling schemes. We illustrate our method with observational electronic health record data on HIV outcomes from the Vanderbilt Comprehensive Care Clinic., Comment: 42 pages, 6 figures

As Artificial Intelligence models, such as Large Video-Language models (VLMs), grow in size, their deployment in real-world applications becomes increasingly challenging due to hardware limitations and computational costs. To address this, we design a hybrid edge-cloud solution that leverages the efficiency of smaller models for local processing while deferring to larger, more accurate cloud-based models when necessary. Specifically, we propose a novel unsupervised data generation method, Dual-Model Distillation (DMD), to train a lightweight switcher model that can predict when the edge model's output is uncertain and selectively offload inference to the large model in the cloud. Experimental results on the action classification task show that our framework not only requires less computational overhead, but also improves accuracy compared to using a large model alone. Our framework provides a scalable and adaptable solution for action classification in resource-constrained environments, with potential applications beyond healthcare. Noteworthy, while DMD-generated data is used for optimizing performance and resource usage in our pipeline, we expect the concept of DMD to further support future research on knowledge alignment across multiple models.

Large Language Models (LLMs) have made significant strides in text generation and comprehension, with recent advancements extending into multimodal LLMs that integrate visual and audio inputs. However, these models continue to struggle with fine-grained, cross-modal temporal understanding, particularly when correlating events across audio and video streams. We address these challenges with two key contributions: a new dataset and model, called OCTAV and OMCAT respectively. OCTAV (Omni Context and Temporal Audio Video) is a novel dataset designed to capture event transitions across audio and video. Second, OMCAT (Omni Context Aware Transformer) is a powerful model that leverages RoTE (Rotary Time Embeddings), an innovative extension of RoPE, to enhance temporal grounding and computational efficiency in time-anchored tasks. Through a robust three-stage training pipeline-feature alignment, instruction tuning, and OCTAV-specific training-OMCAT excels in cross-modal temporal understanding. Our model demonstrates state-of-the-art performance on Audio-Visual Question Answering (AVQA) tasks and the OCTAV benchmark, showcasing significant gains in temporal reasoning and cross-modal alignment, as validated through comprehensive experiments and ablation studies. Our dataset and code will be made publicly available. The link to our demo page is https://om-cat.github.io., Comment: Demo page: https://om-cat.github.io

The utility of synthetic data to enhance pretraining data quality and hence to improve downstream task accuracy has been widely explored in recent large language models (LLMs). Yet, these approaches fall inadequate in complex, multi-hop and mathematical reasoning tasks as the synthetic data typically fails to add complementary knowledge to the existing raw corpus. In this work, we propose a novel large-scale and diverse Math Informed syNthetic Dialogue (MIND) generation method that improves the mathematical reasoning ability of LLMs. Specifically, using MIND, we generate synthetic conversations based on OpenWebMath (OWM), resulting in a new math corpus, MIND-OWM. Our experiments with different conversational settings reveal that incorporating knowledge gaps between dialog participants is essential for generating high-quality math data. We further identify an effective way to format and integrate synthetic and raw data during pretraining to maximize the gain in mathematical reasoning, emphasizing the need to restructure raw data rather than use it as-is. Compared to pretraining just on raw data, a model pretrained on MIND-OWM shows significant boost in mathematical reasoning (GSM8K: +13.42%, MATH: +2.30%), including superior performance in specialized knowledge (MMLU: +4.55%, MMLU-STEM: +4.28%) and general purpose reasoning tasks (GENERAL REASONING: +2.51%)., Comment: 31 pages, 5 figures, 14 tables

This paper presents a comprehensive approach to nonlinear dynamics identification for UAVs using a combination of data-driven techniques and theoretical modeling. Two key methodologies are explored: Proportional-Derivative (PD) approximation and Sparse Identification of Nonlinear Dynamics (SINDy). The UAV dynamics are first modeled using the Euler-Lagrange formulation, providing a set of generalized coordinates. However, platform constraints limit the control inputs to attitude angles, and linear and angular velocities along the z-axis. To accommodate these limitations, thrust and torque inputs are approximated using a PD controller, serving as the foundation for nonlinear system identification. In parallel, SINDy, a data-driven method, is employed to derive a compact and interpretable model of the UAV dynamics from experimental data. Both identified models are then integrated into a Model Predictive Control (MPC) framework for accurate trajectory tracking, where model accuracy, informed by data-driven insights, plays a critical role in optimizing control performance. This fusion of data-driven approaches and theoretical modeling enhances the system's robustness and adaptability in real-world conditions, offering a detailed analysis of the UAV's dynamic behavior.

This paper presents the findings from the third edition of the Chat Translation Shared Task. As with previous editions, the task involved translating bilingual customer support conversations, specifically focusing on the impact of conversation context in translation quality and evaluation. We also include two new language pairs: English-Korean and English-Dutch, in addition to the set of language pairs from previous editions: English-German, English-French, and English-Brazilian Portuguese. We received 22 primary submissions and 32 contrastive submissions from eight teams, with each language pair having participation from at least three teams. We evaluated the systems comprehensively using both automatic metrics and human judgments via a direct assessment framework. The official rankings for each language pair were determined based on human evaluation scores, considering performance in both translation directions--agent and customer. Our analysis shows that while the systems excelled at translating individual turns, there is room for improvement in overall conversation-level translation quality., Comment: 12 pages, 5 figures, 13 tables

Edge computing has emerged as an alternative to reduce transmission and processing delay and preserve privacy of the video streams. However, the ever-increasing complexity of Deep Neural Networks (DNNs) used in video-based applications (e.g. object detection) exerts pressure on memory-constrained edge devices. Model merging is proposed to reduce the DNNs' memory footprint by keeping only one copy of merged layers' weights in memory. In existing model merging techniques, (i) only architecturally identical layers can be shared; (ii) requires computationally expensive retraining in the cloud; (iii) assumes the availability of ground truth for retraining. The re-evaluation of a merged model's performance, however, requires a validation dataset with ground truth, typically runs at the cloud. Common metrics to guide the selection of shared layers include the size or computational cost of shared layers or representation size. We propose a new model merging scheme by sharing representations (i.e., outputs of layers) at the edge, guided by representation similarity S. We show that S is extremely highly correlated with merged model's accuracy with Pearson Correlation Coefficient |r| > 0.94 than other metrics, demonstrating that representation similarity can serve as a strong validation accuracy indicator without ground truth. We present our preliminary results of the newly proposed model merging scheme with identified challenges, demonstrating a promising research future direction., Comment: 3 pages, 4 figures, ACM MobiCom '24, November 18-22, 2024, Washington D.C., DC, USA

Randomized controlled trials (RCTs) can be used to generate guarantees on treatment effects. However, RCTs often spend unnecessary resources exploring sub-optimal treatments, which can reduce the power of treatment guarantees. To address these concerns, we develop a two-stage RCT where, first on a data-driven screening stage, we prune low-impact treatments, while in the second stage, we develop high probability lower bounds on the treatment effect. Unlike existing adaptive RCT frameworks, our method is simple enough to be implemented in scenarios with limited adaptivity. We derive optimal designs for two-stage RCTs and demonstrate how we can implement such designs through sample splitting. Empirically, we demonstrate that two-stage designs improve upon single-stage approaches, especially in scenarios where domain knowledge is available in the form of a prior. Our work is thus, a simple, yet effective, method to estimate high probablility certificates for high performant treatment effects on a RCT.

Neural codecs have demonstrated strong performance in high-fidelity compression of audio signals at low bitrates. The token-based representations produced by these codecs have proven particularly useful for generative modeling. While much research has focused on improvements in compression ratio and perceptual transparency, recent works have largely overlooked another desirable codec property -- idempotence, the stability of compressed outputs under multiple rounds of encoding. We find that state-of-the-art neural codecs exhibit varied degrees of idempotence, with some degrading audio outputs significantly after as few as three encodings. We investigate possible causes of low idempotence and devise a method for improving idempotence through fine-tuning a codec model. We then examine the effect of idempotence on a simple conditional generative modeling task, and find that increased idempotence can be achieved without negatively impacting downstream modeling performance -- potentially extending the usefulness of neural codecs for practical file compression and iterative generative modeling workflows., Comment: Submitted to ICASSP 2025

Knowledge Tracing (KT) is vital in educational data mining, enabling personalized learning by tracking learners' knowledge states and forecasting their academic outcomes. This study introduces the LOKT (Large Language Model Option-weighted Knowledge Tracing) model to address the cold start problem where limited historical data available using large language models (LLMs). While traditional KT models have incorporated option weights, our research extends this by integrating these weights into an LLM-based KT framework. Moving beyond the binary classification of correct and incorrect responses, we emphasize that different types of incorrect answers offer valuable insights into a learner's knowledge state. By converting these responses into text-based ordinal categories, we enable LLMs to assess learner understanding with greater clarity, although our approach focuses on the final knowledge state rather than the progression of learning over time. Using five public datasets, we demonstrate that the LOKT model sustains high predictive accuracy even with limited data, effectively addressing both "learner cold-start" and "system cold-start" scenarios. These findings showcase LOKT's potential to enhance LLM-based learning tools and support early-stage personalization., Comment: 11 pages

Echocardiography is the most widely used cardiac imaging modality, capturing ultrasound video data to assess cardiac structure and function. Artificial intelligence (AI) in echocardiography has the potential to streamline manual tasks and improve reproducibility and precision. However, most echocardiography AI models are single-view, single-task systems that do not synthesize complementary information from multiple views captured during a full exam, and thus lead to limited performance and scope of applications. To address this problem, we introduce EchoPrime, a multi-view, view-informed, video-based vision-language foundation model trained on over 12 million video-report pairs. EchoPrime uses contrastive learning to train a unified embedding model for all standard views in a comprehensive echocardiogram study with representation of both rare and common diseases and diagnoses. EchoPrime then utilizes view-classification and a view-informed anatomic attention model to weight video-specific interpretations that accurately maps the relationship between echocardiographic views and anatomical structures. With retrieval-augmented interpretation, EchoPrime integrates information from all echocardiogram videos in a comprehensive study and performs holistic comprehensive clinical echocardiography interpretation. In datasets from two independent healthcare systems, EchoPrime achieves state-of-the art performance on 23 diverse benchmarks of cardiac form and function, surpassing the performance of both task-specific approaches and prior foundation models. Following rigorous clinical evaluation, EchoPrime can assist physicians in the automated preliminary assessment of comprehensive echocardiography., Comment: 30 pages, 3 tables, 3 figures

Dynamic quantum circuits (DQCs) incorporate mid-circuit measurements and gates conditioned on these measurement outcomes. DQCs can prepare certain long-range entangled states in constant depth, making them a promising route to preparing complex quantum states on devices with a limited coherence time. Almost all constructions of DQCs for state preparation have been formulated analytically, relying on special structure in the target states. In this work, we approach the problem of state preparation variationally, developing scalable tensor network algorithms which find high-fidelity DQC preparations for generic states. We apply our algorithms to critical states, random matrix product states, and subset states. We consider both DQCs with a fixed number of ancillae and those with an extensive number of ancillae. Even in the few ancillae regime, the DQCs discovered by our algorithms consistently prepare states with lower infidelity than a static quantum circuit of the same depth. Notably, we observe constant fidelity gains across system sizes and circuit depths. For DQCs with an extensive number of ancillae, we introduce scalable methods for decoding measurement outcomes, including a neural network decoder and a real-time decoding protocol. Our work demonstrates the power of an algorithmic approach to generating DQC circuits, broadening their scope of applications to new areas of quantum computing.

We present Mahi-Mahi, the first asynchronous BFT consensus protocol that achieves sub-second latency in the WAN while processing over 100,000 transactions per second. We accomplish this remarkable performance by building Mahi-Mahi on an uncertified structured Directed Acyclic Graph (DAG). By forgoing explicit certification, we significantly reduce the number of messages required to commit and minimize CPU overhead associated with certificate verification. Mahi-Mahi introduces a novel commit rule that allows committing multiple blocks in each DAG round, while ensuring liveness in the presence of an asynchronous adversary. Mahi-Mahi can be parametrized to either attempt to commit within 5 message delays, maximizing the probability of commitment under a continuously active asynchronous adversary, or within 4 message delays, which reduces latency under a more moderate and realistic asynchronous adversary. We demonstrate the safety and liveness of Mahi-Mahi in a Byzantine context. Subsequently, we evaluate Mahi-Mahi in a geo-replicated setting and compare its performance against state-of-the-art asynchronous consensus protocols, showcasing Mahi-Mahi's significantly lower latency.

Recent works integrating Knowledge Graphs (KGs) have led to promising improvements in enhancing reasoning accuracy of Large Language Models (LLMs). However, current benchmarks mainly focus on closed tasks, leaving a gap in the assessment of more complex, real-world scenarios. This gap has also obscured the evaluation of KGs' potential to mitigate the problem of hallucination in LLMs. To fill the gap, we introduce OKGQA, a new benchmark specifically designed to assess LLMs enhanced with KGs under open-ended, real-world question answering scenarios. OKGQA is designed to closely reflect the complexities of practical applications using questions from different types, and incorporates specific metrics to measure both the reduction in hallucinations and the enhancement in reasoning capabilities. To consider the scenario in which KGs may have varying levels of mistakes, we further propose another experiment setting OKGQA-P to assess model performance when the semantics and structure of KGs are deliberately perturbed and contaminated. OKGQA aims to (1) explore whether KGs can make LLMs more trustworthy in an open-ended setting, and (2) conduct a comparative analysis to shed light on methods and future directions for leveraging KGs to reduce LLMs' hallucination. We believe that this study can facilitate a more complete performance comparison and encourage continuous improvement in integrating KGs with LLMs., Comment: Work in progress

Federated Domain Adaptation (FDA) is a Federated Learning (FL) scenario where models are trained across multiple clients with unique data domains but a shared category space, without transmitting private data. The primary challenge in FDA is data heterogeneity, which causes significant divergences in gradient updates when using conventional averaging-based aggregation methods, reducing the efficacy of the global model. This further undermines both in-domain and out-of-domain performance (within the same federated system but outside the local client). To address this, we propose a novel framework called \textbf{M}ulti-domain \textbf{P}rototype-based \textbf{F}ederated Fine-\textbf{T}uning (MPFT). MPFT fine-tunes a pre-trained model using multi-domain prototypes, i.e., pretrained representations enriched with domain-specific information from category-specific local data. This enables supervised learning on the server to derive a globally optimized adapter that is subsequently distributed to local clients, without the intrusion of data privacy. Empirical results show that MPFT significantly improves both in-domain and out-of-domain accuracy over conventional methods, enhancing knowledge preservation and adaptation in FDA. Notably, MPFT achieves convergence within a single communication round, greatly reducing computation and communication costs. To ensure privacy, MPFT applies differential privacy to protect the prototypes. Additionally, we develop a prototype-based feature space hijacking attack to evaluate robustness, confirming that raw data samples remain unrecoverable even after extensive training epochs. The complete implementation of MPFL is available at \url{https://anonymous.4open.science/r/DomainFL/}.

One of the most challenging problems in the computational study of localization in quantum manybody systems is to capture the effects of rare events, which requires sampling over exponentially many disorder realizations. We implement an efficient procedure on a quantum processor, leveraging quantum parallelism, to efficiently sample over all disorder realizations. We observe localization without disorder in quantum many-body dynamics in one and two dimensions: perturbations do not diffuse even though both the generator of evolution and the initial states are fully translationally invariant. The disorder strength as well as its density can be readily tuned using the initial state. Furthermore, we demonstrate the versatility of our platform by measuring Renyi entropies. Our method could also be extended to higher moments of the physical observables and disorder learning.

Upcycling pre-trained dense language models into sparse mixture-of-experts (MoE) models is an efficient approach to increase the model capacity of already trained models. However, optimal techniques for upcycling at scale remain unclear. In this work, we conduct an extensive study of upcycling methods and hyperparameters for billion-parameter scale language models. We propose a novel "virtual group" initialization scheme and weight scaling approach to enable upcycling into fine-grained MoE architectures. Through ablations, we find that upcycling outperforms continued dense model training. In addition, we show that softmax-then-topK expert routing improves over topK-then-softmax approach and higher granularity MoEs can help improve accuracy. Finally, we upcycled Nemotron-4 15B on 1T tokens and compared it to a continuously trained version of the same model on the same 1T tokens: the continuous trained model achieved 65.3% MMLU, whereas the upcycled model achieved 67.6%. Our results offer insights and best practices to effectively leverage upcycling for building MoE language models.

We study the effects of AGN feedback on the Lyman-$\alpha$ forest 1D flux power spectrum (P1D). Using the Simba cosmological-hydrodynamic simulations, we examine the impact that adding different AGN feedback modes has on the predicted P1D. We find that, for Simba, the impact of AGN feedback is most dramatic at lower redshifts ($z<1$) and that AGN jet feedback plays the most significant role in altering the P1D. The effects of AGN feedback can be seen across a large range of wavenumbers ($1.5\times10^{-3}2.0$), AGN feedback has a $2\%$ effect on the P1D for $k<5\times10^{-2}$ s/km and an $8\%$ effect for $k>5\times10^{-2}$ s/km. We show that the small scale effect is reduced when normalizing the simulation to the observed mean flux. On large scales, the effect of AGN feedback appears via a change in the IGM temperature and is thus unlikely to bias cosmological parameters. The strong AGN jets in the Simba simulation can reproduce the $z>2$ Lyman-$\alpha$ forest. We stress that analyses comparing different AGN feedback models to future higher precision data will be necessary to determine the full extent of this effect., Comment: 17 pages, 5 figures, submitted to ApJ

Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration for score-based diffusion transformers via reducing both sampling steps and cost per step. To reduce steps, we develop a new score-based distribution matching distillation (DMD) method for the EDM-family of diffusion models, the first GAN-based distillation method for TTM. To reduce the cost per step, we develop a simple, but powerful improvement to a recent layer distillation method that improves learning via better preserving hidden state variance. Finally, we combine our step and layer distillation methods together for a dual-faceted approach. We evaluate our step and layer distillation methods independently and show each yield best-in-class performance. Our combined distillation method can generate high-quality outputs with improved diversity, accelerating our base model by 10-18x (230/435ms latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) -- the fastest high-quality TTM to our knowledge. Sound examples can be found at https://presto-music.github.io/web/.

Data valuation is a class of techniques for quantitatively assessing the value of data for applications like pricing in data marketplaces. Existing data valuation methods define a value for a discrete dataset. However, in many use cases, users are interested in not only the value of the dataset, but that of the distribution from which the dataset was sampled. For example, consider a buyer trying to evaluate whether to purchase data from different vendors. The buyer may observe (and compare) only a small preview sample from each vendor, to decide which vendor's data distribution is most useful to the buyer and purchase. The core question is how should we compare the values of data distributions from their samples? Under a Huber characterization of the data heterogeneity across vendors, we propose a maximum mean discrepancy (MMD)-based valuation method which enables theoretically principled and actionable policies for comparing data distributions from samples. We empirically demonstrate that our method is sample-efficient and effective in identifying valuable data distributions against several existing baselines, on multiple real-world datasets (e.g., network intrusion detection, credit card fraud detection) and downstream applications (classification, regression)., Comment: Accepted to NeurIPS 2024 as a poster. Main paper with appendix (38 pages in total). Code will be released soon at https://github.com/XinyiYS/Data_Distribution_Valuation

Text-to-image (T2I) diffusion models have revolutionized visual content creation, but extending these capabilities to text-to-video (T2V) generation remains a challenge, particularly in preserving temporal consistency. Existing methods that aim to improve consistency often cause trade-offs such as reduced imaging quality and impractical computational time. To address these issues we introduce VideoGuide, a novel framework that enhances the temporal consistency of pretrained T2V models without the need for additional training or fine-tuning. Instead, VideoGuide leverages any pretrained video diffusion model (VDM) or itself as a guide during the early stages of inference, improving temporal quality by interpolating the guiding model's denoised samples into the sampling model's denoising process. The proposed method brings about significant improvement in temporal consistency and image fidelity, providing a cost-effective and practical solution that synergizes the strengths of various video diffusion models. Furthermore, we demonstrate prior distillation, revealing that base models can achieve enhanced text coherence by utilizing the superior data prior of the guiding model through the proposed method. Project Page: https://dohunlee1.github.io/videoguide.github.io/, Comment: 24 pages, 14 figures, Project Page: https://dohunlee1.github.io/videoguide.github.io/

We constrain the angular momentum architecture of VHS J125601.92-125723.9, a 140 $\pm$ 20 Myr old hierarchical triple system composed of a low-mass binary and a widely-separated planetary-mass companion VHS 1256 b. VHS 1256 b has been a prime target for multiple characterization efforts, revealing the highest measured substellar photometric variability to date and the presence of silicate clouds and disequilibrium chemistry. Here we add a key piece to the characterization of this super-Jupiter on a Tatooine-like orbit; we measure its spin-axis tilt relative to its orbit, i.e. the obliquity of VHS 1256 b. We accomplish this by combining three measurements. We find a projected rotation rate $v \sin{i_p} = 8.7 \pm 0.1 \,\mathrm{km~s^{-1}}$ for VHS 1256 b using near-IR high-resolution spectra from Gemini/IGRINS. Combining this with a published photometric rotation period indicates that the companion is viewed edge-on, with a line-of-sight spin axis inclination of $i_{\rm p} = 90^\circ \pm 18^\circ$. We refit available astrometry measurements to confirm an orbital inclination of $i_{\rm o} = 23 \substack{+10 \\ -13}^\circ$. Taken together, VHS 1256 b has a large planetary obliquity of $\psi = 90^\circ \pm 25^\circ$. In total, we have three measured angular momentum vectors for the system: the binary orbit normal, companion orbit normal, and companion spin axis. All three are misaligned with respect to each other. Although VHS 1256 b is tilted like Uranus, their origins are distinct. We rule out planet-like scenarios including collisions and spin-orbit resonances, and suggest that top-down formation via core/filament fragmentation is promising., Comment: accepted to AJ. 16 pages, 9 figures

A fundamental challenge in developing data-driven approaches to ecological systems for tasks such as state estimation and prediction is the paucity of the observational or measurement data. For example, modern machine-learning techniques such as deep learning or reservoir computing typically require a large quantity of data. Leveraging synthetic data from paradigmatic nonlinear but non-ecological dynamical systems, we develop a meta-learning framework with time-delayed feedforward neural networks to predict the long-term behaviors of ecological systems as characterized by their attractors. We show that the framework is capable of accurately reconstructing the ``dynamical climate'' of the ecological system with limited data. Three benchmark population models in ecology, namely the Hastings-Powell model, a three-species food chain, and the Lotka-Volterra system, are used to demonstrate the performance of the meta-learning based prediction framework. In all cases, enhanced accuracy and robustness are achieved using five to seven times less training data as compared with the corresponding machine-learning method trained solely from the ecosystem data. A number of issues affecting the prediction performance are addressed., Comment: 16 pages, 13 figures

This paper proposes a simple yet effective jailbreak attack named FlipAttack against black-box LLMs. First, from the autoregressive nature, we reveal that LLMs tend to understand the text from left to right and find that they struggle to comprehend the text when noise is added to the left side. Motivated by these insights, we propose to disguise the harmful prompt by constructing left-side noise merely based on the prompt itself, then generalize this idea to 4 flipping modes. Second, we verify the strong ability of LLMs to perform the text-flipping task, and then develop 4 variants to guide LLMs to denoise, understand, and execute harmful behaviors accurately. These designs keep FlipAttack universal, stealthy, and simple, allowing it to jailbreak black-box LLMs within only 1 query. Experiments on 8 LLMs demonstrate the superiority of FlipAttack. Remarkably, it achieves $\sim$98\% attack success rate on GPT-4o, and $\sim$98\% bypass rate against 5 guardrail models on average. The codes are available at GitHub\footnote{https://github.com/yueliu1999/FlipAttack}., Comment: 43 pages, 31 figures

There is increasing interest in ''decision-focused'' machine learning methods which train models to account for how their predictions are used in downstream optimization problems. Doing so can often improve performance on subsequent decision problems. However, current methods for uncertainty quantification do not incorporate any information at all about downstream decisions. We develop a framework based on conformal prediction to produce prediction sets that account for a downstream decision loss function, making them more appropriate to inform high-stakes decision-making. Our approach harnesses the strengths of conformal methods--modularity, model-agnosticism, and statistical coverage guarantees--while incorporating downstream decisions and user-specified utility functions. We prove that our methods retain standard coverage guarantees. Empirical evaluation across a range of datasets and utility metrics demonstrates that our methods achieve significantly lower decision loss compared to standard conformal methods. Additionally, we present a real-world use case in healthcare diagnosis, where our method effectively incorporates the hierarchical structure of dermatological diseases. It successfully generates sets with coherent diagnostic meaning, aiding the triage process during dermatology diagnosis and illustrating how our method can ground high-stakes decision-making on external domain knowledge.

Various visual foundation models have distinct strengths and weaknesses, both of which can be improved through heterogeneous multi-teacher knowledge distillation without labels, termed "agglomerative models." We build upon this body of work by studying the effect of the teachers' activation statistics, particularly the impact of the loss function on the resulting student model quality. We explore a standard toolkit of statistical normalization techniques to better align the different distributions and assess their effects. Further, we examine the impact on downstream teacher-matching metrics, which motivates the use of Hadamard matrices. With these matrices, we demonstrate useful properties, showing how they can be used for isotropic standardization, where each dimension of a multivariate distribution is standardized using the same scale. We call this technique "PHI Standardization" (PHI-S) and empirically demonstrate that it produces the best student model across the suite of methods studied.

We present Synthio, a novel approach for augmenting small-scale audio classification datasets with synthetic data. Our goal is to improve audio classification accuracy with limited labeled data. Traditional data augmentation techniques, which apply artificial transformations (e.g., adding random noise or masking segments), struggle to create data that captures the true diversity present in real-world audios. To address this shortcoming, we propose to augment the dataset with synthetic audio generated from text-to-audio (T2A) diffusion models. However, synthesizing effective augmentations is challenging because not only should the generated data be acoustically consistent with the underlying small-scale dataset, but they should also have sufficient compositional diversity. To overcome the first challenge, we align the generations of the T2A model with the small-scale dataset using preference optimization. This ensures that the acoustic characteristics of the generated data remain consistent with the small-scale dataset. To address the second challenge, we propose a novel caption generation technique that leverages the reasoning capabilities of Large Language Models to (1) generate diverse and meaningful audio captions and (2) iteratively refine their quality. The generated captions are then used to prompt the aligned T2A model. We extensively evaluate Synthio on ten datasets and four simulated limited-data settings. Results indicate our method consistently outperforms all baselines by 0.1%-39% using a T2A model trained only on weakly-captioned AudioSet., Comment: Code and Checkpoints will be soon available here: https://github.com/Sreyan88/Synthio

Generative Artificial Intelligence (GenAI) is becoming ubiquitous in our daily lives. The increase in computational power and data availability has led to a proliferation of both single- and multi-modal models. As the GenAI ecosystem matures, the need for extensible and model-agnostic risk identification frameworks is growing. To meet this need, we introduce the Python Risk Identification Toolkit (PyRIT), an open-source framework designed to enhance red teaming efforts in GenAI systems. PyRIT is a model- and platform-agnostic tool that enables red teamers to probe for and identify novel harms, risks, and jailbreaks in multimodal generative AI models. Its composable architecture facilitates the reuse of core building blocks and allows for extensibility to future models and modalities. This paper details the challenges specific to red teaming generative AI systems, the development and features of PyRIT, and its practical applications in real-world scenarios.

Polycyclic aromatic hydrocarbons (PAHs) are among the most ubiquitous compounds in the universe, accounting for up to ~25% of all interstellar carbon. Since most unsubstituted PAHs do not possess permanent dipole moments, they are invisible to radio astronomy. Constraining their abundances relies on the detection of polar chemical proxies, such as aromatic nitriles. We report the detection of 2- and 4-cyanopyrene, isomers of the recently detected 1-cyanopyrene. We find that these isomers are present in an abundance ratio of ~2:1:2, which mirrors the number of equivalent sites available for CN addition. We conclude that there is evidence that the cyanopyrene isomers formed by direct CN addition to pyrene under kinetic control in hydrogen-rich gas at 10 K and discuss constraints on the H/CN ratio for PAHs in TMC-1., Comment: Submitted version to comply with licensing agreements

Polycyclic aromatic hydrocarbons (PAHs) are expected to be the most abundant class of organic molecules in space. Their interstellar lifecycle is not well understood, and progress is hampered by difficulties detecting individual PAH molecules. Here, we present the discovery of CN-functionalized pyrene, a 4-ring PAH, in the dense cloud TMC-1 using the 100-m Green Bank Telescope. We derive an abundance of 1-cyanopyrene of ~1.52 x $10^{12}$ cm$^{-2}$, and from this estimate that the un-substituted pyrene accounts for up to ~0.03-0.3% of the carbon budget in the dense interstellar medium which trace the birth sites of stars and planets. The presence of pyrene in this cold (~10 K) molecular cloud agrees with its recent measurement in asteroid Ryugu where isotopic clumping suggest a cold, interstellar origin. The direct link to the birth site of our solar system is strengthened when we consider the solid state pyrene content in the pre-stellar materials compared to comets, which represent the most pristine material in the solar system. We estimate that solid state pyrene can account for 1% of the carbon within comets carried by this one single organic molecule. The abundance indicates pyrene is an "island of stability" in interstellar PAH chemistry and suggests a potential cold molecular cloud origin of the carbon carried by PAHs that is supplied to forming planetary systems, including habitable worlds such as our own., Comment: Version of manuscript revised to comply with licensing requirements

Large language models excel at creative generation but continue to struggle with the issues of hallucination and bias. While retrieval-augmented generation (RAG) provides a framework for grounding LLMs' responses in accurate and up-to-date information, it still raises the question of bias: which sources should be selected for inclusion in the context? And how should their importance be weighted? In this paper, we study the challenge of cross-lingual RAG and present a dataset to investigate the robustness of existing systems at answering queries about geopolitical disputes, which exist at the intersection of linguistic, cultural, and political boundaries. Our dataset is sourced from Wikipedia pages containing information relevant to the given queries and we investigate the impact of including additional context, as well as the composition of this context in terms of language and source, on an LLM's response. Our results show that existing RAG systems continue to be challenged by cross-lingual use cases and suffer from a lack of consistency when they are provided with competing information in multiple languages. We present case studies to illustrate these issues and outline steps for future research to address these challenges. We make our dataset and code publicly available at https://github.com/manestay/bordIRlines., Comment: NLP for Wikipedia workshop at EMNLP 2024

We extend Smale's singular bridge principle (see \cite{NS2}) for $n$-dimensional strictly stable minimal cones in $\mathbb{R}^{n+1}$ $(n \geq 7$) to arbitrary codimension when $n \geq 4$. We then apply the procedure to copies of the Lawson-Osserman cone to produce a four dimensional minimal graph in $\mathbb{R}^7$ with any finite number of isolated singularities., Comment: 53 pages

We give a novel nonparametric pointwise consistent statistical test (the Markov Checker) of the Markov condition for directed acyclic graph (DAG) or completed partially directed acyclic graph (CPDAG) models given a dataset. We also introduce the Cross-Algorithm Frugality Search (CAFS) for rejecting DAG models that either do not pass the Markov Checker test or that are not edge minimal. Edge minimality has been used previously by Raskutti and Uhler as a nonparametric simplicity criterion, though CAFS readily generalizes to other simplicity conditions. Reference to the ground truth is not necessary for CAFS, so it is useful for finding causal structure learning algorithms and tuning parameter settings that output causal models that are approximately true from a given data set. We provide a software tool for this analysis that is suitable for even quite large or dense models, provided a suitably fast pointwise consistent test of conditional independence is available. In addition, we show in simulation that the CAFS procedure can pick approximately correct models without knowing the ground truth., Comment: 19 pages, 14 figures, 1 table

Extragalactic science and cosmology with Stage IV galaxy surveys will rely almost exclusively on redshift measurements derived solely from photometry, which are subject to systematic and statistical uncertainties with numerous analysis choices, such as that of an estimator and prior information and no universal solution. Single-survey photometric redshift estimates ought to be improved by combining data from multiple surveys, with common wisdom asserting that optical data benefits from additional infrared coverage but not from additional ultraviolet coverage. The degree of improvement for either case is not well-characterized and attempts to quantify it necessitate assumptions of a chosen estimator and its prior information. We apply an information theoretic metric of potentially recoverable redshift information to assess the impact of multi-survey photometry combinations without assuming an estimator nor priors in the context of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) in the optical, Roman and Euclid Surveys in the infrared, and Cosmological Advanced Survey Telescope for Optical-UV Research (CASTOR) in the ultraviolet. We conclude that the addition of UV photometry can benefit redshift determination of certain galaxy populations, but that gain is tempered by their decreased chance of meeting detection criteria at higher wavelengths. We explore the spectral energy distribution (SEDs) of galaxies whose potentially recoverable redshift information is most impacted by additional photometry to provide a basis for future evaluation of which science cases may motivate which combinations. The holistic assessment approach we develop here is generic and may be applied to quantify the impact of combining photometric datasets, changing experimental design choices, including observing strategy optimization, and evaluating systematics mitigation procedures., Comment: 13 pages, 8 figures, to be submitted to ApJ; comments welcome

In this paper, we investigate the internal (finite) sum of submonoids of rank-$1$ torsion-free abelian groups. These submonoids, when not groups, are isomorphic to nontrivial submonoids of the nonnegative cone of $\mathbb Q$, known as Puiseux monoids, and have been actively studied during the last few years. Here we study how the atomicity and arithmetic of Puiseux monoids behave under their internal (finite) sum inside the abelian group $\mathbb Q$. We study the factorization properties of such internal sums, giving priority to Cohn's notion of atomicity and the classical bounded and finite factorization properties introduced and studied in 1990 by Anderson, Anderson, and Zafrullah in the setting of integral domains, and then generalized by Halter-Koch to commutative monoids. We pay special attention to how each of the considered properties behaves under the internal sum of a Puiseux monoid with a finitely generated Puiseux monoid. Throughout the paper, we also discuss examples showing that our primary results do not hold for submonoids of torsion-free abelian groups with rank larger than $1$., Comment: 14 pages. To appear in the Korean Journal of Mathematics

We study how differences in persuasive language across Wikipedia articles, written in either English and Russian, can uncover each culture's distinct perspective on different subjects. We develop a large language model (LLM) powered system to identify instances of persuasive language in multilingual texts. Instead of directly prompting LLMs to detect persuasion, which is subjective and difficult, we propose to reframe the task to instead ask high-level questions (HLQs) which capture different persuasive aspects. Importantly, these HLQs are authored by LLMs themselves. LLMs over-generate a large set of HLQs, which are subsequently filtered to a small set aligned with human labels for the original task. We then apply our approach to a large-scale, bilingual dataset of Wikipedia articles (88K total), using a two-stage identify-then-extract prompting strategy to find instances of persuasion. We quantify the amount of persuasion per article, and explore the differences in persuasion through several experiments on the paired articles. Notably, we generate rankings of articles by persuasion in both languages. These rankings match our intuitions on the culturally-salient subjects; Russian Wikipedia highlights subjects on Ukraine, while English Wikipedia highlights the Middle East. Grouping subjects into larger topics, we find politically-related events contain more persuasion than others. We further demonstrate that HLQs obtain similar performance when posed in either English or Russian. Our methodology enables cross-lingual, cross-cultural understanding at scale, and we release our code, prompts, and data.

This work introduces Text2FX, a method that leverages CLAP embeddings and differentiable digital signal processing to control audio effects, such as equalization and reverberation, using open-vocabulary natural language prompts (e.g., "make this sound in-your-face and bold"). Text2FX operates without retraining any models, relying instead on single-instance optimization within the existing embedding space. We show that CLAP encodes valuable information for controlling audio effects and propose two optimization approaches using CLAP to map text to audio effect parameters. While we demonstrate with CLAP, this approach is applicable to any shared text-audio embedding space. Similarly, while we demonstrate with equalization and reverberation, any differentiable audio effect may be controlled. We conduct a listener study with diverse text prompts and source audio to evaluate the quality and alignment of these methods with human perception., Comment: Submitted to ICASSP 2025

Novel solutions for pseudometrics quantifying deviation from topological conjugacy between dynamical systems are presented. Deviation from conjugacy is quantified in a Pareto optimal sense that accounts for spectral properties of Koopman operators as well as trajectory geometry. Theoretical justification is provided for computing such pseudometrics in Koopman eigenfunction space rather than observable space. Furthermore, it is shown deriving the pseudometrics from unitary transformations is sufficient to recover a value of zero if two systems are topologically conjugate. Therefore the pseudometrics for quantifying deviation from conjugacy are based on analytical solutions for unitary transformations in Koopman eigenfunction space. Finally, geometric considerations for the Pareto optimality problem associated with deviation from conjugacy are used to develop pseudometrics that account for all possible solutions given just two Pareto points based on analytical solutions.

Synthetic face recognition (SFR) aims to generate synthetic face datasets that mimic the distribution of real face data, which allows for training face recognition models in a privacy-preserving manner. Despite the remarkable potential of diffusion models in image generation, current diffusion-based SFR models struggle with generalization to real-world faces. To address this limitation, we outline three key objectives for SFR: (1) promoting diversity across identities (inter-class diversity), (2) ensuring diversity within each identity by injecting various facial attributes (intra-class diversity), and (3) maintaining identity consistency within each identity group (intra-class identity preservation). Inspired by these goals, we introduce a diffusion-fueled SFR model termed $\text{ID}^3$. $\text{ID}^3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances. Theoretically, we show that minimizing this loss is equivalent to maximizing the lower bound of an adjusted conditional log-likelihood over ID-preserving data. This equivalence motivates an ID-preserving sampling algorithm, which operates over an adjusted gradient vector field, enabling the generation of fake face recognition datasets that approximate the distribution of real-world faces. Extensive experiments across five challenging benchmarks validate the advantages of $\text{ID}^3$., Comment: Accepted to NeurIPS 2024

Lattice gauge theories (LGTs) can be employed to understand a wide range of phenomena, from elementary particle scattering in high-energy physics to effective descriptions of many-body interactions in materials. Studying dynamical properties of emergent phases can be challenging as it requires solving many-body problems that are generally beyond perturbative limits. We investigate the dynamics of local excitations in a $\mathbb{Z}_2$ LGT using a two-dimensional lattice of superconducting qubits. We first construct a simple variational circuit which prepares low-energy states that have a large overlap with the ground state; then we create particles with local gates and simulate their quantum dynamics via a discretized time evolution. As the effective magnetic field is increased, our measurements show signatures of transitioning from deconfined to confined dynamics. For confined excitations, the magnetic field induces a tension in the string connecting them. Our method allows us to experimentally image string dynamics in a (2+1)D LGT from which we uncover two distinct regimes inside the confining phase: for weak confinement the string fluctuates strongly in the transverse direction, while for strong confinement transverse fluctuations are effectively frozen. In addition, we demonstrate a resonance condition at which dynamical string breaking is facilitated. Our LGT implementation on a quantum processor presents a novel set of techniques for investigating emergent particle and string dynamics.

The BICEP3 and BICEP Array polarimeters are small-aperture refracting telescopes located at the South Pole designed to measure primordial gravitational wave signatures in the Cosmic Microwave Background (CMB) polarization, predicted by inflation. Constraining the inflationary signal requires not only excellent sensitivity, but also careful control of instrumental systematics. Both instruments use antenna-coupled orthogonally polarized detector pairs, and the polarized sky signal is reconstructed by taking the difference in each detector pair. As a result, the differential response between detectors within a pair becomes an important systematic effect we must control. Additionally, mapping the intensity and polarization response in regions away from the main beam can inform how sidelobe levels affect CMB measurements. Extensive calibration measurements are taken in situ every austral summer for control of instrumental systematics and instrument characterisation. In this work, we detail the set of beam calibration measurements that we conduct on the BICEP receivers, from deep measurements of main beam response to polarized beam response and sidelobe mapping. We discuss the impact of these measurements for instrumental systematics studies and design choices for future CMB receivers., Comment: 13 pages, 7 figures, 1 table, Proceedings paper SPIE 2024

Quality-Diversity (QD) algorithms have exhibited promising results across many domains and applications. However, uncertainty in fitness and behaviour estimations of solutions remains a major challenge when QD is used in complex real-world applications. While several approaches have been proposed to improve the performance in uncertain applications, many fail to address a key challenge: determining how to prioritise solutions that perform consistently under uncertainty, in other words, solutions that are reproducible. Most prior methods improve fitness and reproducibility jointly, ignoring the possibility that they could be contradictory objectives. For example, in robotics, solutions may reliably walk at 90% of the maximum velocity in uncertain environments, while solutions that walk faster are also more prone to falling over. As this is a trade-off, neither one of these two solutions is "better" than the other. Thus, algorithms cannot intrinsically select one solution over the other, but can only enforce given preferences over these two contradictory objectives. In this paper, we formalise this problem as the performance-reproducibility trade-off for uncertain QD. We propose four new a-priori QD algorithms that find optimal solutions for given preferences over the trade-offs. We also propose an a-posteriori QD algorithm for when these preferences cannot be defined in advance. Our results show that our approaches successfully find solutions that satisfy given preferences. Importantly, by simply accounting for this trade-off, our approaches perform better than existing uncertain QD methods. This suggests that considering the performance-reproducibility trade-off unlocks important stepping stones that are usually missed when only performance is optimised.

We demonstrate an optical clock based on Ramsey-Bord\'e interferometry in a laser-cooled ${}^{40}$Ca beam. The mean velocity is reduced by an order of magnitude relative to a thermal beam and the transverse temperature approaches the Doppler limit, enabling the measurement of sub-kHz linewidth fringes in a compact interferometer. Using tailored phase and intensity modulation of the spectroscopy laser to add uniform frequency sidebands, we interrogate atoms throughout the transverse velocity distribution, increasing the Ramsey-Bord\'e fringe amplitude by a factor of 14 and improving the Allan deviation to $3.4\times 10^{-15}$ at one second averaging time., Comment: 6 pages, 4 figures

In modern software development, vulnerability detection is crucial due to the inevitability of bugs and vulnerabilities in complex software systems. Effective detection and elimination of these vulnerabilities during the testing phase are essential. Current methods, such as fuzzing, are widely used for this purpose. While fuzzing is efficient in identifying a broad range of bugs and vulnerabilities by using random mutations or generations, it does not guarantee correctness or absence of vulnerabilities. Therefore, non-random methods are preferable for ensuring the safety and security of critical infrastructure and control systems. This paper presents a vulnerability detection approach based on symbolic execution and control flow graph analysis to identify various types of software weaknesses. Our approach employs a divide-and-conquer algorithm to eliminate irrelevant program information, thus accelerating the process and enabling the analysis of larger programs compared to traditional symbolic execution and model checking methods.

E-commerce stores enable multilingual product discovery which require accurate product title translation. Multilingual large language models (LLMs) have shown promising capacity to perform machine translation tasks, and it can also enhance and translate product titles cross-lingually in one step. However, product title translation often requires more than just language conversion because titles are short, lack context, and contain specialized terminology. This study proposes a retrieval-augmented generation (RAG) approach that leverages existing bilingual product information in e-commerce by retrieving similar bilingual examples and incorporating them as few-shot prompts to enhance LLM-based product title translation. Experiment results show that our proposed RAG approach improve product title translation quality with chrF score gains of up to 15.3% for language pairs where the LLM has limited proficiency., Comment: 6 Pages,In Proceedings of ACM CIKM Workshop on Data-Centric AI (CIKM DCAI 2024)