Search

We present 850 $\mu$m polarization observations of the IC 348 star-forming region in the Perseus molecular cloud as part of the B-fields In STar-forming Region Observation (BISTRO) survey. We study the magnetic properties of two cores (HH 211 MMS and IC 348 MMS) and a filamentary structure of IC 348. We find that the overall field tends to be more perpendicular than parallel to the filamentary structure of the region. The polarization fraction decreases with intensity, and we estimate the trend by power-law and the mean of the Rice distribution fittings. The power indices for the cores are much smaller than 1, indicative of possible grain growth to micron size in the cores. We also measure the magnetic field strengths of the two cores and the filamentary area separately by applying the Davis-Chandrasekhar-Fermi method and its alternative version for compressed medium. The estimated mass-to-flux ratios are 0.45-2.20 and 0.63-2.76 for HH 211 MMS and IC 348 MMS, respectively, while the ratios for the filament is 0.33-1.50. This result may suggest that the transition from subcritical to supercritical conditions occurs at the core scale ($\sim$ 0.05 pc) in the region. In addition, we study the energy balance of the cores and find that the relative strength of turbulence to the magnetic field tends to be stronger for IC 348 MMS than HH 211 MMS. The result could potentially explain the different configurations inside the two cores: a single protostellar system in HH 211 MMS and multiple protostars in IC 348 MMS., Comment: Accepted for publication in ApJ. 21 pages, 12 figures

Recent advances in quantum information processing with superconducting qubits have fueled a growing demand for scaling and miniaturizing circuit layouts. Despite significant progress, accurately predicting the Hamiltonian of complex circuits remains a challenging task. In this work, we propose an improved method for quantizing superconducting circuits that incorporates material- and geometry-dependent kinetic inductance. Our approach models thin superconducting films as equivalent reactive boundary elements, seamlessly integrating into the conventional circuit quantization framework without adding computational complexity. As a proof of concept, we experimentally verify our method using planar superconducting quantum devices made of 35 nm-thick disordered niobium films, known to exhibit large kinetic inductance values. We demonstrate significantly improved accuracy in predicting the Hamiltonian based solely on the chip layout and material properties of the superconducting films and Josephson junctions, with discrepancies in mode frequencies remaining below 2%. Our method enables systematic studies of superconducting devices based on disordered thin films or compact, fine-pitched elements and, more broadly, facilitates the precise engineering of superconducting quantum circuits at scale., Comment: 12 pages, 4 figures, 1 table

In the bottomonium sector, the hindered magnetic dipole (M1) transitions between P-wave states $h_b(2P) \rightarrow \chi_{bJ}(1P) \gamma$, $J=0, \, 1, \, 2$, are expected to be severely suppressed according to the Relativized Quark Model, due to the spin flip of the $b$ quark. Nevertheless, a recent model following the coupled-channel approach predicts the corresponding branching fractions to be enhanced by orders of magnitude. In this Letter, we report the first search for such transitions. We find no significant signals and set upper limits at 90% CL on the corresponding branching fractions: $\mathcal{B}[h_b(2P)\to\gamma\chi_{b0}(1P)] < 2.7 \times 10^{-1}$, $\mathcal{B}[h_b(2P)\to\gamma\chi_{b1}(1P)] < 5.4 \times 10^{-3}$ and $\mathcal{B}[h_b(2P)\to\gamma\chi_{b2}(1P)] < 1.3 \times 10^{-2}$. These values help to constrain the parameters of the coupled-channel models. The results are obtained using a $121.4 \, fb^{-1}$ data sample taken around $\sqrt{s}= 10.860 \, GeV$ with the Belle detector at the KEKB asymmetric-energy $e^+e^-$ collider.

Accelerating end-to-end inference of transformer-based large language models (LLMs) is a critical component of AI services in datacenters. However, diverse compute characteristics of end-to-end LLM inference present challenges as previously proposed accelerators only address certain operations or stages (e.g., self-attention, generation stage, etc.). To address the unique challenges of accelerating end-to-end inference, we propose IANUS -- Integrated Accelerator based on NPU-PIM Unified Memory System. IANUS is a domain-specific system architecture that combines a Neural Processing Unit (NPU) with a Processing-in-Memory (PIM) to leverage both the NPU's high computation throughput and the PIM's high effective memory bandwidth. In particular, IANUS employs a unified main memory system where the PIM memory is used both for PIM operations and for NPU's main memory. The unified main memory system ensures that memory capacity is efficiently utilized and the movement of shared data between NPU and PIM is minimized. However, it introduces new challenges since normal memory accesses and PIM computations cannot be performed simultaneously. Thus, we propose novel PIM Access Scheduling that manages normal memory accesses and PIM computations through workload mapping and scheduling across the PIM and the NPU. Our detailed simulation evaluations show that IANUS improves the performance of GPT-2 by 6.2$\times$ and 3.2$\times$, on average, compared to the NVIDIA A100 GPU and the state-of-the-art accelerator. As a proof-of-concept, we develop a prototype of IANUS with a commercial PIM, NPU, and an FPGA-based PIM controller to demonstrate the feasibility of IANUS., Comment: Updated version of the paper accepted to ASPLOS 2024

We examine diverging preferences in human-labeled preference datasets. We develop a taxonomy of disagreement sources spanning 10 categories across four high-level classes -- task underspecification, response style, refusals, and annotation errors. We find that the majority of disagreements are in opposition with standard reward modeling approaches, which are designed with the assumption that annotator disagreement is noise. We then explore how these findings impact two areas of LLM development: reward modeling and evaluation. In our experiments, we demonstrate how standard reward modeling methods, like the Bradley-Terry model, fail to differentiate whether a given preference judgment is the result of unanimous agreement among annotators or the majority opinion among diverging user preferences. We also find that these tendencies are also echoed by popular LLM-as-Judge evaluation methods, which consistently identify a winning response in cases of diverging preferences. These findings highlight remaining challenges in LLM evaluations, which are greatly influenced by divisive features like response style, and in developing pluralistically aligned LLMs. To address these issues, we develop methods for identifying diverging preferences to mitigate their influence on evaluation and training.

The goal of this paper is to accelerate codec-based speech synthesis systems with minimum sacrifice to speech quality. We propose an enhanced inference method that allows for flexible trade-offs between speed and quality during inference without requiring additional training. Our core idea is to predict multiple tokens per inference step of the AR module using multiple prediction heads, resulting in a linear reduction in synthesis time as the number of heads increases. Furthermore, we introduce a novel speculative decoding technique that utilises a Viterbi-based algorithm to select the optimal sequence of generated tokens at each decoding step. In our experiments, we demonstrate that the time required to predict each token is reduced by a factor of 4 to 5 compared to baseline models, with minimal quality trade-off or even improvement in terms of speech intelligibility. Audio samples are available at: multpletokensprediction.github.io/multipletokensprediction.github.io/., Comment: Submitted to IEEE ICASSP 2025

Large language models (LLMs) have demonstrated remarkable performance in the legal domain, with GPT-4 even passing the Uniform Bar Exam in the U.S. However their efficacy remains limited for non-standardized tasks and tasks in languages other than English. This underscores the need for careful evaluation of LLMs within each legal system before application. Here, we introduce KBL, a benchmark for assessing the Korean legal language understanding of LLMs, consisting of (1) 7 legal knowledge tasks (510 examples), (2) 4 legal reasoning tasks (288 examples), and (3) the Korean bar exam (4 domains, 53 tasks, 2,510 examples). First two datasets were developed in close collaboration with lawyers to evaluate LLMs in practical scenarios in a certified manner. Furthermore, considering legal practitioners' frequent use of extensive legal documents for research, we assess LLMs in both a closed book setting, where they rely solely on internal knowledge, and a retrieval-augmented generation (RAG) setting, using a corpus of Korean statutes and precedents. The results indicate substantial room and opportunities for improvement., Comment: EMNLP 2024 Findings

Large Language Models (LLMs) excel at understanding the semantic relationships between queries and documents, even with lengthy and complex long-tail queries. These queries are challenging for feedback-based rankings due to sparse user engagement and limited feedback, making LLMs' ranking ability highly valuable. However, the large size and slow inference of LLMs necessitate the development of smaller, more efficient models (sLLMs). Recently, integrating ranking label generation into distillation techniques has become crucial, but existing methods underutilize LLMs' capabilities and are cumbersome. Our research, RRADistill: Re-Ranking Ability Distillation, propose an efficient label generation pipeline and novel sLLM training methods for both encoder and decoder models. We introduce an encoder-based method using a Term Control Layer to capture term matching signals and a decoder-based model with a ranking layer for enhanced understanding. A/B testing on a Korean-based search platform, validates the effectiveness of our approach in improving re-ranking for long-tail queries., Comment: Accepted to EMNLP 2024 Industry Track. First two authors contributed equally

Large language models (LLMs) have achieved significant performance gains using advanced prompting techniques over various tasks. However, the increasing length of prompts leads to high computational costs and often obscures crucial information. Prompt compression has been proposed to alleviate these issues, but it faces challenges in (i) capturing the global context and (ii) training the compressor effectively. To tackle these challenges, we introduce a novel prompt compression method, namely Reading To Compressing (R2C), utilizing the Fusion-in-Decoder (FiD) architecture to identify the important information in the prompt. Specifically, the cross-attention scores of the FiD are used to discern essential chunks and sentences from the prompt. R2C effectively captures the global context without compromising semantic consistency while detouring the necessity of pseudo-labels for training the compressor. Empirical results show that R2C retains key contexts, enhancing the LLM performance by 6% in out-of-domain evaluations while reducing the prompt length by 80%., Comment: Findings of the Association for Computational Linguistics: EMNLP 2024; 21 pages; 10 figures and 7 tables

A common challenge in aggregating data from multiple sources can be formalized as an \textit{Optimal Transport} (OT) barycenter problem, which seeks to compute the average of probability distributions with respect to OT discrepancies. However, the presence of outliers and noise in the data measures can significantly hinder the performance of traditional statistical methods for estimating OT barycenters. To address this issue, we propose a novel, scalable approach for estimating the \textit{robust} continuous barycenter, leveraging the dual formulation of the \textit{(semi-)unbalanced} OT problem. To the best of our knowledge, this paper is the first attempt to develop an algorithm for robust barycenters under the continuous distribution setup. Our method is framed as a $\min$-$\max$ optimization problem and is adaptable to \textit{general} cost function. We rigorously establish the theoretical underpinnings of the proposed method and demonstrate its robustness to outliers and class imbalance through a number of illustrative experiments., Comment: 19 pages, 4 figures

Optimal Transport (OT) theory investigates the cost-minimizing transport map that moves a source distribution to a target distribution. Recently, several approaches have emerged for learning the optimal transport map for a given cost function using neural networks. We refer to these approaches as the OT Map. OT Map provides a powerful tool for diverse machine learning tasks, such as generative modeling and unpaired image-to-image translation. However, existing methods that utilize max-min optimization often experience training instability and sensitivity to hyperparameters. In this paper, we propose a novel method to improve stability and achieve a better approximation of the OT Map by exploiting displacement interpolation, dubbed Displacement Interpolation Optimal Transport Model (DIOTM). We derive the dual formulation of displacement interpolation at specific time $t$ and prove how these dual problems are related across time. This result allows us to utilize the entire trajectory of displacement interpolation in learning the OT Map. Our method improves the training stability and achieves superior results in estimating optimal transport maps. We demonstrate that DIOTM outperforms existing OT-based models on image-to-image translation tasks., Comment: 20 pages

Unpaired point cloud completion explores methods for learning a completion map from unpaired incomplete and complete point cloud data. In this paper, we propose a novel approach for unpaired point cloud completion using the unbalanced optimal transport map, called Unbalanced Optimal Transport Map for Unpaired Point Cloud Completion (UOT-UPC). We demonstrate that the unpaired point cloud completion can be naturally interpreted as the Optimal Transport (OT) problem and introduce the Unbalanced Optimal Transport (UOT) approach to address the class imbalance problem, which is prevalent in unpaired point cloud completion datasets. Moreover, we analyze the appropriate cost function for unpaired completion tasks. This analysis shows that the InfoCD cost function is particularly well-suited for this task. Our model is the first attempt to leverage UOT for unpaired point cloud completion, achieving competitive or superior results on both single-category and multi-category datasets. In particular, our model is especially effective in scenarios with class imbalance, where the proportions of categories are different between the incomplete and complete point cloud datasets., Comment: 20 pages, 10 figures

The Optimal Transport (OT) problem investigates a transport map that connects two distributions while minimizing a given cost function. Finding such a transport map has diverse applications in machine learning, such as generative modeling and image-to-image translation. In this paper, we introduce a scalable and simulation-free approach for solving the Entropic Unbalanced Optimal Transport (EUOT) problem. We derive the dynamical form of this EUOT problem, which is a generalization of the Schr\"odinger bridges (SB) problem. Based on this, we derive dual formulation and optimality conditions of the EUOT problem from the stochastic optimal control interpretation. By leveraging these properties, we propose a simulation-free algorithm to solve EUOT, called Simulation-free EUOT (SF-EUOT). While existing SB models require expensive simulation costs during training and evaluation, our model achieves simulation-free training and one-step generation by utilizing the reciprocal property. Our model demonstrates significantly improved scalability in generative modeling and image-to-image translation tasks compared to previous SB methods., Comment: 26 pages

Place recognition plays an important role in achieving robust long-term autonomy. Real-world robots face a wide range of weather conditions (e.g. overcast, heavy rain, and snowing) and most sensors (i.e. camera, LiDAR) essentially functioning within or near-visible electromagnetic waves are sensitive to adverse weather conditions, making reliable localization difficult. In contrast, radar is gaining traction due to long electromagnetic waves, which are less affected by environmental changes and weather independence. In this work, we propose a radar-based lightweight and robust place recognition. We achieve rotational invariance and lightweight by selecting a one-dimensional ring-shaped description and robustness by mitigating the impact of false detection utilizing opposite noise characteristics between free space and feature. In addition, the initial heading can be estimated, which can assist in building a SLAM pipeline that combines odometry and registration, which takes into account onboard computing. The proposed method was tested for rigorous validation across various scenarios (i.e. single session, multi-session, and different weather conditions). In particular, we validate our descriptor achieving reliable place recognition performance through the results of extreme environments that lacked structural information such as an OORD dataset., Comment: 8 pages, 8 figures, accepted to RA-L

Text-to-SQL systems have become crucial for translating natural language into SQL queries in various industries, enabling non-technical users to perform complex data operations. The need for accurate evaluation methods has increased as these systems have grown more sophisticated. However, the Execution Accuracy (EX), the most prevalent evaluation metric, still shows many false positives and negatives. Thus, this paper introduces FLEX (False-Less EXecution), a novel approach to evaluating text-to-SQL systems using large language models (LLMs) to emulate human expert-level evaluation of SQL queries. Our metric improves agreement with human experts (from 62 to 87.04 in Cohen's kappa) with comprehensive context and sophisticated criteria. Our extensive experiments yield several key insights: (1) Models' performance increases by over 2.6 points on average, substantially affecting rankings on Spider and BIRD benchmarks; (2) The underestimation of models in EX primarily stems from annotation quality issues; and (3) Model performance on particularly challenging questions tends to be overestimated. This work contributes to a more accurate and nuanced evaluation of text-to-SQL systems, potentially reshaping our understanding of state-of-the-art performance in this field., Comment: preprint, under review

For over 25 years, the DAMA/LIBRA collaboration has claimed to observe an annual modulation signal, suggesting the existence of dark matter interactions. However, no other experiments have replicated their result using different detector materials. To address this puzzle, the COSINE-100 collaboration conducted a model-independent test using 106 kg of sodium iodide as detectors, the same target material as DAMA/LIBRA. Analyzing data collected over 6.4 years, with improved energy calibration and time-dependent background description, we found no evidence of an annual modulation signal, challenging the DAMA/LIBRA result with a confidence level greater than 3$\sigma$. This finding represents a significant step toward resolving the long-standing debate surrounding DAMA/LIBRA's dark matter claim, indicating that the observed modulation is unlikely to be caused by dark matter interactions.

We report the first measurement of the azimuthal anisotropy of J$/\psi$ at forward rapidity ($1.2<|\eta|<2.2$) in Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$ GeV at the Relativistic Heavy Ion Collider. The data were collected by the PHENIX experiment in 2014 and 2016 with integrated luminosity of 14.5~nb$^{-1}$. The second Fourier coefficient ($v_2$) of the azimuthal distribution of $J/\psi$ is determined as a function of the transverse momentum ($p_T$) using the event-plane method. The measurements were performed for several selections of collision centrality: 0\%--50\%, 10\%--60\%, and 10\%-40\%. We find that in all cases the values of $v_2(p_T)$, which quantify the elliptic flow of J$/\psi$, are consistent with zero. The results are consistent with measurements at midrapidity, indicating no significant elliptic flow of the J$/\psi$ within the quark-gluon-plasma medium at collision energies of $\sqrt{s_{_{NN}}}=200$ GeV., Comment: 369 authors from 72 institutions, 12 pages, 7 figures, 5 tables. v1 is version submitted to Physical Review C. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

We present the first forward-rapidity measurements of elliptic anisotropy of open-heavy-flavor muons at the BNL Relativistic Heavy Ion Collider. The measurements are based on data samples of Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$ GeV collected by the PHENIX experiment in 2014 and 2016 with integrated luminosity of 14.5~nb$^{-1}$. The measurements are performed in the pseudorapidity range $1.2<|\eta|<2$ and cover transverse momenta $1

We propose a multichannel-to-multichannel target sound extraction (M2M-TSE) framework for separating multichannel target signals from a multichannel mixture of sound sources. Target sound extraction (TSE) isolates a specific target signal using user-provided clues, typically focusing on single-channel extraction with class labels or temporal activation maps. However, to preserve and utilize spatial information in multichannel audio signals, it is essential to extract multichannel signals of a target sound source. Moreover, the clue for extraction can also include spatial or temporal cues like direction-of-arrival (DoA) or timestamps of source activation. To address these challenges, we present an M2M framework that extracts a multichannel sound signal based on spatio-temporal clues. We demonstrate that our transformer-based architecture can successively accomplish the M2M-TSE task for multichannel signals synthesized from audio signals of diverse classes in different room environments. Furthermore, we show that the multichannel extraction task introduces sufficient inductive bias in the DNN, allowing it to directly handle DoA clues without utilizing hand-crafted spatial features., Comment: 5 pages, 4 figures

Air pollution, particularly particulate matter (PM2.5), poses significant risks to public health and the environment, necessitating accurate prediction and continuous monitoring for effective air quality management. However, air quality monitoring (AQM) data often suffer from missing records due to various technical difficulties. This study explores the application of Sparse Identification of Nonlinear Dynamics (SINDy) for imputing missing PM2.5 data by predicting, using training data from 2016, and comparing its performance with the established Soft Impute (SI) and K-Nearest Neighbors (KNN) methods.

State-of-the-art (SOTA) video denoising methods employ multi-frame simultaneous denoising mechanisms, resulting in significant delays (e.g., 16 frames), making them impractical for real-time cameras. To overcome this limitation, we propose a multi-fusion gated recurrent Transformer network (GRTN) that achieves SOTA denoising performance with only a single-frame delay. Specifically, the spatial denoising module extracts features from the current frame, while the reset gate selects relevant information from the previous frame and fuses it with current frame features via the temporal denoising module. The update gate then further blends this result with the previous frame features, and the reconstruction module integrates it with the current frame. To robustly compute attention for noisy features, we propose a residual simplified Swin Transformer with Euclidean distance (RSSTE) in the spatial and temporal denoising modules. Comparative objective and subjective results show that our GRTN achieves denoising performance comparable to SOTA multi-frame delay networks, with only a single-frame delay., Comment: 5 pages, 5 figures

The $J/\psi$ and $\psi(2S)$ charmonium states, composed of $c\bar{c}$ quark pairs and known since the 1970s, are widely believed to serve as ideal probes to test quantum chromodynamics in high-energy hadronic interactions. However, there is not yet a complete understanding of the charmonium-production mechanism. Recent measurements of $J/\psi$ production as a function of event charged-particle multiplicity at the collision energies of both the Large Hadron Collider (LHC) and the Relativistic Heavy Ion Collider (RHIC) show enhanced $J/\psi$ production yields with increasing multiplicity. One potential explanation for this type of dependence is multiparton interactions (MPI). We carry out the first measurements of self-normalized $J/\psi$ yields and the $\psi(2S)$ to $J/\psi$ ratio at both forward and backward rapidities as a function of self-normalized charged-particle multiplicity in $p$$+$$p$ collisions at $\sqrt{s}=200$ GeV. In addition, detailed {\sc pythia} studies tuned to RHIC energies were performed to investigate the MPI impacts. We find that the PHENIX data at RHIC are consistent with recent LHC measurements and can only be described by {\sc pythia} calculations that include MPI effects. The forward and backward $\psi(2S)$ to $J/\psi$ ratio, which serves as a unique and powerful approach to study final-state effects on charmonium production, is found to be less dependent on the charged-particle multiplicity., Comment: 301 authors from 69 institutions, 8 pages, 3 figures. v1 is version submitted to Physical Review D Letters. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

We present the first measurement of the missing energy due to nuclear effects in monoenergetic, muon neutrino charged-current interactions on carbon, originating from $K^+ \rightarrow \mu^+ \nu_\mu$ decay-at-rest ($E_{\nu_\mu}=235.5$ MeV), performed with the JSNS$^2$ liquid scintillator based experiment. Towards characterizing the neutrino interaction, ostensibly $\nu_\mu n \rightarrow \mu^- p$ or $\nu_\mu$$^{12}\mathrm{C}$ $\rightarrow \mu^-$$^{12}\mathrm{N}$, and in analogy to similar electron scattering based measurements, we define the missing energy as the energy transferred to the nucleus ($\omega$) minus the kinetic energy of the outgoing proton(s), $E_{m} \equiv \omega-\sum T_p$, and relate this to visible energy in the detector, $E_{m}=E_{\nu_\mu}~(235.5~\mathrm{MeV})-m_\mu~(105.7~\mathrm{MeV}) - E_{vis}$. The missing energy, which is naively expected to be zero in the absence of nuclear effects (e.g. nucleon separation energy, Fermi momenta, and final-state interactions), is uniquely sensitive to many aspects of the interaction, and has previously been inaccessible with neutrinos. The shape-only, differential cross section measurement reported, based on a $(77\pm3)$% pure double-coincidence KDAR signal (621 total events), provides an important benchmark for models and event generators at 100s-of-MeV neutrino energies, characterized by the difficult-to-model transition region between neutrino-nucleus and neutrino-nucleon scattering, and relevant for applications in nuclear physics, neutrino oscillation measurements, and Type-II supernova studies.

Generating long sequences of tokens given a long-context input imposes a heavy computational burden for large language models (LLMs). One of the computational bottleneck comes from computing attention over a long sequence of input at each generation step. In this paper, we propose Recycled Attention, an inference-time method which alternates between full context attention and attention over a subset of input tokens. When performing partial attention, we recycle the attention pattern of a previous token that has performed full attention and attend only to the top K most attended tokens, reducing the cost of data movement and attention computation. Compared to previously proposed inference-time acceleration method which attends only to local context or tokens with high accumulative attention scores, our approach flexibly chooses tokens that are relevant to the current decoding step. We evaluate our methods on RULER, a suite of tasks designed to comprehensively evaluate long-context abilities, and long-context language modeling tasks. Applying our method to off-the-shelf LLMs achieves comparable speedup to baselines which only consider local context while improving the performance by 2x. We further explore two ideas to improve performance-efficiency trade-offs: (1) dynamically decide when to perform recycled or full attention step based on the query similarities and (2) continued pre-training the model with Recycled Attention.

As global warming increases the complexity of weather patterns; the precision of weather forecasting becomes increasingly important. Our study proposes a novel preprocessing method and convolutional autoencoder model developed to improve the interpretation of synoptic weather maps. These are critical for meteorologists seeking a thorough understanding of weather conditions. This model could recognize historical synoptic weather maps that nearly match current atmospheric conditions, marking a significant step forward in modern technology in meteorological forecasting. This comprises unsupervised learning models like VQ-VQE, as well as supervised learning models like VGG16, VGG19, Xception, InceptionV3, and ResNet50 trained on the ImageNet dataset, as well as research into newer models like EfficientNet and ConvNeXt. Our findings proved that, while these models perform well in various settings, their ability to identify comparable synoptic weather maps has certain limits. Our research, motivated by the primary goal of significantly increasing meteorologists' efficiency in labor-intensive tasks, discovered that cosine similarity is the most effective metric, as determined by a combination of quantitative and qualitative assessments to accurately identify relevant historical weather patterns. This study broadens our understanding by shifting the emphasis from numerical precision to practical application, ensuring that our model is effective in theory practical, and accessible in the complex and dynamic field of meteorology.

Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spoken language model that comprehends a wide range of natural language instructions is critical for bridging communication gaps and facilitating more intuitive interactions. However, the absence of a comprehensive evaluation benchmark poses a significant challenge. We present Dynamic-SUPERB Phase-2, an open and evolving benchmark for the comprehensive evaluation of instruction-based universal speech models. Building upon the first generation, this second version incorporates 125 new tasks contributed collaboratively by the global research community, expanding the benchmark to a total of 180 tasks, making it the largest benchmark for speech and audio evaluation. While the first generation of Dynamic-SUPERB was limited to classification tasks, Dynamic-SUPERB Phase-2 broadens its evaluation capabilities by introducing a wide array of novel and diverse tasks, including regression and sequence generation, across speech, music, and environmental audio. Evaluation results indicate that none of the models performed well universally. SALMONN-13B excelled in English ASR, while WavLLM demonstrated high accuracy in emotion recognition, but current models still require further innovations to handle a broader range of tasks. We will soon open-source all task data and the evaluation pipeline.

The performance of vision-language models (VLMs), such as CLIP, in visual classification tasks, has been enhanced by leveraging semantic knowledge from large language models (LLMs), including GPT. Recent studies have shown that in zero-shot classification tasks, descriptors incorporating additional cues, high-level concepts, or even random characters often outperform those using only the category name. In many classification tasks, while the top-1 accuracy may be relatively low, the top-5 accuracy is often significantly higher. This gap implies that most misclassifications occur among a few similar classes, highlighting the model's difficulty in distinguishing between classes with subtle differences. To address this challenge, we introduce a novel concept of comparative descriptors. These descriptors emphasize the unique features of a target class against its most similar classes, enhancing differentiation. By generating and integrating these comparative descriptors into the classification framework, we refine the semantic focus and improve classification accuracy. An additional filtering process ensures that these descriptors are closer to the image embeddings in the CLIP space, further enhancing performance. Our approach demonstrates improved accuracy and robustness in visual classification tasks by addressing the specific challenge of subtle inter-class differences., Comment: Accepted to WACV 2025. Main paper with 8 pages

In this paper, we give a characterization of Fano type varieties of plc pairs. This extends a result due to Choi and Park for pklt pairs. Moreover, we show that for a plc pair $(X,\Delta)$, if no plc centers are contained in the augmented base locus $\mathbf{B}_{+}(-(K_X+\Delta))$, then $(X,\Delta)$ admits a good $-(K_X+\Delta)$-minimal model.

We present a novel OCR-free document understanding framework based on pretrained Multimodal Large Language Models (MLLMs). Our approach employs multi-scale visual features to effectively handle various font sizes within document images. To address the increasing costs of considering the multi-scale visual inputs for MLLMs, we propose the Hierarchical Visual Feature Aggregation (HVFA) module, designed to reduce the number of input tokens to LLMs. Leveraging a feature pyramid with cross-attentive pooling, our approach effectively manages the trade-off between information loss and efficiency without being affected by varying document image sizes. Furthermore, we introduce a novel instruction tuning task, which facilitates the model's text-reading capability by learning to predict the relative positions of input text, eventually minimizing the risk of truncated text caused by the limited capacity of LLMs. Comprehensive experiments validate the effectiveness of our approach, demonstrating superior performance in various document understanding tasks., Comment: NeurIPS 2024

This demo is about automatic authoring of various motion effects that are provided with audiovisual content to improve user experiences. Traditionally, motion effects have been used for simulators, e.g., flight simulators for pilots and astronauts, to present physically accurate vestibular feedback. At present, we have greatly wider use of motion effects for entertainment purposes, such as 4D rides in amusement parks and even shopping malls, 4D films in theaters, and relative new virtual reality games with head-mounted displays and personal motion platforms. However, the production of motion effects is done solely by manual authoring or coding, and this costly process prevents the faster and wider dissemination of 4D content. It is imperative to facilitate motion effect production by providing automatic synthesis algorithms. This demo video presents nine different automatic synthesis algorithms for motion effects and a recorded demonstration of each., Comment: Part of proceedings of 6th International Conference AsiaHaptics 2024

To increase social bonding with interlocutors, humans naturally acquire the ability to respond appropriately in a given situation by considering which conversational skill is most suitable for the response - a process we call skill-of-mind. For large language model (LLM)-based conversational agents, planning appropriate conversational skills, as humans do, is challenging due to the complexity of social dialogue, especially in interactive scenarios. To address this, we propose a skill-of-mind-annotated conversation dataset, named Multifaceted Skill-of-Mind, which includes multi-turn and multifaceted conversational skills across various interactive scenarios (e.g., long-term, counseling, task-oriented), grounded in diverse social contexts (e.g., demographics, persona, rules of thumb). This dataset consists of roughly 100K conversations. Using this dataset, we introduce a new family of skill-of-mind-infused LLMs, named Thanos, with model sizes of 1B, 3B, and 8B parameters. With extensive experiments, these models successfully demonstrate the skill-of-mind process and exhibit strong generalizability in inferring multifaceted skills across a variety of domains. Moreover, we show that Thanos significantly enhances the quality of responses generated by LLM-based conversational agents and promotes prosocial behavior in human evaluations., Comment: Code: https://github.com/passing2961/Thanos

We explore the use of a spatial mode sorter to image a nanomechanical resonator, with the goal of studying the quantum limits of active imaging and extending the toolbox for optomechanical force sensing. In our experiment, we reflect a Gaussian laser beam from a vibrating nanoribbon and pass the reflected beam through a commercial spatial mode demultiplexer (Cailabs Proteus). The intensity in each demultiplexed channel depends on the mechanical mode shapes and encodes information about their displacement amplitudes. As a concrete demonstration, we monitor the angular displacement of the ribbon's fundamental torsion mode by illuminating in the fundamental Hermite-Gauss mode (HG$_{00}$) and reading out in the HG$_{01}$ mode. We show that this technique permits readout of the ribbon's torsional vibration with a precision near the quantum limit. Our results highlight new opportunities at the interface of quantum imaging and quantum optomechanics., Comment: 8 pages, 5 figures

Machine unlearning has emerged as an effective strategy for forgetting specific information in the training data. However, with the increasing integration of visual data, privacy concerns in Vision Language Models (VLMs) remain underexplored. To address this, we introduce Facial Identity Unlearning Benchmark (FIUBench), a novel VLM unlearning benchmark designed to robustly evaluate the effectiveness of unlearning algorithms under the Right to be Forgotten setting. Specifically, we formulate the VLM unlearning task via constructing the Fictitious Facial Identity VQA dataset and apply a two-stage evaluation pipeline that is designed to precisely control the sources of information and their exposure levels. In terms of evaluation, since VLM supports various forms of ways to ask questions with the same semantic meaning, we also provide robust evaluation metrics including membership inference attacks and carefully designed adversarial privacy attacks to evaluate the performance of algorithms. Through the evaluation of four baseline VLM unlearning algorithms within FIUBench, we find that all methods remain limited in their unlearning performance, with significant trade-offs between model utility and forget quality. Furthermore, our findings also highlight the importance of privacy attacks for robust evaluations. We hope FIUBench will drive progress in developing more effective VLM unlearning algorithms.

Gas-phase molecules capable of repeatable, narrow-band spontaneous photon scattering are prized for direct laser cooling and quantum state detection. Recently, large molecules incorporating phenyl rings have been shown to exhibit similar vibrational closure to the small molecules demonstrated so far, and it is not yet known if the high vibrational-mode density of even larger species will eventually compromise optical cycling. Here, we systematically increase the size of hydrocarbon ligands attached to single alkaline-earth-phenoxides from (-H) to -C$_{14}$H$_{19}$ while measuring the vibrational branching fractions of the optical transition. We find that varying the ligand size from 1 to more than 30 atoms does not systematically reduce the cycle closure, which remains around 90%. Theoretical extensions to larger diamondoids and bulk diamond surface suggest that alkaline earth phenoxides may maintain the desirable scattering behavior as the system size grows further, with no indication of an upper limit., Comment: 9 pages, 5 figures, SI(34 pages)

Accurate and robust 3D object detection is a critical component in autonomous vehicles and robotics. While recent radar-camera fusion methods have made significant progress by fusing information in the bird's-eye view (BEV) representation, they often struggle to effectively capture the motion of dynamic objects, leading to limited performance in real-world scenarios. In this paper, we introduce CRT-Fusion, a novel framework that integrates temporal information into radar-camera fusion to address this challenge. Our approach comprises three key modules: Multi-View Fusion (MVF), Motion Feature Estimator (MFE), and Motion Guided Temporal Fusion (MGTF). The MVF module fuses radar and image features within both the camera view and bird's-eye view, thereby generating a more precise unified BEV representation. The MFE module conducts two simultaneous tasks: estimation of pixel-wise velocity information and BEV segmentation. Based on the velocity and the occupancy score map obtained from the MFE module, the MGTF module aligns and fuses feature maps across multiple timestamps in a recurrent manner. By considering the motion of dynamic objects, CRT-Fusion can produce robust BEV feature maps, thereby improving detection accuracy and robustness. Extensive evaluations on the challenging nuScenes dataset demonstrate that CRT-Fusion achieves state-of-the-art performance for radar-camera-based 3D object detection. Our approach outperforms the previous best method in terms of NDS by +1.7%, while also surpassing the leading approach in mAP by +1.4%. These significant improvements in both metrics showcase the effectiveness of our proposed fusion strategy in enhancing the reliability and accuracy of 3D object detection., Comment: Accepted at NeurIPS2024

We design a variational quantum algorithm to solve multi-dimensional Poisson equations with mixed boundary conditions that are typically required in various fields of computational science. Employing an objective function that is formulated with the concept of the minimal potential energy, we not only present in-depth discussion on the cost-efficient & noise-robust design of quantum circuits that are essential for evaluation of the objective function, but, more remarkably, employ the proposed algorithm to calculate bias-dependent spatial distributions of electric fields in semiconductor systems that are described with a two-dimensional domain and up to 10-qubit circuits. Extending the application scope to multi-dimensional problems with mixed boundary conditions for the first time, fairly solid computational results of this work clearly demonstrate the potential of variational quantum algorithms to tackle Poisson equations derived from physically meaningful problems., Comment: 10 pages, 7 figures

While piano music has become a significant area of study in Music Information Retrieval (MIR), there is a notable lack of datasets for piano solo music with text labels. To address this gap, we present PIAST (PIano dataset with Audio, Symbolic, and Text), a piano music dataset. Utilizing a piano-specific taxonomy of semantic tags, we collected 9,673 tracks from YouTube and added human annotations for 2,023 tracks by music experts, resulting in two subsets: PIAST-YT and PIAST-AT. Both include audio, text, tag annotations, and transcribed MIDI utilizing state-of-the-art piano transcription and beat tracking models. Among many possible tasks with the multi-modal dataset, we conduct music tagging and retrieval using both audio and MIDI data and report baseline performances to demonstrate its potential as a valuable resource for MIR research., Comment: Accepted for publication at the 3rd Workshop on NLP for Music and Audio (NLP4MusA 2024)

The advent of Large Language Models (LLMs) have shown promise in various creative domains, including culinary arts. However, many LLMs still struggle to deliver the desired level of culinary creativity, especially when tasked with adapting recipes to meet specific cultural requirements. This study focuses on cuisine transfer-applying elements of one cuisine to another-to assess LLMs' culinary creativity. We employ a diverse set of LLMs to generate and evaluate culturally adapted recipes, comparing their evaluations against LLM and human judgments. We introduce the ASH (authenticity, sensitivity, harmony) benchmark to evaluate LLMs' recipe generation abilities in the cuisine transfer task, assessing their cultural accuracy and creativity in the culinary domain. Our findings reveal crucial insights into both generative and evaluative capabilities of LLMs in the culinary domain, highlighting strengths and limitations in understanding and applying cultural nuances in recipe creation. The code and dataset used in this project will be openly available in \url{http://github.com/dmis-lab/CulinaryASH}.

While numerous studies have explored the field of research and development (R&D) landscaping, the preponderance of these investigations has emphasized predictive analysis based on R&D outcomes, specifically patents, and academic literature. However, the value of research proposals and novelty analysis has seldom been addressed. This study proposes a systematic approach to constructing and navigating the R&D landscape that can be utilized to guide organizations to respond in a reproducible and timely manner to the challenges presented by increasing number of research proposals. At the heart of the proposed approach is the composite use of the transformer-based language model and the local outlier factor (LOF). The semantic meaning of the research proposals is captured with our further-trained transformers, thereby constructing a comprehensive R&D landscape. Subsequently, the novelty of the newly selected research proposals within the annual landscape is quantified on a numerical scale utilizing the LOF by assessing the dissimilarity of each proposal to others preceding and within the same year. A case study examining research proposals in the energy and resource sector in South Korea is presented. The systematic process and quantitative outcomes are expected to be useful decision-support tools, providing future insights regarding R&D planning and roadmapping.

A long-standing goal in robot learning is to develop methods for robots to acquire new skills autonomously. While reinforcement learning (RL) comes with the promise of enabling autonomous data collection, it remains challenging to scale in the real-world partly due to the significant effort required for environment design and instrumentation, including the need for designing reset functions or accurate success detectors. On the other hand, imitation learning (IL) methods require little to no environment design effort, but instead require significant human supervision in the form of collected demonstrations. To address these shortcomings, recent works in autonomous IL start with an initial seed dataset of human demonstrations that an autonomous policy can bootstrap from. While autonomous IL approaches come with the promise of addressing the challenges of autonomous RL as well as pure IL strategies, in this work, we posit that such techniques do not deliver on this promise and are still unable to scale up autonomous data collection in the real world. Through a series of real-world experiments, we demonstrate that these approaches, when scaled up to realistic settings, face much of the same scaling challenges as prior attempts in RL in terms of environment design. Further, we perform a rigorous study of autonomous IL methods across different data scales and 7 simulation and real-world tasks, and demonstrate that while autonomous data collection can modestly improve performance, simply collecting more human data often provides significantly more improvement. Our work suggests a negative result: that scaling up autonomous data collection for learning robot policies for real-world tasks is more challenging and impractical than what is suggested in prior work. We hope these insights about the core challenges of scaling up data collection help inform future efforts in autonomous learning., Comment: 21 pages, 25 figures. Conference on Robot Learning (CoRL) 2024

Artificial neural networks (ANNs) have fundamentally transformed the field of computer vision, providing unprecedented performance. However, these ANNs for image processing demand substantial computational resources, often hindering real-time operation. In this paper, we demonstrate an optical encoder that can perform convolution simultaneously in three color channels during the image capture, effectively implementing several initial convolutional layers of a ANN. Such an optical encoding results in ~24,000 times reduction in computational operations, with a state-of-the art classification accuracy (~73.2%) in free-space optical system. In addition, our analog optical encoder, trained for CIFAR-10 data, can be transferred to the ImageNet subset, High-10, without any modifications, and still exhibits moderate accuracy. Our results evidence the potential of hybrid optical/digital computer vision system in which the optical frontend can pre-process an ambient scene to reduce the energy and latency of the whole computer vision system., Comment: 21 pages, 4 figures, 2 tables

Metallic glass is a frozen liquid with structural disorder that retains degenerate free energy without spontaneous symmetry breaking to become a solid. For over half a century, this puzzling structure has raised fundamental questions about how structural disorder impacts glass-liquid phase transition kinetics, which remain elusive without direct evidence. In this study, through single-pulse, time-resolved imaging using X-ray free-electron lasers, we visualized the glass-to-liquid transition, revealing a previously hidden dormant phase that does not involve any macroscopic volume change within the crossover regime between the two phases. Although macroscopically inactive, nanoscale redistribution occurs, forming channeld low-density bands within this dormant phase that drives the glass transition. By providing direct microscopic evidence, this work presents a new perspective on the phase transition process in disordered materials, which can be extended to various liquid and solid phases in other complex systems., Comment: 25 pages, 4 figures

Recent advancements in text-to-image diffusion models have enabled the personalization of these models to generate custom images from textual prompts. This paper presents an efficient LoRA-based personalization approach for on-device subject-driven generation, where pre-trained diffusion models are fine-tuned with user-specific data on resource-constrained devices. Our method, termed Hollowed Net, enhances memory efficiency during fine-tuning by modifying the architecture of a diffusion U-Net to temporarily remove a fraction of its deep layers, creating a hollowed structure. This approach directly addresses on-device memory constraints and substantially reduces GPU memory requirements for training, in contrast to previous methods that primarily focus on minimizing training steps and reducing the number of parameters to update. Additionally, the personalized Hollowed Net can be transferred back into the original U-Net, enabling inference without additional memory overhead. Quantitative and qualitative analyses demonstrate that our approach not only reduces training memory to levels as low as those required for inference but also maintains or improves personalization performance compared to existing methods., Comment: NeurIPS 2024

We demonstrate the efficacy of using intermediate representations from a single foundation model to enhance various music downstream tasks. We introduce SoniDo, a music foundation model (MFM) designed to extract hierarchical features from target music samples. By leveraging hierarchical intermediate features, SoniDo constrains the information granularity, leading to improved performance across various downstream tasks including both understanding and generative tasks. We specifically evaluated this approach on representative tasks such as music tagging, music transcription, music source separation, and music mixing. Our results reveal that the features extracted from foundation models provide valuable enhancements in training downstream task models. This highlights the capability of using features extracted from music foundation models as a booster for downstream tasks. Our approach not only benefits existing task-specific models but also supports music downstream tasks constrained by data scarcity. This paves the way for more effective and accessible music processing solutions., Comment: 41 pages with 14 figures

To automatically detect Anterior Mediastinum Lesions (AMLs) in the Anterior Mediastinum (AM), the primary requirement will be an automatic segmentation model specifically designed for the AM. The prevalence of AML is extremely low, making it challenging to conduct screening research similar to lung cancer screening. Retrospectively reviewing chest CT scans over a specific period to investigate the prevalence of AML requires substantial time. Therefore, developing an Artificial Intelligence (AI) model to find location of AM helps radiologist to enhance their ability to manage workloads and improve diagnostic accuracy for AMLs. In this paper, we introduce a U-shaped structure network to segment AM. Two attention mechanisms were used for maintaining long-range dependencies and localization. In order to have the potential of Multi-Head Self-Attention (MHSA) and a lightweight network, we designed a parallel MHSA named Wide-MHSA (W-MHSA). Maintaining long-range dependencies is crucial for segmentation when we upsample feature maps. Therefore, we designed a Dilated Depth-Wise Parallel Path connection (DDWPP) for this purpose. In order to design a lightweight architecture, we introduced an expanding convolution block and combine it with the proposed W-MHSA for feature extraction in the encoder part of the proposed U-shaped network. The proposed network was trained on 2775 AM cases, which obtained an average Dice Similarity Coefficient (DSC) of 87.83%, mean Intersection over Union (IoU) of 79.16%, and Sensitivity of 89.60%. Our proposed architecture exhibited superior segmentation performance compared to the most advanced segmentation networks, such as Trans Unet, Attention Unet, Res Unet, and Res Unet++.

Controlling the effects of photoexcited polarons in transition metal oxides can enable the long timescale charge separation necessary for renewable energy applications as well as controlling new quantum phases through dynamically tunable electron-phonon coupling. In previously studied transition metal oxides, polaron formation is facilitated by a photoexcited ligand-to-metal charge transfer (LMCT). When the polaron is formed, oxygen atoms move away from iron centers, which increases carrier localization at the metal center and decreases charge hopping. Studies of yttrium iron garnet and erbium iron oxide have suggested that strong electron and spin correlations can modulate photoexcited polaron formation. To understand the interplay between strong spin and electronic correlations in highly polar materials, we studied gadolinium iron oxide (GdFeO3), which selectively forms photoexcited polarons through an Fe-O-Fe superexchange interaction. Excitation-wavelength-dependent transient extreme ultraviolet (XUV) spectroscopy selectively excites LMCT and metal-to-metal charge transfer transitions (MMCT). The LMCT transition suppresses photoexcited polaron formation due to dominant Hubbard interactions, while MMCT transitions result in photoexcited polaron formation within ~373+/-137 fs due to enhanced superexchange interactions. Ab initio theory demonstrates that both electron and hole polarons localize on iron centers following MMCT. In addition to understanding how strong electronic and spin correlations can control strong electron-phonon coupling, these experiments separately measure electron and hole polaron interactions on neighboring metal centers for the first time, providing insight into a large range of charge-transfer and Mott-Hubbard insulators., Comment: 29 pages, 14 figures