Search

Quantum machine learning (QML) is a rapidly growing field that combines quantum computing principles with traditional machine learning. It seeks to revolutionize machine learning by harnessing the unique capabilities of quantum mechanics and employs machine learning techniques to advance quantum computing research. This paper introduces quantum computing for the machine learning paradigm, where variational quantum circuits (VQC) are used to develop QML architectures on noisy intermediate-scale quantum (NISQ) devices. We discuss machine learning for the quantum computing paradigm, showcasing our recent theoretical and empirical findings. In particular, we delve into future directions for studying QML, exploring the potential industrial impacts of QML research., Comment: In submission

Quantum Machine Learning (QML) offers tremendous potential but is currently limited by the availability of qubits. We introduce an innovative approach that utilizes pre-trained neural networks to enhance Variational Quantum Circuits (VQC). This technique effectively separates approximation error from qubit count and removes the need for restrictive conditions, making QML more viable for real-world applications. Our method significantly improves parameter optimization for VQC while delivering notable gains in representation and generalization capabilities, as evidenced by rigorous theoretical analysis and extensive empirical testing on quantum dot classification tasks. Moreover, our results extend to applications such as human genome analysis, demonstrating the broad applicability of our approach. By addressing the constraints of current quantum hardware, our work paves the way for a new era of advanced QML applications, unlocking the full potential of quantum computing in fields such as machine learning, materials science, medicine, mimetics, and various interdisciplinary areas., Comment: In submission

Construction of a general-purpose post-recognition error corrector poses a crucial question: how can we most effectively train a model on a large mixture of domain datasets? The answer would lie in learning dataset-specific features and digesting their knowledge in a single model. Previous methods achieve this by having separate correction language models, resulting in a significant increase in parameters. In this work, we present Mixture-of-Experts as a solution, highlighting that MoEs are much more than a scalability tool. We propose a Multi-Task Correction MoE, where we train the experts to become an ``expert'' of speech-to-text, language-to-text and vision-to-text datasets by learning to route each dataset's tokens to its mapped expert. Experiments on the Open ASR Leaderboard show that we explore a new state-of-the-art performance by achieving an average relative $5.0$% WER reduction and substantial improvements in BLEU scores for speech and translation tasks. On zero-shot evaluation, NeKo outperforms GPT-3.5 and Claude-Opus with $15.5$% to $27.6$% relative WER reduction in the Hyporadise benchmark. NeKo performs competitively on grammar and post-OCR correction as a multi-task model., Comment: NeKo work has been done in June 2024. NeKo LMs will be open source on https://huggingface.co/nvidia under the MIT license

Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spoken language model that comprehends a wide range of natural language instructions is critical for bridging communication gaps and facilitating more intuitive interactions. However, the absence of a comprehensive evaluation benchmark poses a significant challenge. We present Dynamic-SUPERB Phase-2, an open and evolving benchmark for the comprehensive evaluation of instruction-based universal speech models. Building upon the first generation, this second version incorporates 125 new tasks contributed collaboratively by the global research community, expanding the benchmark to a total of 180 tasks, making it the largest benchmark for speech and audio evaluation. While the first generation of Dynamic-SUPERB was limited to classification tasks, Dynamic-SUPERB Phase-2 broadens its evaluation capabilities by introducing a wide array of novel and diverse tasks, including regression and sequence generation, across speech, music, and environmental audio. Evaluation results indicate that none of the models performed well universally. SALMONN-13B excelled in English ASR, while WavLLM demonstrated high accuracy in emotion recognition, but current models still require further innovations to handle a broader range of tasks. We will soon open-source all task data and the evaluation pipeline.

The streaming instability (SI) is a leading mechanism for concentrating solid particles into regions dense enough to form planetesimals. Its efficiency in clumping particles depends primarily on the dimensionless stopping time ($\tau_s$, a proxy for particle size) and dust-to-gas surface density ratio ($Z$). Previous simulations identified a critical $Z$ ($Z_{\rm{crit}}$) above which strong clumping occurs, where particle densities exceed the Hill density (thus satisfying a condition for gravitational collapse), over a wide range of $\tau_s$. These works found that for $\tau_s \leq 0.01$, $Z_{\rm{crit}}$ was above the ISM value $(\sim 0.01)$. In this work, we reexamine the clumping threshold using 2D axisymmetric, stratified simulations at high resolution and with relatively large (compared to many previous simulations) domain sizes. Our main results are as follows: First, when $\tau_s = 0.01$, strong clumping occurs even at $Z \lesssim 0.01$, lower than $Z_{\rm{crit}}$ found in all previous studies. Consequently, we revise a previously published fit to the $Z_{\rm{crit}}$ curve to account for this updated $Z_{\rm{crit}}$. Second, higher resolution results in a thicker dust layer, which may result from other instabilities manifesting, such as the vertical shearing streaming instability. Third, despite this thicker layer, higher resolution can lead to strong clumping even with lower midplane dust-to-gas density ratios (which results from the thicker particle layer) so long as $Z \gtrsim Z_{\rm{crit}}$. Our results demonstrate the efficiency of the SI in clumping small particles at $Z \sim 0.01$, which is a significant refinement of the conditions for planetesimal formation by the SI., Comment: 25 pages, 12 figures, submitted to the Astrophysical Journal

Scaling laws offer valuable insights into the design of time series foundation models (TSFMs). However, previous research has largely focused on the scaling laws of TSFMs for in-distribution (ID) data, leaving their out-of-distribution (OOD) scaling behavior and the influence of model architectures less explored. In this work, we examine two common TSFM architectures, encoder-only and decoder-only Transformers, and investigate their scaling behavior on both ID and OOD data. These models are trained and evaluated across varying parameter counts, compute budgets, and dataset sizes. Our experiments reveal that the log-likelihood loss of TSFMs exhibits similar scaling behavior in both OOD and ID settings. We further compare the scaling properties across different architectures, incorporating two state-of-the-art TSFMs as case studies, showing that model architecture plays a significant role in scaling. The encoder-only Transformers demonstrate better scalability than the decoder-only Transformers, while the architectural enhancements in the two advanced TSFMs primarily improve ID performance but reduce OOD scalability. While scaling up TSFMs is expected to drive performance breakthroughs, the lack of a comprehensive understanding of TSFM scaling laws has hindered the development of a robust framework to guide model scaling. We fill this gap in this work by synthesizing our findings and providing practical guidelines for designing and scaling larger TSFMs with enhanced model capabilities.

Accurate short-term passenger flow prediction of subway stations plays a vital role in enabling subway station personnel to proactively address changes in passenger volume. Despite existing literature in this field, there is a lack of research on effectively integrating features from different periods, particularly intra-period and inter-period features, for subway station passenger flow prediction. In this paper, we propose a novel model called \textbf{M}uti \textbf{P}eriod \textbf{S}patial \textbf{T}emporal \textbf{N}etwork \textbf{MPSTN}) that leverages features from different periods by transforming one-dimensional time series data into two-dimensional matrices based on periods. The folded matrices exhibit structural characteristics similar to images, enabling the utilization of image processing techniques, specifically convolutional neural networks (CNNs), to integrate features from different periods. Therefore, our MPSTN model incorporates a CNN module to extract temporal information from different periods and a graph neural network (GNN) module to integrate spatial information from different stations. We compared our approach with various state-of-the-art methods for spatiotemporal data prediction using a publicly available dataset and achieved minimal prediction errors. The code for our model is publicly available in the following repository: https://github.com/xiannanhuang/MPSTN, Comment: accepted by TRBAM 2024

Short-term traffic volume prediction is crucial for intelligent transportation system and there are many researches focusing on this field. However, most of these existing researches concentrated on refining model architecture and ignored amount of training data. Therefore, there remains a noticeable gap in thoroughly exploring the effect of augmented dataset, especially extensive historical data in training. In this research, two datasets containing taxi and bike usage spanning over eight years in New York were used to test such effects. Experiments were conducted to assess the precision of models trained with data in the most recent 12, 24, 48, and 96 months. It was found that the training set encompassing 96 months, at times, resulted in diminished accuracy, which might be owing to disparities between historical traffic patterns and present ones. An analysis was subsequently undertaken to discern potential sources of inconsistent patterns, which may include both covariate shift and concept shift. To address these shifts, we proposed an innovative approach that aligns covariate distributions using a weighting scheme to manage covariate shift, coupled with an environment aware learning method to tackle the concept shift. Experiments based on real word datasets demonstrate the effectiveness of our method which can significantly decrease testing errors and ensure an improvement in accuracy when training with large-scale historical data. As far as we know, this work is the first attempt to assess the impact of contiguously expanding training dataset on the accuracy of traffic prediction models. Besides, our training method is able to be incorporated into most existing short-term traffic prediction models and make them more suitable for long term historical training dataset., Comment: submitted to IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to $52.06\%$ of the original LoRA for GPT-2 with a slight performance gain of $0.75\%$, and to $16.84\%$ for Gemma-2, with a marginal performance improvement of $0.07\%$. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware., Comment: 21 pages, 6 figures

The Hadamard product of tensor train (TT) tensors is one of the most fundamental nonlinear operations in scientific computing and data analysis. Due to its tendency to significantly increase TT ranks, the Hadamard product presents a major computational challenge in TT tensor-based algorithms. Therefore, it is essential to develop recompression algorithms that mitigate the effects of this rank increase. Existing recompression algorithms require an explicit representation of the Hadamard product, resulting in high computational and storage complexity. In this work, we propose the Hadamard avoiding TT recompression (HaTT) algorithm. Leveraging the structure of the Hadamard product in TT tensors and its Hadamard product-free property, the overall complexity of the HaTT algorithm is significantly lower than that of existing TT recompression algorithms. This is validated through complexity analysis and several numerical experiments., Comment: 17 pages, 9 figures

In this study, we aim to explore Multitask Speech Language Model (SpeechLM) efficient inference via token reduction. Unlike other modalities such as vision or text, speech has unique temporal dependencies, making previous efficient inference works on other modalities not directly applicable. Furthermore, methods for efficient SpeechLM inference on long sequence and sparse signals remain largely unexplored. Then we propose FastAdaSP, a weighted token merging framework specifically designed for various speech-related tasks to improve the trade-off between efficiency and performance. Experimental results on WavLLM and Qwen-Audio show that our method achieves the state-of-the-art (SOTA) efficiency-performance trade-off compared with other baseline methods. Specifically, FastAdaSP achieved 7x memory efficiency and 1.83x decoding throughput without any degradation on tasks like Emotion Recognition (ER) and Spoken Question Answering (SQA). The code will be available at https://github.com/yichen14/FastAdaSP, Comment: EMNLP 2024 Industry Track

Recent end-to-end speech language models (SLMs) have expanded upon the capabilities of large language models (LLMs) by incorporating pre-trained speech models. However, these SLMs often undergo extensive speech instruction-tuning to bridge the gap between speech and text modalities. This requires significant annotation efforts and risks catastrophic forgetting of the original language capabilities. In this work, we present a simple yet effective automatic process for creating speech-text pair data that carefully injects speech paralinguistic understanding abilities into SLMs while preserving the inherent language capabilities of the text-based LLM. Our model demonstrates general capabilities for speech-related tasks without the need for speech instruction-tuning data, achieving impressive performance on Dynamic-SUPERB and AIR-Bench-Chat benchmarks. Furthermore, our model exhibits the ability to follow complex instructions derived from LLMs, such as specific output formatting and chain-of-thought reasoning. Our approach not only enhances the versatility and effectiveness of SLMs but also reduces reliance on extensive annotated datasets, paving the way for more efficient and capable speech understanding systems., Comment: Submitted to ICASSP 2025

Thanks to the rapidly increasing time-domain facilities, we are entering a golden era of research on gamma-ray bursts (GRBs). In this Letter, we report our observations of GRB 240529A with the Burst Optical Observer and Transient Exploring System, the 1.5-meter telescope at Observatorio Sierra Nevada, the 2.5-meter Wide Field Survey Telescope of China, the Large Binocular Telescope, and the Telescopio Nazionale Galileo. The prompt emission of GRB 240529A shows two comparable energetic episodes separated by a quiescence time of roughly 400 s. Combining all available data on the GRB Coordinates Network, we reveal the simultaneous apparent X-ray plateau and optical re-brightening around $10^3-10^4$ s after the burst. Rather than the energy injection from the magnetar as widely invoked for similar GRBs, the multi-wavelength emissions could be better explained as two shocks launched from the central engine separately. The optical peak time and our numerical modeling suggest that the initial bulk Lorentz factor of the later shock is roughly 50, which indicates that the later jet should be accretion-driven and have a higher mass loading than a typical one. The quiescence time between the two prompt emission episodes may be caused by the transition between different accretion states of a central magnetar or black hole, or the fall-back accretion process. A sample of similar bursts with multiple emission episodes in the prompt phase and sufficient follow-up could help to probe the underlying physics of GRB central engines., Comment: Resubmitted to ApJL after addressing the referee's comments; comments are welcome

Large language models are typically fine-tuned to align with human preferences, but tuning large models is computationally intensive and complex. In this work, we introduce $\textit{Integrated Value Guidance}$ (IVG), a method that uses implicit and explicit value functions to guide language model decoding at token and chunk-level respectively, efficiently aligning large language models purely at inference time. This approach circumvents the complexities of direct fine-tuning and outperforms traditional methods. Empirically, we demonstrate the versatility of IVG across various tasks. In controlled sentiment generation and summarization tasks, our method significantly improves the alignment of large models using inference-time guidance from $\texttt{gpt2}$-based value functions. Moreover, in a more challenging instruction-following benchmark AlpacaEval 2.0, we show that both specifically tuned and off-the-shelf value functions greatly improve the length-controlled win rates of large models against $\texttt{gpt-4-turbo}$ (e.g., $19.51\% \rightarrow 26.51\%$ for $\texttt{Mistral-7B-Instruct-v0.2}$ and $25.58\% \rightarrow 33.75\%$ for $\texttt{Mixtral-8x7B-Instruct-v0.1}$ with Tulu guidance)., Comment: EMNLP 2024 Findings

Ensuring that the outputs of neural networks satisfy specific constraints is crucial for applying neural networks to real-life decision-making problems. In this paper, we consider making a batch of neural network outputs satisfy bounded and general linear constraints. We first reformulate the neural network output projection problem as an entropy-regularized linear programming problem. We show that such a problem can be equivalently transformed into an unconstrained convex optimization problem with Lipschitz continuous gradient according to the duality theorem. Then, based on an accelerated gradient descent algorithm with numerical performance enhancement, we present our architecture, GLinSAT, to solve the problem. To the best of our knowledge, this is the first general linear satisfiability layer in which all the operations are differentiable and matrix-factorization-free. Despite the fact that we can explicitly perform backpropagation based on automatic differentiation mechanism, we also provide an alternative approach in GLinSAT to calculate the derivatives based on implicit differentiation of the optimality condition. Experimental results on constrained traveling salesman problems, partial graph matching with outliers, predictive portfolio allocation and power system unit commitment demonstrate the advantages of GLinSAT over existing satisfiability layers. Our implementation is available at \url{https://github.com/HunterTracer/GLinSAT}., Comment: This paper has been accepted by 2024 Advances in Neural Information Processing Systems. The reviews and comments can be found in https://openreview.net/forum?id=m1PVjNHvtP

Annotating and recognizing speech emotion using prompt engineering has recently emerged with the advancement of Large Language Models (LLMs), yet its efficacy and reliability remain questionable. In this paper, we conduct a systematic study on this topic, beginning with the proposal of novel prompts that incorporate emotion-specific knowledge from acoustics, linguistics, and psychology. Subsequently, we examine the effectiveness of LLM-based prompting on Automatic Speech Recognition (ASR) transcription, contrasting it with ground-truth transcription. Furthermore, we propose a Revise-Reason-Recognize prompting pipeline for robust LLM-based emotion recognition from spoken language with ASR errors. Additionally, experiments on context-aware learning, in-context learning, and instruction tuning are performed to examine the usefulness of LLM training schemes in this direction. Finally, we investigate the sensitivity of LLMs to minor prompt variations. Experimental results demonstrate the efficacy of the emotion-specific prompts, ASR error correction, and LLM training schemes for LLM-based emotion recognition. Our study aims to refine the use of LLMs in emotion recognition and related domains.

Large language models (LLMs) have demonstrated remarkable advancements in language understanding and generation. Building on the success of text-based LLMs, recent research has adapted these models to use speech embeddings for prompting, resulting in Speech-LLM models that exhibit strong performance in automatic speech recognition (ASR) and automatic speech translation (AST). In this work, we propose a novel approach to leverage ASR transcripts as prompts for AST in a Speech-LLM built on an encoder-decoder text LLM. The Speech-LLM model consists of a speech encoder and an encoder-decoder structure Megatron-T5. By first decoding speech to generate ASR transcripts and subsequently using these transcripts along with encoded speech for prompting, we guide the speech translation in a two-step process like chain-of-thought (CoT) prompting. Low-rank adaptation (LoRA) is used for the T5 LLM for model adaptation and shows superior performance to full model fine-tuning. Experimental results show that the proposed CoT prompting significantly improves AST performance, achieving an average increase of 2.4 BLEU points across 6 En->X or X->En AST tasks compared to speech prompting alone. Additionally, compared to a related CoT prediction method that predicts a concatenated sequence of ASR and AST transcripts, our method performs better by an average of 2 BLEU points.

The multifractal critical phase (MCP) fundamentally differs from extended and localized phases, exhibiting delocalized distributions in both position and momentum spaces. The investigation on the MCP has largely focused on one-dimensional quasiperiodic systems. Here, we introduce a two-dimensional (2D) quasiperiodic model with a MCP. We present its phase diagram and investigate the characteristics of the 2D system's MCP in terms of wave packet diffusion and transport based on this model. We further investigate the movement of the phase boundary induced by the introduction of next-nearest-neighbor hopping by calculating the fidelity susceptibility. Finally, we consider how to realize our studied model in superconducting circuits. Our work opens the door to exploring MCP in 2D systems., Comment: To appear in Phys. Rev. A

Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model. To explore new capabilities in language modeling for speech processing, we introduce the generative speech transcription error correction (GenSEC) challenge. This challenge comprises three post-ASR language modeling tasks: (i) post-ASR transcription correction, (ii) speaker tagging, and (iii) emotion recognition. These tasks aim to emulate future LLM-based agents handling voice-based interfaces while remaining accessible to a broad audience by utilizing open pretrained language models or agent-based APIs. We also discuss insights from baseline evaluations, as well as lessons learned for designing future evaluations., Comment: IEEE SLT 2024. The initial draft version has been done in December 2023. Post-ASR Text Processing and Understanding Community and LlaMA-7B pre-training correction model: https://huggingface.co/GenSEC-LLM/SLT-Task1-Llama2-7b-HyPo-baseline

This paper presents a novel approach to distributed pose estimation in the multi-agent system based on an invariant Kalman filter with covariance intersection. Our method models uncertainties using Lie algebra and applies object-level observations within Lie groups, which have practical application value. We integrate covariance intersection to handle estimates that are correlated and use the invariant Kalman filter for merging independent data sources. This strategy allows us to effectively tackle the complex correlations of cooperative localization among agents, ensuring our estimates are neither too conservative nor overly confident. Additionally, we examine the consistency and stability of our algorithm, providing evidence of its reliability and effectiveness in managing multi-agent systems.

Recently, Greenfeld and Tao disprove the conjecture that translational tilings of a single tile can always be periodic [Ann. Math. 200(2024), 301-363]. In another paper [to appear in J. Eur. Math. Soc.], they also show that if the dimension $n$ is part of the input, the translational tiling for subsets of $\mathbb{Z}^n$ with one tile is undecidable. These two results are very strong pieces of evidence for the conjecture that translational tiling of $\mathbb{Z}^n$ with a monotile is undecidable, for some fixed $n$. This paper shows that translational tiling of the $3$-dimensional space with a set of $5$ polycubes is undecidable. By introducing a technique that lifts a set of polycubes and its tiling from $3$-dimensional space to $4$-dimensional space, we manage to show that translational tiling of the $4$-dimensional space with a set of $4$ tiles is undecidable. This is a step towards the attempt to settle the conjecture of the undecidability of translational tiling of the $n$-dimensional space with a monotile, for some fixed $n$., Comment: 18 pages, 20 figures

We introduce a new framework to study the dynamics of open quantum systems with linearly coupled Gaussian baths. Our approach replaces the continuous bath with an auxiliary discrete set of pseudomodes with dissipative dynamics, but we further relax the complete positivity requirement in the Lindblad master equation and formulate a quasi-Lindblad pseudomode theory. We show that this quasi-Lindblad pseudomode formulation directly leads to a representation of the bath correlation function in terms of a complex weighted sum of complex exponentials, an expansion that is known to be rapidly convergent in practice and thus leads to a compact set of pseudomodes. The pseudomode representation is not unique and can differ by a gauge choice. When the global dynamics can be simulated exactly, the system dynamics is unique and independent of the specific pseudomode representation. However, the gauge choice may affect the stability of the global dynamics, and we provide an analysis of why and when the global dynamics can retain stability despite losing positivity. We showcase the performance of this formulation across various spectral densities in both bosonic and fermionic problems, finding significant improvements over conventional pseudomode formulations., Comment: 13 pages, 6 figures (main text); 8 pages, 1 figure (Supplementary Material)

With the strong representational power of large language models (LLMs), generative error correction (GER) for automatic speech recognition (ASR) aims to provide semantic and phonetic refinements to address ASR errors. This work explores how LLM-based GER can enhance and expand the capabilities of Japanese language processing, presenting the first GER benchmark for Japanese ASR with 0.9-2.6k text utterances. We also introduce a new multi-pass augmented generative error correction (MPA GER) by integrating multiple system hypotheses on the input side with corrections from multiple LLMs on the output side and then merging them. To the best of our knowledge, this is the first investigation of the use of LLMs for Japanese GER, which involves second-pass language modeling on the output transcriptions generated by the ASR system (e.g., N-best hypotheses). Our experiments demonstrated performance improvement in the proposed methods of ASR quality and generalization both in SPREDS-U1-ja and CSJ data.

Recent years have witnessed the remarkable success of recommendation systems (RSs) in alleviating the information overload problem. As a new paradigm of RSs, session-based recommendation (SR) specializes in users' short-term preference capture and aims to provide a more dynamic and timely recommendation based on the ongoing interacted actions. In this survey, we will give a comprehensive overview of the recent works on SR. First, we clarify the definitions of various SR tasks and introduce the characteristics of session-based recommendation against other recommendation tasks. Then, we summarize the existing methods in two categories: sequential neural network based methods and graph neural network (GNN) based methods. The standard frameworks and technical are also introduced. Finally, we discuss the challenges of SR and new research directions in this area.

Anion doping offers immense potential for tailoring material properties, but achieving precise control over anion incorporation remains a challenge due to complex synthesis processes and limitations in local dopant detection. Here, we investigate the F-ion intercalation within an infinite layer NdNiO2+x/SrTiO3 superlattice film using a two-step synthesis approach. We employ advanced four-dimensional scanning transmission electron microscopy (4D-STEM) coupled with electron energy loss spectroscopy to map the F distribution and its impact on the atomic and electronic structure. Our observations reveal a striking geometric reconstruction of the infinite layer structure upon fluorination, resulting in a more distorted orthorhombic phase compared to the pristine perovskite. Notably, F-ion intercalation occurs primarily at the apical sites of the polyhedron, with some occupation of basal sites in localized regions. This process leads to the formation of two distinct domains within the nickelate layer, reflecting a competition between polyhedral distortion and geometric frustration-induced neodymium (Nd) displacement near domain interfaces. Interestingly, we observe an anomalous structural distortion where basal site anions are displaced in the same direction as Nd atoms, potentially linked to the partial basal site F-ion occupation. This coexistence of diverse structural distortions signifies a locally disordered F-ion distribution within the infinite layer structure with distinct F-ion configurations. These findings provide crucial insights into understanding and manipulating anion doping at the atomic level, paving the way for the development of novel materials with precisely controlled functionalities.

Power dispatch is essential for providing stable, cost-effective, and eco-friendly electricity to society. However, traditional methods falter as power systems grow in scale and complexity, struggling with multitasking, swift problem-solving, and human-machine collaboration. This paper introduces GAIA, the pioneering Large Language Model (LLM) tailored for power dispatch tasks. We have developed a novel dataset construction technique that harnesses a range of data sources to fine-tune GAIA for optimal performance in this domain. This approach streamlines LLM training, allowing for the seamless integration of multidimensional data in power system management. Additionally, we have crafted specialized prompt strategies to boost GAIA's input-output efficiency in dispatch scenarios. When evaluated on the ElecBench benchmark, GAIA surpasses the baseline model LLaMA2 on multiple metrics. In practical applications, GAIA has demonstrated its ability to enhance decision-making processes, improve operational efficiency, and facilitate better human-machine interactions in power dispatch operations. This paper expands the application of LLMs to power dispatch and validates their practical utility, paving the way for future innovations in this field.

In challenging low light and adverse weather conditions,thermal vision algorithms,especially object detection,have exhibited remarkable potential,contrasting with the frequent struggles encountered by visible vision algorithms. Nevertheless,the efficacy of thermal vision algorithms driven by deep learning models remains constrained by the paucity of available training data samples. To this end,this paper introduces a novel approach termed the edge guided conditional diffusion model. This framework aims to produce meticulously aligned pseudo thermal images at the pixel level,leveraging edge information extracted from visible images. By utilizing edges as contextual cues from the visible domain,the diffusion model achieves meticulous control over the delineation of objects within the generated images. To alleviate the impacts of those visible-specific edge information that should not appear in the thermal domain,a two-stage modality adversarial training strategy is proposed to filter them out from the generated images by differentiating the visible and thermal modality. Extensive experiments on LLVIP demonstrate ECDM s superiority over existing state-of-the-art approaches in terms of image generation quality., Comment: accepted by ACM MM 2024/ACM MM24

This paper focuses on the undecidability of translational tiling of $n$-dimensional space $\mathbb{Z}^n$ with a set of $k$ tiles. It is known that tiling $\mathbb{Z}^2$ with translated copies with a set of $8$ tiles is undecidable. Greenfeld and Tao gave strong evidence in a series of works that for sufficiently large dimension $n$, the translational tiling problem for $\mathbb{Z}^n$ might be undecidable for just one tile. This paper shows the undecidability of translational tiling of $\mathbb{Z}^3$ with a set of $6$ tiles.

The legal landscape encompasses a wide array of lawsuit types, presenting lawyers with challenges in delivering timely and accurate information to clients, particularly concerning critical aspects like potential imprisonment duration or financial repercussions. Compounded by the scarcity of legal experts, there's an urgent need to enhance the efficiency of traditional legal workflows. Recent advances in deep learning, especially Large Language Models (LLMs), offer promising solutions to this challenge. Leveraging LLMs' mathematical reasoning capabilities, we propose a novel approach integrating LLM-based methodologies with specially designed prompts to address precision requirements in legal Artificial Intelligence (LegalAI) applications. The proposed work seeks to bridge the gap between traditional legal practices and modern technological advancements, paving the way for a more accessible, efficient, and equitable legal system. To validate this method, we introduce a curated dataset tailored to precision-oriented LegalAI tasks, serving as a benchmark for evaluating LLM-based approaches. Extensive experimentation confirms the efficacy of our methodology in generating accurate numerical estimates within the legal domain, emphasizing the role of LLMs in streamlining legal processes and meeting the evolving demands of LegalAI., Comment: The paper has been accepted by the 33rd ACM International Conference on Information and Knowledge Management (CIKM) in 2024

Building upon the strength of modern large language models (LLMs), generative error correction (GEC) has emerged as a promising paradigm that can elevate the performance of modern automatic speech recognition (ASR) systems. One representative approach is to leverage in-context learning to prompt LLMs so that a better hypothesis can be generated by the LLMs based on a carefully-designed prompt and an $N$-best list of hypotheses produced by ASR systems. However, it is yet unknown whether the existing prompts are the most effective ones for the task of post-ASR error correction. In this context, this paper first explores alternative prompts to identify an initial set of effective prompts, and then proposes to employ an evolutionary prompt optimization algorithm to refine the initial prompts. Evaluations results on the CHiME-4 subset of the Task $1$ of the SLT $2024$ GenSEC challenge show the effectiveness and potential of the proposed algorithms., Comment: in submission

The many-body mobility edge (MBME) in energy, which separates thermal states from many-body localization (MBL) states, is a critical yet controversial concept in many-body systems. Here we examine the quasiperiodic $t_1-t_2$ model that features a mobility edge. With the addition of nearest-neighbor interactions, we demonstrate the potential existence of a MBME. Then we investigate the impact of a type of bond dissipation on the many-body system by calculating the steady-state density matrix and analyzing the transport behavior, and demonstrate that dissipation can cause the system to predominantly occupy either the thermal region or the MBL region, irrespective of the initial state. Finally, we discuss the effects of increasing system size. Our results indicate that dissipation can induce transitions between thermal and MBL states, providing a new approach for experimentally determining the existence of the MBME.

Understanding the dynamics of nonequilibrium quantum many-body systems is an important research topic in a wide range of fields across condensed matter physics, quantum optics, and high-energy physics. However, numerical studies of large-scale nonequilibrium phenomena in realistic materials face serious challenges due to intrinsic high-dimensionality of quantum many-body problems. The nonequilibrium properties of many-body systems can be described by the dynamics of the Green's function of the system, whose time evolution is given by a high-dimensional system of integro-differential equations, known as the Kadanoff-Baym equations (KBEs). The time-convolution term in KBEs, which needs to be recalculated at each time step, makes it difficult to perform long-time simulations. In this paper, we develop an operator-learning framework based on Recurrent Neural Networks (RNNs) to address this challenge. The proposed framework utilizes RNNs to learn the nonlinear mapping between Green's functions and convolution integrals in KBEs. By using the learned operators as a surrogate model in the KBE solver, we obtain a general machine-learning scheme for predicting the dynamics of nonequilibrium Green's functions. This new methodology reduces the temporal computational complexity from $O(N_t^3)$ to $O(N_t)$, where $N_t$ is the total time steps taken in a simulation, thereby making it possible to study large many-body problems which are currently infeasible with conventional KBE solvers. Through different numerical examples, we demonstrate the effectiveness of the operator-learning based approach in providing accurate predictions of physical observables such as the reduced density matrix and time-resolved photoemission spectra. Moreover, our framework exhibits clear numerical convergence and can be easily parallelized, thereby facilitating many possible further developments and applications.

We present a new method for computing the lowest few eigenvalues and the corresponding eigenvectors of a nuclear many-body Hamiltonian represented in a truncated configuration interaction subspace, i.e., the no-core shell model (NCSM). The method uses the hierarchical structure of the NCSM Hamiltonian to partition the Hamiltonian as the sum of two matrices. The first matrix corresponds to the Hamiltonian represented in a small configuration space, whereas the second is viewed as the perturbation to the first matrix. Eigenvalues and eigenvectors of the first matrix can be computed efficiently. Perturbative corrections to the eigenvectors of the first matrix can be obtained from the solutions of a sequence of linear systems of equations defined in the small configuration space. These correction vectors can be combined with the approximate eigenvectors of the first matrix to construct a subspace from which more accurate approximations of the desired eigenpairs can be obtained. We call this method a Subspace Projection with Perturbative Corrections (SPPC) method. We show by numerical examples that the SPPC method can be more efficient than conventional iterative methods for solving large-scale eigenvalue problems such as the Lanczos, block Lanczos and the locally optimal block preconditioned conjugate gradient (LOBPCG) method. The method can also be combined with other methods to avoid convergence stagnation.

Quantum reinforcement learning utilizes quantum layers to process information within a machine learning model. However, both pure and hybrid quantum reinforcement learning face challenges such as data encoding and the use of quantum computers during the inference stage. We apply the Quantum-Train method to reinforcement learning tasks, called QTRL, training the classical policy network model using a quantum machine learning model with polylogarithmic parameter reduction. This QTRL approach eliminates the data encoding issues of conventional quantum machine learning and reduces the training parameters of the corresponding classical policy network. Most importantly, the training result of the QTRL is a classical model, meaning the inference stage only requires classical computer. This is extremely practical and cost-efficient for reinforcement learning tasks, where low-latency feedback from the policy model is essential., Comment: 6 pages, 1 figure

Accurately describing properties of challenging problems in physical sciences often requires complex mathematical models that are unmanageable to tackle head-on. Therefore, developing reduced dimensionality representations that encapsulate complex correlation effects in many-body systems is crucial to advance the understanding of these complicated problems. However, a numerical evaluation of these predictive models can still be associated with a significant computational overhead. To address this challenge, in this paper, we discuss a combined framework that integrates recent advances in the development of active-space representations of coupled cluster (CC) downfolded Hamiltonians with neural network approaches. The primary objective of this effort is to train neural networks to eliminate the computationally expensive steps required for evaluating hundreds or thousands of Hugenholtz diagrams, which correspond to multidimensional tensor contractions necessary for evaluating a many-body form of downfolded/effective Hamiltonians. Using small molecular systems (the H2O and HF molecules) as examples, we demonstrate that training neural networks employing effective Hamiltonians for a few nuclear geometries of molecules can accurately interpolate/ extrapolate their forms to other geometrical configurations characterized by different intensities of correlation effects. We also discuss differences between effective interactions that define CC downfolded Hamiltonians with those of bare Hamiltonians defined by Coulomb interactions in the active spaces.

Disk vortices, seen in numerical simulations of protoplanetary disks and found observationally in ALMA and VLA images of these objects, are promising sites for planet formation given their pebble trapping abilities. Previous works have shown strong concentration of pebbles in vortices, but gravitational collapse has only been shown in low-resolution, two-dimensional, global models. In this letter, we aim to study the pebble concentration and gravitational collapse of pebble clouds in vortices via high-resolution, three-dimensional, local models. We performed simulations of the dynamics of gas and solids in a local shearing box where the gas is subject to convective overstability, generating a persistent giant vortex. We find that the vortex produces objects of Moon and Mars mass, with mass function of power law $d\ln N/d\ln M=-1.6\pm 0.3$. The protoplanets grow rapidly, doubling in mass in about 5 orbits, following pebble accretion rates. The mass range and mass doubling rate are in broad agreement with previous low resolution global models. We conclude that Mars-mass planetary embryos are the natural outcome of planet formation inside the disk vortices seen in millimeter and radio images of protoplanetary disks., Comment: 9 pages, 5 figures, accepted for publication in ApJ letters

Large language models (LLMs) have enhanced the capacity of vision-language models to caption visual text. This generative approach to image caption enrichment further makes textual captions more descriptive, improving alignment with the visual context. However, while many studies focus on benefits of generative caption enrichment (GCE), are there any negative side effects? We compare standard-format captions and recent GCE processes from the perspectives of "gender bias" and "hallucination", showing that enriched captions suffer from increased gender bias and hallucination. Furthermore, models trained on these enriched captions amplify gender bias by an average of 30.9% and increase hallucination by 59.5%. This study serves as a caution against the trend of making captions more descriptive.

Large language models (LLMs) now support extremely long context windows, but the quadratic complexity of vanilla attention results in significantly long Time-to-First-Token (TTFT) latency. Existing approaches to address this complexity require additional pretraining or finetuning, and often sacrifice model accuracy. In this paper, we first provide both theoretical and empirical foundations for near-lossless sparse attention. We find dynamically capturing head-specific sparse patterns at runtime with low overhead is crucial. To address this, we propose SampleAttention, an adaptive structured and near-lossless sparse attention. Leveraging observed significant sparse patterns, SampleAttention attends to a fixed percentage of adjacent tokens to capture local window patterns, and employs a two-stage query-guided key-value filtering approach, which adaptively select a minimum set of key-values with low overhead, to capture column stripe patterns. Comprehensive evaluations show that SampleAttention can seamlessly replace vanilla attention in off-the-shelf LLMs with nearly no accuracy loss, and reduces TTFT by up to $2.42\times$ compared with FlashAttention.

Direct Preference Optimization (DPO), a standard method for aligning language models with human preferences, is traditionally applied to offline preferences. Recent studies show that DPO benefits from iterative training with online preferences labeled by a trained reward model. In this work, we identify a pitfall of vanilla iterative DPO - improved response quality can lead to increased verbosity. To address this, we introduce iterative length-regularized DPO (iLR-DPO) to penalize response length. Our empirical results show that iLR-DPO can enhance a 7B model to perform on par with GPT-4 without increasing verbosity. Specifically, our 7B model achieves a $50.5\%$ length-controlled win rate against $\texttt{GPT-4 Preview}$ on AlpacaEval 2.0, and excels across standard benchmarks including MT-Bench, Arena-Hard and OpenLLM Leaderboard. These results demonstrate the effectiveness of iterative DPO in aligning language models with human feedback.

Relaxation time plays a crucial role in describing the relaxation processes of quantum systems. We study the effect of a type of bond dissipation on the relaxation time of boundary dissipative systems and find that it can change the scaling of the relaxation time $T_c\sim L^{z}$ from $z=3$ to a value significantly less than $3$. We further reveal that the reason such bond dissipation can significantly reduce the relaxation time is that it can selectively target specific states. For Anderson localized systems, the scaling behavior of the relaxation time changes from an exponential form to a power-law form as the system size varies. This is because the bond dissipation we consider can not only select specific states but also disrupt the localization properties. Our work reveals that in open systems, one type of dissipation can be used to regulate the effects produced by another type of dissipation.

Our goal is to $\textit{efficiently}$ discover a compact set of temporal logic rules to explain irregular events of interest. We introduce a neural-symbolic rule induction framework within the temporal point process model. The negative log-likelihood is the loss that guides the learning, where the explanatory logic rules and their weights are learned end-to-end in a $\textit{differentiable}$ way. Specifically, predicates and logic rules are represented as $\textit{vector embeddings}$, where the predicate embeddings are fixed and the rule embeddings are trained via gradient descent to obtain the most appropriate compositional representations of the predicate embeddings. To make the rule learning process more efficient and flexible, we adopt a $\textit{sequential covering algorithm}$, which progressively adds rules to the model and removes the event sequences that have been explained until all event sequences have been covered. All the found rules will be fed back to the models for a final rule embedding and weight refinement. Our approach showcases notable efficiency and accuracy across synthetic and real datasets, surpassing state-of-the-art baselines by a wide margin in terms of efficiency.

Modern high-stakes systems, such as healthcare or robotics, often generate vast streaming event sequences. Our goal is to design an efficient, plug-and-play tool to elicit logic tree-based explanations from Large Language Models (LLMs) to provide customized insights into each observed event sequence. Built on the temporal point process model for events, our method employs the likelihood function as a score to evaluate generated logic trees. We propose an amortized Expectation-Maximization (EM) learning framework and treat the logic tree as latent variables. In the E-step, we evaluate the posterior distribution over the latent logic trees using an LLM prior and the likelihood of the observed event sequences. LLM provides a high-quality prior for the latent logic trees, however, since the posterior is built over a discrete combinatorial space, we cannot get the closed-form solution. We propose to generate logic tree samples from the posterior using a learnable GFlowNet, which is a diversity-seeking generator for structured discrete variables. The M-step employs the generated logic rules to approximate marginalization over the posterior, facilitating the learning of model parameters and refining the tunable LLM prior parameters. In the online setting, our locally built, lightweight model will iteratively extract the most relevant rules from LLMs for each sequence using only a few iterations. Empirical demonstrations showcase the promising performance and adaptability of our framework.

Compared with the remarkable progress made in parallel numerical solvers of partial differential equations,the development of algorithms for generating unstructured triangular/tetrahedral meshes has been relatively sluggish. In this paper, we propose a novel, consistent parallel advancing front technique (CPAFT) by combining the advancing front technique, the domain decomposition method based on space-filling curves, the distributed forest-of-overlapping-trees approach, and the consistent parallel maximal independent set algorithm. The newly proposed CPAFT algorithm can mathematically ensure that the generated unstructured triangular/tetrahedral meshes are independent of the number of processors and the implementation of domain decomposition. Several numerical tests are conducted to validate the parallel consistency and outstanding parallel efficiency of the proposed algorithm, which scales effectively up to two thousand processors. This is, as far as we know, the first parallel unstructured triangular/tetrahedral mesh generator with scalability to O(1,000) CPU processors.

Large language models are usually fine-tuned to align with human preferences. However, fine-tuning a large language model can be challenging. In this work, we introduce $\textit{weak-to-strong search}$, framing the alignment of a large language model as a test-time greedy search to maximize the log-probability difference between small tuned and untuned models while sampling from the frozen large model. This method serves both as (1) a compute-efficient model up-scaling strategy that avoids directly tuning the large model and as (2) an instance of weak-to-strong generalization that enhances a strong model with weak test-time guidance. Empirically, we demonstrate the flexibility of weak-to-strong search across different tasks. In controlled-sentiment generation and summarization, we use tuned and untuned $\texttt{gpt2}$s to improve the alignment of large models without additional training. Crucially, in a more difficult instruction-following benchmark, AlpacaEval 2.0, we show that reusing off-the-shelf small models (e.g., $\texttt{zephyr-7b-beta}$ and its untuned version) can improve the length-controlled win rates of both white-box and black-box large models against $\texttt{gpt-4-turbo}$ (e.g., $34.4\% \rightarrow 37.9\%$ for $\texttt{Llama-3-70B-Instruct}$ and $16.0\% \rightarrow 20.1\%$ for $\texttt{gpt-3.5-turbo-instruct}$), despite the small models' low win rates $\approx 10.0\%$., Comment: NeurIPS 2024

We propose an unsupervised adaptation framework, Self-TAught Recognizer (STAR), which leverages unlabeled data to enhance the robustness of automatic speech recognition (ASR) systems in diverse target domains, such as noise and accents. STAR is developed for prevalent speech foundation models based on Transformer-related architecture with auto-regressive decoding (e.g., Whisper, Canary). Specifically, we propose a novel indicator that empirically integrates step-wise information during decoding to assess the token-level quality of pseudo labels without ground truth, thereby guiding model updates for effective unsupervised adaptation. Experimental results show that STAR achieves an average of 13.5% relative reduction in word error rate across 14 target domains, and it sometimes even approaches the upper-bound performance of supervised adaptation. Surprisingly, we also observe that STAR prevents the adapted model from the common catastrophic forgetting problem without recalling source-domain data. Furthermore, STAR exhibits high data efficiency that only requires less than one-hour unlabeled data, and seamless generality to alternative large speech models and speech translation tasks. Our code aims to open source to the research communities., Comment: 23 pages, Preprint

Solving the Boltzmann-BGK equation with traditional numerical methods suffers from high computational and memory costs due to the curse of dimensionality. In this paper, we propose a novel accuracy-preserved tensor-train (APTT) method to efficiently solve the Boltzmann-BGK equation. A second-order finite difference scheme is applied to discretize the Boltzmann-BGK equation, resulting in a tensor algebraic system at each time step. Based on the low-rank TT representation, the tensor algebraic system is then approximated as a TT-based low-rank system, which is efficiently solved using the TT-modified alternating least-squares (TT-MALS) solver. Thanks to the low-rank TT representation, the APTT method can significantly reduce the computational and memory costs compared to traditional numerical methods. Theoretical analysis demonstrates that the APTT method maintains the same convergence rate as that of the finite difference scheme. The convergence rate and efficiency of the APTT method are validated by several benchmark test cases.

In recent years, the researches about solving partial differential equations (PDEs) based on artificial neural network have attracted considerable attention. In these researches, the neural network models are usually designed depend on human experience or trial and error. Despite the emergence of several model searching methods, these methods primarily concentrate on optimizing the hyperparameters of fully connected neural network model based on the framework of physics-informed neural networks (PINNs), and the corresponding search spaces are relatively restricted, thereby limiting the exploration of superior models. This article proposes an evolutionary computation method aimed at discovering the PINNs model with higher approximation accuracy and faster convergence rate. In addition to searching the numbers of layers and neurons per hidden layer, this method concurrently explores the optimal shortcut connections between the layers and the novel parametric activation functions expressed by the binary trees. In evolution, the strategy about dynamic population size and training epochs (DPSTE) is adopted, which significantly increases the number of models to be explored and facilitates the discovery of models with fast convergence rate. In experiments, the performance of different models that are searched through Bayesian optimization, random search and evolution is compared in solving Klein-Gordon, Burgers, and Lam\'e equations. The experimental results affirm that the models discovered by the proposed evolutionary computation method generally exhibit superior approximation accuracy and convergence rate, and these models also show commendable generalization performance with respect to the source term, initial and boundary conditions, equation coefficient and computational domain. The corresponding code is available at https://github.com/MathBon/Discover-PINNs-Model.

Theoretical descriptions of non equilibrium dynamics of quantum many-body systems essentially employ either (i) explicit treatments, relying on truncation of the expansion of the many-body wave function, (ii) compressed representations of the many-body wave function, or (iii) evolution of an effective (downfolded) representation through Green's functions. In this work, we select representative cases of each of the methods and address how these complementary approaches capture the dynamics driven by intense field perturbations to non equilibrium states. Under strong driving, the systems are characterized by strong entanglement of the single particle density matrix and natural populations approaching those of a strongly interacting equilibrium system. We generate a representative set of results that are numerically exact and form a basis for critical comparison of the distinct families of methods. We demonstrate that the compressed formulation based on similarity transformed Hamiltonians (coupled cluster approach) is practically exact in weak fields and, hence, weakly or moderately correlated systems. Coupled cluster, however, struggles for strong driving fields, under which the system exhibits strongly correlated behavior, as measured by the von Neumann entropy of the single particle density matrix. The dynamics predicted by Green's functions in the (widely popular) GW approximation are less accurate by improve significantly upon the mean-field results in the strongly driven regime., Comment: 11 pages, 2 figures in main text. 11 page supplemental information with 6 figures

This work aims to study a scalable state-space model (SSM), Mamba, for the speech enhancement (SE) task. We exploit a Mamba-based regression model to characterize speech signals and build an SE system upon Mamba, termed SEMamba. We explore the properties of Mamba by integrating it as the core model in both basic and advanced SE systems, along with utilizing signal-level distances as well as metric-oriented loss functions. SEMamba demonstrates promising results and attains a PESQ score of 3.55 on the VoiceBank-DEMAND dataset. When combined with the perceptual contrast stretching technique, the proposed SEMamba yields a new state-of-the-art PESQ score of 3.69.

Antiferromagnet is a promising candidate for the next generation spintronic devices, benefiting from its ultrafast dynamics and spontaneous zero stray field. However, the understanding of their ultrafast spin behaviors is lacking due to the challenges of controlling/detecting the quenched net magnetization. Unconventional compensated semiconducting antiferromagnets present strong time-reversal symmetry breaking, spin splitting in the momentum space, and suitable bandgap for optical control/detection. Thus, it is a powerful platform to uncover the ultrafast dynamics of antiferromagnets. Here, we show an exotic wavelength-dependent spin dynamic in the unconventional compensated semiconducting antiferromagnet {\alpha}-MnTe via time-resolved quadratic magneto-optical Kerr effect measurement, where the probing photon energy of the laser matches its bandgap. This direct excitation and detection of distinct magnon modes reveal varying spin behaviors and time characteristics in a broad temperature range. It originates from the spins triggered at different bands of electronic structures and is depicted in an energy transfer model among electrons, phonons, and magnons. Our study of exotic optical properties in this unconventional semiconducting antiferromagnet fulfills the missing information of spin evolution in the time domain and paves the way for its utilization in ultrafast spintronic devices.