Search

Large language models (LLMs) and large multimodal models (LMMs) have shown great potential in automating complex tasks like web browsing and gaming. However, their ability to generalize across diverse applications remains limited, hindering broader utility. To address this challenge, we present OSCAR: Operating System Control via state-Aware reasoning and Re-planning. OSCAR is a generalist agent designed to autonomously navigate and interact with various desktop and mobile applications through standardized controls, such as mouse and keyboard inputs, while processing screen images to fulfill user commands. OSCAR translates human instructions into executable Python code, enabling precise control over graphical user interfaces (GUIs). To enhance stability and adaptability, OSCAR operates as a state machine, equipped with error-handling mechanisms and dynamic task re-planning, allowing it to efficiently adjust to real-time feedback and exceptions. We demonstrate OSCAR's effectiveness through extensive experiments on diverse benchmarks across desktop and mobile platforms, where it transforms complex workflows into simple natural language commands, significantly boosting user productivity. Our code will be open-source upon publication., Comment: Work in progress

Long time series forecasting aims to utilize historical information to forecast future states over extended horizons. Traditional RNN-based series forecasting methods struggle to effectively address long-term dependencies and gradient issues in long time series problems. Recently, SegRNN has emerged as a leading RNN-based model tailored for long-term series forecasting, demonstrating state-of-the-art performance while maintaining a streamlined architecture through innovative segmentation and parallel decoding techniques. Nevertheless, SegRNN has several limitations: its fixed segmentation disrupts data continuity and fails to effectively leverage information across different segments, the segmentation strategy employed by SegRNN does not fundamentally address the issue of information loss within the recurrent structure. To address these issues, we propose the ISMRNN method with three key enhancements: we introduce an implicit segmentation structure to decompose the time series and map it to segmented hidden states, resulting in denser information exchange during the segmentation phase. Additionally, we incorporate residual structures in the encoding layer to mitigate information loss within the recurrent structure. To extract information more effectively, we further integrate the Mamba architecture to enhance time series information extraction. Experiments on several real-world long time series forecasting datasets demonstrate that our model surpasses the performance of current state-of-the-art models.

Studying the generalized Hamming weights of linear codes is a significant research area within coding theory, as it provides valuable structural information about the codes and plays a crucial role in determining their performance in various applications. However, determining the generalized Hamming weights of linear codes, particularly their weight hierarchy, is generally a challenging task. In this paper, we focus on investigating the generalized Hamming weights of three classes of linear codes over finite fields. These codes are constructed by different defining sets. By analysing the intersections between the definition sets and the duals of all $r$-dimensional subspaces, we get the inequalities on the sizes of these intersections. Then constructing subspaces that reach the upper bounds of these inequalities, we successfully determine the complete weight hierarchies of these codes.

The effectiveness of anomaly signal detection can be significantly undermined by the inherent uncertainty of relying on one specified model. Under the framework of model average methods, this paper proposes a novel criterion to select the weights on aggregation of multiple models, wherein the focal loss function accounts for the classification of extremely imbalanced data. This strategy is further integrated into Random Forest algorithm by replacing the conventional voting method. We have evaluated the proposed method on benchmark datasets across various domains, including network intrusion. The findings indicate that our proposed method not only surpasses the model averaging with typical loss functions but also outstrips common anomaly detection algorithms in terms of accuracy and robustness.

For partial differential equations on domains of arbitrary shapes, existing works of neural operators attempt to learn a mapping from geometries to solutions. It often requires a large dataset of geometry-solution pairs in order to obtain a sufficiently accurate neural operator. However, for many industrial applications, e.g., engineering design optimization, it can be prohibitive to satisfy the requirement since even a single simulation may take hours or days of computation. To address this issue, we propose reference neural operators (RNO), a novel way of implementing neural operators, i.e., to learn the smooth dependence of solutions on geometric deformations. Specifically, given a reference solution, RNO can predict solutions corresponding to arbitrary deformations of the referred geometry. This approach turns out to be much more data efficient. Through extensive experiments, we show that RNO can learn the dependence across various types and different numbers of geometry objects with relatively small datasets. RNO outperforms baseline models in accuracy by a large lead and achieves up to 80% error reduction.

Large language models (LLMs) are primarily evaluated by overall performance on various text understanding and generation tasks. However, such a paradigm fails to comprehensively differentiate the fine-grained language and cognitive skills, rendering the lack of sufficient interpretation to LLMs' capabilities. In this paper, we present FAC$^2$E, a framework for Fine-grAined and Cognition-grounded LLMs' Capability Evaluation. Specifically, we formulate LLMs' evaluation in a multi-dimensional and explainable manner by dissociating the language-related capabilities and the cognition-related ones. Besides, through extracting the intermediate reasoning from LLMs, we further break down the process of applying a specific capability into three sub-steps: recalling relevant knowledge, utilizing knowledge, and solving problems. Finally, FAC$^2$E evaluates each sub-step of each fine-grained capability, providing a two-faceted diagnosis for LLMs. Utilizing FAC$^2$E, we identify a common shortfall in knowledge utilization among models and propose a straightforward, knowledge-enhanced method to mitigate this issue. Our results not only showcase promising performance enhancements but also highlight a direction for future LLM advancements., Comment: Accepted at EMNLP 2024 main conference

BCH codes are an interesting class of cyclic codes due to their efficient encoding and decoding algorithms. In the past sixty years, a lot of progress on the study of BCH codes has been made, but little is known about the properties of their duals. Recently, in order to study the duals of BCH codes and the lower bounds on their minimum distances, a new concept called dually-BCH code was proposed by authors in \cite{GDL21}. In this paper, the lower bounds on the minimum distances of the duals of narrow-sense BCH codes with length $\frac{q^m-1}{\lambda}$ over $\mathbb{F}_q$ are developed, where $\lambda$ is a positive integer satisfying $\lambda\, |\, q-1$, or $\lambda=q^s-1$ and $s\, |\,m$. In addition, the sufficient and necessary conditions in terms of the designed distances for these codes being dually-BCH codes are presented. Many considered codes in \cite{GDL21} and \cite{Wang23} are the special cases of the codes showed in this paper. Our lower bounds on the minimum distances of the duals of BCH codes include the bounds stated in \cite{GDL21} as a special case. Several examples show that the lower bounds are good in some cases.

The study of the generalized Hamming weight of linear codes is a significant research topic in coding theory as it conveys the structural information of the codes and determines their performance in various applications. However, determining the generalized Hamming weights of linear codes, especially the weight hierarchy, is generally challenging. In this paper, we investigate the generalized Hamming weights of a class of linear code $\C$ over $\bF_q$, which is constructed from defining sets. These defining sets are either special simplicial complexes or their complements in $\bF_q^m$. We determine the complete weight hierarchies of these codes by analyzing the maximum or minimum intersection of certain simplicial complexes and all $r$-dimensional subspaces of $\bF_q^m$, where $1\leq r\leq {\rm dim}_{\bF_q}(\C)$.

The ability to modulate the spin-orbit (SO) interaction is crucial for engineering a wide range of spintronics-based quantum devices, extending from state-of-the-art data storage to materials for quantum computing. The use of proximity-induced effects for this purpose may become the mainstream approach, whereas their experimental verification using angle-resolved photoelectron spectroscopy (ARPES) has so far been elusive. Here, using the advantages of soft-X-ray ARPES on its probing depth and intrinsic resolution in three-dimensional momentum k, we identify a distinct modulation of the SO interaction in a van der Waals semiconductor (MoSe2) proximitized to a high-Z metal (Pb), and measure its variation through the k-space. The strong SO field from Pb boosts the SO splitting by up to 30% at the H-point of the bulk Brillouin zone, the spin-orbit hotspot of MoSe2. Tunability of the splitting via the Pb thickness allows its tailoring to particular applications in emerging quantum devices.

The high-density data storage technology aims to design high-capacity storage at a relatively low cost. In order to achieve this goal, symbol-pair codes were proposed by Cassuto and Blaum \cite{CB10,CB11} to handle channels that output pairs of overlapping symbols. Such a channel is called symbol-pair read channel, which introduce new concept called symbol-pair weight and minimum symbol-pair distance. In this paper, we consider the parameters of two classes of reducible cyclic codes under the symbol-pair metric. Based on the theory of cyclotomic numbers and Gaussian period over finite fields, we show the possible symbol-pair weights of these codes. Their minimum symbol-pair distances are twice the minimum Hamming distances under some conditions. Moreover, we obtain some three symbol-pair weight codes and determine their symbol-pair weight distribution. A class of MDS symbol-pair codes is also established. Among other results, we determine the values of some generalized cyclotomic numbers.

We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. Existing question generation systems widely differentiate questions by $\textit{literal}$ information such as question words and answer types to generate semantically relevant questions for a given context. However, they rarely consider the $\textit{comprehension}$ nature of questions, i.e. the different comprehension capabilities embodied by different questions. In comparison, our $\texttt{SkillQG}$ is able to tailor a fine-grained assessment and improvement to the capabilities of question answering models built on it. Specifically, we first frame the comprehension type of questions based on a hierarchical skill-based schema, then formulate $\texttt{SkillQG}$ as a skill-conditioned question generator. Furthermore, to improve the controllability of generation, we augment the input text with question focus and skill-specific knowledge, which are constructed by iteratively prompting the pre-trained language models. Empirical results demonstrate that $\texttt{SkillQG}$ outperforms baselines in terms of quality, relevance, and skill-controllability while showing a promising performance boost in downstream question answering task., Comment: Accepted at ACL 2023 Findings

We previously proposed contextual spelling correction (CSC) to correct the output of end-to-end (E2E) automatic speech recognition (ASR) models with contextual information such as name, place, etc. Although CSC has achieved reasonable improvement in the biasing problem, there are still two drawbacks for further accuracy improvement. First, due to information limitation in text only hypothesis or weak performance of ASR model on rare domains, the CSC model may fail to correct phrases with similar pronunciation or anti-context cases where all biasing phrases are not present in the utterance. Second, there is a discrepancy between the training and inference of CSC. The bias list in training is randomly selected but in inference there may be more similarity between ground truth phrase and other phrases. To solve above limitations, in this paper we propose an improved non-autoregressive (NAR) spelling correction model for contextual biasing in E2E neural transducer-based ASR systems to improve the previous CSC model from two perspectives: Firstly, we incorporate acoustics information with an external attention as well as text hypotheses into CSC to better distinguish target phrase from dissimilar or irrelevant phrases. Secondly, we design a semantic aware data augmentation schema in training phrase to reduce the mismatch between training and inference to further boost the biasing accuracy. Experiments show that the improved method outperforms the baseline ASR+Biasing system by as much as 20.3% relative name recall gain and achieves stable improvement compared to the previous CSC method over different bias list name coverage ratio., Comment: Accepted by ICASSP 2023

The generalized Hamming weight of linear codes is a natural generalization of the minimum Hamming distance. They convey the structural information of a linear code and determine its performance in various applications, and have become one of important research topics in coding theory. Recently, Li (IEEE Trans. Inf. Theory, 67(1): 124-129, 2021) and Li and Li (Discrete Math., 345: 112718, 2022) obtained the complete weight hierarchy of linear codes from a quadratic form over a finite field of odd characteristic by analysis of the solutions of the restricted quadratic equation in its subspace. In this paper, we further determine the complete weight hierarchy of linear codes from a quadratic form over a finite field of even characteristic by carefully studying the behavior of the quadratic form on the subspaces of this field and its dual space, and complement the results of Li and Li.

BCH codes and their dual codes are two special subclasses of cyclic codes and are the best linear codes in many cases. A lot of progress on the study of BCH cyclic codes has been made, but little is known about the minimum distances of the duals of BCH codes. Recently, a new concept called dually-BCH code was introduced to investigate the duals of BCH codes and the lower bounds on their minimum distances in \cite{GDL21}. For a prime power $q$ and an integer $m \ge 4$, let $n=\frac{q^m-1}{q+1}$ \ ($m$ even), or $n=\frac{q^m-1}{q-1}$ \ ($q>2$). In this paper, some sufficient and necessary conditions in terms of the designed distance will be given to ensure that the narrow-sense BCH codes of length $n$ are dually-BCH codes, which extended the results in \cite{GDL21}. Lower bounds on the minimum distances of their dual codes are developed for $n=\frac{q^m-1}{q+1}$ \ ($m$ even). As byproducts, we present the largest coset leader $\delta_1$ modulo $n$ being of two types, which proves a conjecture in \cite{WLP19} and partially solves an open problem in \cite{Li2017}. We also investigate the parameters of the narrow-sense BCH codes of length $n$ with design distance $\delta_1$. The BCH codes presented in this paper have good parameters in general.

Constacyclic codes over finite fields are a family of linear codes and contain cyclic codes as a subclass. Constacyclic codes are related to many areas of mathematics and outperform cyclic codes in several aspects. Hence, constacyclic codes are of theoretical importance. On the other hand, constacyclic codes are important in practice, as they have rich algebraic structures and may have efficient decoding algorithms. In this paper, two classes of constacyclic codes are constructed using a general construction of constacyclic codes with cyclic codes. The first class of constacyclic codes is motivated by the punctured Dilix cyclic codes and the second class is motivated by the punctured generalised Reed-Muller codes. The two classes of constacyclic codes contain optimal linear codes. The parameters of the two classes of constacyclic codes are analysed and some open problems are presented in this paper.

Interplay between coding theory and combinatorial $t$-designs has been a hot topic for many years for combinatorialists and coding theorists. Some infinite families of cyclic codes supporting infinite families of $3$-designs have been constructed in the past 50 years. However, no infinite family of negacyclic codes supporting an infinite family of $3$-designs has been reported in the literature. This is the main motivation of this paper. Let $q=p^m$, where $p$ is an odd prime and $m \geq 2$ is an integer. The objective of this paper is to present an infinite family of cyclic codes over $\gf(q)$ supporting an infinite family of $3$-designs and two infinite families of negacyclic codes over $\gf(q^2)$ supporting two infinite families of $3$-designs. The parameters and the weight distributions of these codes are determined. The subfield subcodes of these negacyclic codes over $\gf(q)$ are studied. Three infinite families of almost MDS codes are also presented. A constacyclic code over GF($4$) supporting a $4$-design and six open problems are also presented in this paper.

Negacyclic BCH codes are a subclass of neagcyclic codes and are the best linear codes in many cases. However, there have been very few results on negacyclic BCH codes. Let $q$ be an odd prime power and $m$ be a positive integer. The objective of this paper is to study negacyclic BCH codes with length $\frac{q^m-1}{2}$ and $\frac{q^m+1}{2}$ over the finite field $\mathbf(q)$ and analyse their parameters. The negacyclic BCH codes presented in this paper have good parameters in general, and contain many optimal linear codes. For certain $q$ and $m$, compared with cyclic codes with the same dimension and length, the negacyclic BCH codes presented in this paper have a larger minimum distance in some cases.

In this paper, several families of irreducible constacyclic codes over finite fields and their duals are studied. The weight distributions of these irreducible constacyclic codes and the parameters of their duals are settled. Several families of irreducible constacyclic codes with a few weights and several families of optimal constacyclic codes are constructed. As by-products, a family of $[2n, (n-1)/2, d \geq 2(\sqrt{n}+1)]$ irreducible cyclic codes over $\gf(q)$ and a family of $[(q-1)n, (n-1)/2, d \geq (q-1)(\sqrt{n}+1)]$ irreducible cyclic codes over $\gf(q)$ are presented, where $n$ is a prime such that $\ord_n(q)=(n-1)/2$. The results in this paper complement earlier works on irreducible constacyclic and cyclic codes over finite fields.

This paper proposes a new "decompose-and-edit" paradigm for the text-based speech insertion task that facilitates arbitrary-length speech insertion and even full sentence generation. In the proposed paradigm, global and local factors in speech are explicitly decomposed and separately manipulated to achieve high speaker similarity and continuous prosody. Specifically, we proposed to represent the global factors by multiple tokens, which are extracted by cross-attention operation and then injected back by link-attention operation. Due to the rich representation of global factors, we manage to achieve high speaker similarity in a zero-shot manner. In addition, we introduce a prosody smoothing task to make the local prosody factor context-aware and therefore achieve satisfactory prosody continuity. We further achieve high voice quality with an adversarial training stage. In the subjective test, our method achieves state-of-the-art performance in both naturalness and similarity. Audio samples can be found at https://ydcustc.github.io/retrieverTTS-demo/., Comment: 5 pages, 1 figure, 3 tables. Accepted by Interspeech 2022

Insertion-deletion codes (insdel codes for short) are used for correcting synchronization errors in communications, and in other many interesting fields such as DNA storage, date analysis, race-track memory error correction and language processing, and have recently gained a lot of attention. To determine the insdel distances of linear codes is a very challenging problem. The half-Singleton bound on the insdel distances of linear codes due to Cheng-Guruswami-Haeupler-Li is a basic upper bound on the insertion-deletion error-correcting capabilities of linear codes. On the other hand the natural direct upper bound $d_I(\mathcal C) \leq 2d_H(\mathcal C)$ is valid for any insdel code. In this paper, for a linear insdel code $\mathcal C$ we propose a strict half-Singleton upper bound $d_I(\mathcal C) \leq 2(n-2k+1)$ if $\mathcal C$ does not contain the codeword with all 1s, and a stronger direct upper bound $d_I(\mathcal C) \leq 2(d_H(\mathcal C)-t)$ under a weak condition, where $t\geq 1$ is a positive integer determined by the generator matrix. We also give optimal linear insdel codes attaining our strict half-Singleton bound and direct upper bound, and show that the code length of optimal binary linear insdel codes with respect to the (strict) half-Singleton bound is about twice the dimension. Interestingly explicit optimal linear insdel codes attaining the (strict) half-Singleton bound, with the code length being independent of the finite field size, are given.

Existing metrics for assessing question generation not only require costly human reference but also fail to take into account the input context of generation, rendering the lack of deep understanding of the relevance between the generated questions and input contexts. As a result, they may wrongly penalize a legitimate and reasonable candidate question when it (i) involves complicated reasoning with the context or (ii) can be grounded by multiple evidences in the context. In this paper, we propose $\textbf{QRelScore}$, a context-aware $\underline{\textbf{Rel}}$evance evaluation metric for $\underline{\textbf{Q}}$uestion Generation. Based on off-the-shelf language models such as BERT and GPT2, QRelScore employs both word-level hierarchical matching and sentence-level prompt-based generation to cope with the complicated reasoning and diverse generation from multiple evidences, respectively. Compared with existing metrics, our experiments demonstrate that QRelScore is able to achieve a higher correlation with human judgments while being much more robust to adversarial samples., Comment: 16 pages, 7 figures

Machine Reading Comprehension (MRC) reveals the ability to understand a given text passage and answer questions based on it. Existing research works in MRC rely heavily on large-size models and corpus to improve the performance evaluated by metrics such as Exact Match ($EM$) and $F_1$. However, such a paradigm lacks sufficient interpretation to model capability and can not efficiently train a model with a large corpus. In this paper, we argue that a deep understanding of model capabilities and data properties can help us feed a model with appropriate training data based on its learning status. Specifically, we design an MRC capability assessment framework that assesses model capabilities in an explainable and multi-dimensional manner. Based on it, we further uncover and disentangle the connections between various data properties and model performance. Finally, to verify the effectiveness of the proposed MRC capability assessment framework, we incorporate it into a curriculum learning pipeline and devise a Capability Boundary Breakthrough Curriculum (CBBC) strategy, which performs a model capability-based training to maximize the data value and improve training efficiency. Extensive experiments demonstrate that our approach significantly improves performance, achieving up to an 11.22% / 8.71% improvement of $EM$ / $F_1$ on MRC tasks., Comment: Accepted by ACL 2022

Contextual biasing is an important and challenging task for end-to-end automatic speech recognition (ASR) systems, which aims to achieve better recognition performance by biasing the ASR system to particular context phrases such as person names, music list, proper nouns, etc. Existing methods mainly include contextual LM biasing and adding bias encoder into end-to-end ASR models. In this work, we introduce a novel approach to do contextual biasing by adding a contextual spelling correction model on top of the end-to-end ASR system. We incorporate contextual information into a sequence-to-sequence spelling correction model with a shared context encoder. Our proposed model includes two different mechanisms: autoregressive (AR) and non-autoregressive (NAR). We propose filtering algorithms to handle large-size context lists, and performance balancing mechanisms to control the biasing degree of the model. We demonstrate the proposed model is a general biasing solution which is domain-insensitive and can be adopted in different scenarios. Experiments show that the proposed method achieves as much as 51% relative word error rate (WER) reduction over ASR system and outperforms traditional biasing methods. Compared to the AR solution, the proposed NAR model reduces model size by 43.2% and speeds up inference by 2.1 times., Comment: Accepted in IEEE Transactions on Audio, Speech and Language Processing (TASLP)