Search

Text-to-SQL is a subtask in semantic parsing that has seen rapid progress with the evolution of Large Language Models (LLMs). However, LLMs face challenges due to hallucination issues and a lack of domain-specific database knowledge(such as table schema and cell values). As a result, they can make errors in generating table names, columns, and matching values to the correct columns in SQL statements. This paper introduces a method of knowledge injection to enhance LLMs' ability to understand schema contents by incorporating prior knowledge. This approach improves their performance in Text-to-SQL tasks. Experimental results show that pre-training LLMs on domain-specific database knowledge and fine-tuning them on downstream Text-to-SQL tasks significantly improves the Execution Match (EX) and Exact Match (EM) metrics across various models. This effectively reduces errors in generating column names and matching values to the columns. Furthermore, the knowledge-injected models can be applied to many downstream Text-to-SQL tasks, demonstrating the generalizability of the approach presented in this paper., Comment: This paper has been accepted by ECAI 2024

To extend the context length of Transformer-based large language models (LLMs) and improve comprehension capabilities, we often face limitations due to computational resources and bounded memory storage capacity. This work introduces a method called Recurrent Context Compression (RCC), designed to efficiently expand the context window length of LLMs within constrained storage space. We also investigate the issue of poor model responses when both instructions and context are compressed in downstream tasks, and propose an instruction reconstruction method to mitigate this problem. We validated the effectiveness of our approach on multiple tasks, achieving a compression rate of up to 32x on text reconstruction tasks with a BLEU4 score close to 0.95, and nearly 100\% accuracy on a passkey retrieval task with a sequence length of 1M. Finally, our method demonstrated competitive performance in long-text question-answering tasks compared to non-compressed methods, while significantly saving storage resources in long-text inference tasks. Our code, models, and demo are available at https://github.com/WUHU-G/RCC_Transformer

The observation of a superconducting phase, an intertwined insulating phase, and a continuous transition between the two at a commensurate filling of $\nu=1$ in bilayers of twisted WSe$_2$ (tWSe$_2$) at $\theta=3.65^0$ raises a number of intriguing questions about the origin of this phenomenology. Starting with a simplified three-orbital model of tWSe$_2$, including an on-site and nearest-neighbor density-density interactions, as well as a chiral-exchange interaction, we discuss the possibility of a displacement-field induced direct superconductor to quantum spin-liquid Mott insulator transition at $\nu=1$ using parton mean-field theory. We discuss the nature of these correlated insulators, their expected evolution with the displacement-field, and their phenomenological properties. Further experiments will likely help unravel the mysteries tied to this fascinating experimental platform., Comment: 5 pages, 3 figures

Chain-of-thought (CoT) prompting can guide language models to engage in complex multi-step reasoning. The quality of provided demonstrations significantly impacts the success of downstream inference tasks. While existing automated methods prioritize accuracy and semantics in these demonstrations, we show that the underlying reasoning patterns play a more crucial role in such tasks. In this paper, we propose Pattern-Aware CoT, a prompting method that considers the diversity of demonstration patterns. By incorporating patterns such as step length and reasoning process within intermediate steps, PA-CoT effectively mitigates the issue of bias induced by demonstrations and enables better generalization to diverse scenarios. We conduct experiments on nine reasoning benchmark tasks using two open-source LLMs. The results show that our method substantially enhances reasoning performance and exhibits robustness to errors. The code will be made publicly available.

Multi-orbital electronic models hosting a non-trivial band-topology in the regime of strong electronic interactions are an ideal playground for exploring a host of complex phenomenology. We consider here a sign-problem-free and time-reversal symmetric model with isolated topological (chern) bands involving both spin and valley degrees of freedom in the presence of a class of repulsive electronic interactions. Using a combination of numerically exact quantum Monte Carlo computations and analytical field-theoretic considerations we analyze the phase-diagram as a function of the flat-band filling, temperature and relative interaction strength. The low-energy physics is described in terms of a set of intertwined orders -- a spin-valley hall (SVH) insulator and a spin-singlet superconductor (SC). Our low-temperature phase diagram can be understood in terms of an effective SO(4) pseudo-spin non-linear sigma model. Our work paves the way for building more refined and minimal models of realistic materials, including moir\'e systems, to study the universal aspects of competing insulating phases and superconductivity in the presence of non-trivial band-topology., Comment: 13 pages, 12 figures; published version

Bosonic collective modes are ubiquitous in metals, but over a wide range of energy and momenta suffer from Landau damping, decaying into the continuum of particle-hole excitations. Here we point out that interactions can suppress this decay, protecting a finite fraction of the total spectral weight associated with the collective mode, e.g. a plasmon. The underlying mechanism is level repulsion between a discrete mode and the continuum. We demonstrate the effect using a number of simplified models of strongly correlated Fermi-liquid metals, including a ``solvable" random flavor model in the large$-N$ limit. We discuss in detail the possibility of observing such an avoided decay for plasmons in (moir\'e) graphene-like systems., Comment: 13 pages, 6 figures

Recent spectroscopic measurements in a number of strongly correlated metals that exhibit non-Fermi liquid like properties have observed evidence of anomalous frequency and momentum-dependent charge-density fluctuations. Specifically, in the strange metallic regime of the cuprate superconductors, there is a featureless particle-hole continuum exhibiting unusual power-laws, and experiments suggest that the plasmon mode decays into this continuum in a manner that is distinct from the expectations of conventional Fermi liquid theory. Inspired by these new experimental developments, we address the nature of low-energy collective modes and the particle-hole continua for different "solvable" lattice models of non-Fermi liquids that host a critical Fermi surface -- a sharp electronic Fermi surface without any low-energy electronic quasiparticles. We scrutinize theoretically the possible existence of a long-lived zero-sound mode, which is renormalized to the plasma frequency in the presence of long-ranged coulomb interactions, and its decay into the continuum over a wide range of frequencies and momenta. Quite remarkably, some of the models analyzed here can account for certain aspects of the universal experimental observations, that clearly lie beyond the purview of standard Fermi liquid theory., Comment: 35 pages, 12 figures (includes appendices)

We report the study of spontaneous magnetization (i.e., spin-polarization) for time-reversal symmetry (TRS)-breaking superconductors with unitary pairing potentials, in the absence of external magnetic fields or Zeeman fields. Spin-singlet ($\Delta_s$) and spin-triplet ($\Delta_t$) pairings can coexist in superconductors whose crystal structure lacks inversion symmetry. The TRS can be spontaneously broken once a relative phase of $\pm\pi/2$ is developed, forming a TRS-breaking unitary pairing state ($\Delta_s\pm i\Delta_t$). We demonstrate that such unitary pairing could give rise to spontaneous spin-polarization with the help of spin-orbit coupling. Our result provides an alternative explanation to the TRS breaking, beyond the current understanding of such phenomena in the noncentrosymmetric superconductors. The experimental results of Zr$_3$Ir and CaPtAs are also discussed in the view of our theory., Comment: 4+5 pages, 3+1 figures, comments are welcome

The Berry phase picture provides important insights into the electronic properties of condensed matter systems. The intrinsic anomalous Hall (AH) effect can be understood as a consequence of non-zero Berry curvature in momentum space. The realization of the quantum anomalous Hall effect provided conclusive evidence for the intrinsic mechanism of the AH effect in magnetic topological insulators (TIs). Here we fabricated magnetic TI/TI heterostructures and found both the magnitude and sign of the AH effect in the magnetic TI layer can be altered by tuning the TI thickness and/or the electric gate voltage. The sign change of the AH effect with increasing TI thickness is attributed to the charge transfer across the TI and magnetic TI layers, consistent with first-principles calculations. By fabricating the magnetic TI/TI/magnetic TI sandwich heterostructures with different dopants, we created an artificial topological Hall (TH) effect-like feature in Hall traces. This artificial TH effect is induced by the superposition of two AH effects with opposite signs instead of the formation of chiral spin textures in the samples. Our study provides a new route to engineer the Berry curvature in magnetic topological materials that may lead to potential technological applications., Comment: 28 pages, 5 figures. Accepted version