Search

Large language models encapsulate knowledge and have demonstrated superior performance on various natural language processing tasks. Recent studies have localized this knowledge to specific model parameters, such as the MLP weights in intermediate layers. This study investigates the differences between entity and relational knowledge through knowledge editing. Our findings reveal that entity and relational knowledge cannot be directly transferred or mapped to each other. This result is unexpected, as logically, modifying the entity or the relation within the same knowledge triplet should yield equivalent outcomes. To further elucidate the differences between entity and relational knowledge, we employ causal analysis to investigate how relational knowledge is stored in pre-trained models. Contrary to prior research suggesting that knowledge is stored in MLP weights, our experiments demonstrate that relational knowledge is also significantly encoded in attention modules. This insight highlights the multifaceted nature of knowledge storage in language models, underscoring the complexity of manipulating specific types of knowledge within these models., Comment: CIKM 2024

Training social event detection models through federated learning (FedSED) aims to improve participants' performance on the task. However, existing federated learning paradigms are inadequate for achieving FedSED's objective and exhibit limitations in handling the inherent heterogeneity in social data. This paper proposes a personalized federated learning framework with a dual aggregation mechanism for social event detection, namely DAMe. We present a novel local aggregation strategy utilizing Bayesian optimization to incorporate global knowledge while retaining local characteristics. Moreover, we introduce a global aggregation strategy to provide clients with maximum external knowledge of their preferences. In addition, we incorporate a global-local event-centric constraint to prevent local overfitting and ``client-drift''. Experiments within a realistic simulation of a natural federated setting, utilizing six social event datasets spanning six languages and two social media platforms, along with an ablation study, have demonstrated the effectiveness of the proposed framework. Further robustness analyses have shown that DAMe is resistant to injection attacks., Comment: CIKM 2024

Kagome metals are a unique class of quantum materials characterized by their distinct atomic lattice arrangement, featuring interlocking triangles and expansive hexagonal voids. These lattice structures impart exotic properties, including superconductivity, interaction-driven topological many-body phenomena, and magnetism, among others. The kagome metal CsM3Te5 (where M = Ti, Zr, or Hf) exhibits both superconductivity and nontrivial topological electronic properties, offering a promising platform for exploring topological superconductivity. This study employs first-principles density functional theory calculations to systematically analyze the elastic, mechanical, vibrational, thermodynamic, and electronic properties of CsM3Te5 (M = Ti, Zr, Hf). Our calculations reveal that the studied compounds - CsTi3Te5, CsZr3Te5, and CsHf3Te5 - are ductile metals with elastic properties akin to the hexagonal Bi and Sb, with average elastic constants, including a bulk modulus of 27 GPa, a shear modulus of 11 GPa, and Young's modulus of 29 GPa. We observe peculiar dispersionless, flat, phonon branches in the vibrational spectra of these metals. Additionally, we thoroughly analyze the symmetries of the zone-center phonon eigenvectors and predict vibrational fingerprints of the Raman- and infrared-active phonon modes. The analysis of thermodynamic properties reveals the Einstein temperature for CsTi3Te5, CsZr3Te5, and CsHf3Te5 to be 66, 54, and 53 K, respectively. Our orbital-decomposed electronic structure calculations reveal significant in-plane steric interactions and multiple Dirac band crossings near the Fermi level. We further investigate the role of spin-orbit coupling effect on the studied properties. This theoretical investigation sheds light on the intriguing quantum behaviour of kagome metals and provides valuable insights for their potential applications in future technologies., Comment: 10 pages, 8 figures

Large Language Models (LLMs) have revolutionized open-domain dialogue agents but encounter challenges in multi-character role-playing (MCRP) scenarios. To address the issue, we present Neeko, an innovative framework designed for efficient multiple characters imitation. Unlike existing methods, Neeko employs a dynamic low-rank adapter (LoRA) strategy, enabling it to adapt seamlessly to diverse characters. Our framework breaks down the role-playing process into agent pre-training, multiple characters playing, and character incremental learning, effectively handling both seen and unseen roles. This dynamic approach, coupled with distinct LoRA blocks for each character, enhances Neeko's adaptability to unique attributes, personalities, and speaking patterns. As a result, Neeko demonstrates superior performance in MCRP over most existing methods, offering more engaging and versatile user interaction experiences. Code and data are available at https://github.com/weiyifan1023/Neeko.

Knowledge Editing (KE) for modifying factual knowledge in Large Language Models (LLMs) has been receiving increasing attention. However, existing knowledge editing methods are entity-centric, and it is unclear whether this approach is suitable for a relation-centric perspective. To address this gap, this paper constructs a new benchmark named RaKE, which focuses on Relation based Knowledge Editing. In this paper, we establish a suite of innovative metrics for evaluation and conduct comprehensive experiments involving various knowledge editing baselines. We notice that existing knowledge editing methods exhibit the potential difficulty in their ability to edit relations. Therefore, we further explore the role of relations in factual triplets within the transformer. Our research results confirm that knowledge related to relations is not only stored in the FFN network but also in the attention layers. This provides experimental support for future relation-based knowledge editing methods., Comment: Work in progress

Table-based question answering (TableQA) is an important task in natural language processing, which requires comprehending tables and employing various reasoning ways to answer the questions. This paper introduces TableQAKit, the first comprehensive toolkit designed specifically for TableQA. The toolkit designs a unified platform that includes plentiful TableQA datasets and integrates popular methods of this task as well as large language models (LLMs). Users can add their datasets and methods according to the friendly interface. Also, pleasantly surprised using the modules in this toolkit achieves new SOTA on some datasets. Finally, \tableqakit{} also provides an LLM-based TableQA Benchmark for evaluating the role of LLMs in TableQA. TableQAKit is open-source with an interactive interface that includes visual operations, and comprehensive data for ease of use., Comment: Work in progress

Fact verification aims to automatically probe the veracity of a claim based on several pieces of evidence. Existing works are always engaging in accuracy improvement, let alone explainability, a critical capability of fact verification systems. Constructing an explainable fact verification system in a complex multi-hop scenario is consistently impeded by the absence of a relevant, high-quality dataset. Previous datasets either suffer from excessive simplification or fail to incorporate essential considerations for explainability. To address this, we present EXFEVER, a pioneering dataset for multi-hop explainable fact verification. With over 60,000 claims involving 2-hop and 3-hop reasoning, each is created by summarizing and modifying information from hyperlinked Wikipedia documents. Each instance is accompanied by a veracity label and an explanation that outlines the reasoning path supporting the veracity classification. Additionally, we demonstrate a novel baseline system on our EX-FEVER dataset, showcasing document retrieval, explanation generation, and claim verification, and validate the significance of our dataset. Furthermore, we highlight the potential of utilizing Large Language Models in the fact verification task. We hope our dataset could make a significant contribution by providing ample opportunities to explore the integration of natural language explanations in the domain of fact verification., Comment: ACL 2024 findings paper

Large language models (LLMs) have shown nearly saturated performance on many natural language processing (NLP) tasks. As a result, it is natural for people to believe that LLMs have also mastered abilities such as time understanding and reasoning. However, research on the temporal sensitivity of LLMs has been insufficiently emphasized. To fill this gap, this paper constructs Multiple Sensitive Factors Time QA (MenatQA), which encompasses three temporal factors (scope factor, order factor, counterfactual factor) with total 2,853 samples for evaluating the time comprehension and reasoning abilities of LLMs. This paper tests current mainstream LLMs with different parameter sizes, ranging from billions to hundreds of billions. The results show most LLMs fall behind smaller temporal reasoning models with different degree on these factors. In specific, LLMs show a significant vulnerability to temporal biases and depend heavily on the temporal information provided in questions. Furthermore, this paper undertakes a preliminary investigation into potential improvement strategies by devising specific prompts and leveraging external tools. These approaches serve as valuable baselines or references for future research endeavors., Comment: Accepted to EMNLP 2023 Findings

Answering multi-hop questions over hybrid factual knowledge from the given text and table (TextTableQA) is a challenging task. Existing models mainly adopt a retriever-reader framework, which have several deficiencies, such as noisy labeling in training retriever, insufficient utilization of heterogeneous information over text and table, and deficient ability for different reasoning operations. In this paper, we propose a three-stage TextTableQA framework S3HQA, which comprises of retriever, selector, and reasoner. We use a retriever with refinement training to solve the noisy labeling problem. Then, a hybrid selector considers the linked relationships between heterogeneous data to select the most relevant factual knowledge. For the final stage, instead of adapting a reading comprehension module like in previous methods, we employ a generation-based reasoner to obtain answers. This includes two approaches: a row-wise generator and an LLM prompting generator~(first time used in this task). The experimental results demonstrate that our method achieves competitive results in the few-shot setting. When trained on the full dataset, our approach outperforms all baseline methods, ranking first on the HybridQA leaderboard., Comment: ACL 2023

Hybrid question answering (HybridQA) over the financial report contains both textual and tabular data, and requires the model to select the appropriate evidence for the numerical reasoning task. Existing methods based on encoder-decoder framework employ a expression tree-based decoder to solve numerical reasoning problems. However, encoders rely more on Machine Reading Comprehension (MRC) methods, which take table serialization and text splicing as input, damaging the granularity relationship between table and text as well as the spatial structure information of table itself. In order to solve these problems, the paper proposes a Multi-View Graph (MVG) Encoder to take the relations among the granularity into account and capture the relations from multiple view. By utilizing MVGE as a module, we constuct Tabular View, Relation View and Numerical View which aim to retain the original characteristics of the hybrid data. We validate our model on the publicly available table-text hybrid QA benchmark (TAT-QA) and outperform the state-of-the-art model.

We compute the $p$-adic densities of points with a given splitting type along a (generically) finite map, analogous to the classical Chebotarev theorem over number fields and function fields. Under some mild hypotheses, we prove that these densities satisfy a functional equation in the size of the residue field. This functional equation is a direct reflection of Poincar\'e duality in \'etale cohomology. As a consequence, we prove a conjecture of Bhargava, Cremona, Fisher, and Gajovi\'c on factorization densities of p-adic polynomials. The key tool is the notion of admissible pairs associated to a group, which we use as an invariant of the inertia and decomposition action of a local field on the fibers of the finite map. We compute the splitting densities by M\"obius inverting certain p-adic integrals along the poset of admissible pairs. The conjecture on factorization densities follows immediately for tamely ramified primes from our general results. We reduce the complete conjecture (including the wild primes) to the existence of an explicit "Tate-type" resolution of the "resultant locus" over the integers and complete the proof of the conjecture by constructing this resolution., Comment: The final version, proving the complete conjecture on factorization densities including the wild case

An adaptive implicit-explicit (IMEX) BDF2 scheme is investigated on generalized SAV approach for the Cahn-Hilliard equation by combining with Fourier spectral method in space. It is proved that the modified energy dissipation law is unconditionally preserved at discrete levels. Under a mild ratio restriction, i.e., \Ass{1}: $0

Recent realizations of ultrathin freestanding perovskite oxides offer a unique platform to probe novel properties in two-dimensional oxides. Here, we observed a giant flexoelectric response in freestanding BiFeO3 and SrTiO3 in their bent state arising from strain gradients up to 4x10e7/m, suggesting a promising approach for realizing extremely large polarizations. Additionally, a substantial reversible change in thickness was discovered in bent freestanding BiFeO3, which implies an unusual bending-expansion/shrinkage and thickness-dependence Poisson's ratios in this ferroelectric membrane that has never been seen before in crystalline materials. Our theoretical modeling reveals that this unprecedented flexural deformation within the membrane is attributable to a flexoelectricity-piezoelectricity interplay. The finding unveils intriguing nanoscale electromechanical properties and provides guidance for their practical applications in flexible nanoelectromechanical systems., Comment: 19 pages, 4 figures