Search

Existing research extensively incorporates syntactic information, especially dependency trees, to enhance the performance of relation extraction tasks. However, relying solely on dependency information may not fully exploit the rich semantic and syntactic information contained in sentences, and not all information in dependency trees is substantively helpful for relation extraction. Therefore, this paper proposes the Dependent-Position Relation Graph Convolutional network with Hierarchical Attention (DPR-GHA) for relation extraction, a method that integrates dependency relations and position relations into a hierarchical attention mechanism to effectively capture the relations between entities in text. The method aims to capture rich semantic information and enhance the performance of relation extraction. Specifically, we introduce the dependency relations of sentences and position relations of words to model global dependencies and local features, respectively. Subsequently, a novel hierarchical attention mechanism is introduced into the Graph Convolutional Network (GCN), dynamically adjusting the weights between nodes based on the input of the graph convolutional layer. This adaptive information aggregation enables each node to aggregate information adaptively according to its context and the importance of neighboring nodes. The research results on the SemEval-2010 Task 8 and KBP37 datasets thoroughly validate the effectiveness of the proposed model, demonstrating its significant performance advantage in relation extraction tasks. [ABSTRACT FROM AUTHOR]

Objectives The rapid expansion of biomedical literature necessitates automated techniques to discern relationships between biomedical concepts from extensive free text. Such techniques facilitate the development of detailed knowledge bases and highlight research deficiencies. The LitCoin Natural Language Processing (NLP) challenge, organized by the National Center for Advancing Translational Science, aims to evaluate such potential and provides a manually annotated corpus for methodology development and benchmarking. Materials and Methods For the named entity recognition (NER) task, we utilized ensemble learning to merge predictions from three domain-specific models, namely BioBERT, PubMedBERT, and BioM-ELECTRA, devised a rule-driven detection method for cell line and taxonomy names and annotated 70 more abstracts as additional corpus. We further finetuned the T0pp model, with 11 billion parameters, to boost the performance on relation extraction and leveraged entites' location information (eg, title, background) to enhance novelty prediction performance in relation extraction (RE). Results Our pioneering NLP system designed for this challenge secured first place in Phase I—NER and second place in Phase II—relation extraction and novelty prediction, outpacing over 200 teams. We tested OpenAI ChatGPT 3.5 and ChatGPT 4 in a Zero-Shot setting using the same test set, revealing that our finetuned model considerably surpasses these broad-spectrum large language models. Discussion and Conclusion Our outcomes depict a robust NLP system excelling in NER and RE across various biomedical entities, emphasizing that task-specific models remain superior to generic large ones. Such insights are valuable for endeavors like knowledge graph development and hypothesis formulation in biomedical research. [ABSTRACT FROM AUTHOR]

Extracting knowledge from cyber threat intelligence is essential for understanding cyber threats and implementing proactive defense measures. However, there is a lack of open datasets in the Chinese cybersecurity field that support both entity and relation extraction tasks. This paper addresses this gap by analyzing vulnerability description texts, which are standardized and knowledge-dense, to create a vulnerability knowledge ontology comprising 13 entities and 15 relations. We annotated 27,311 unique vulnerability description sentences from the China National Vulnerability Database, resulting in a dataset named BVTED for cybersecurity knowledge triple extraction tasks. BVTED contains 97,391 entities and 69,614 relations, with entities expressed in a mix of Chinese and English. To evaluate the dataset's value, we trained five deep learning-based named entity recognition models, two relation extraction models, and two joint entity–relation extraction models on BVTED. Experimental results demonstrate that models trained on this dataset achieve excellent performance in vulnerability knowledge extraction tasks. This work enhances the extraction of cybersecurity knowledge triples from mixed Chinese and English threat intelligence corpora by providing a comprehensive ontology and a new dataset, significantly aiding in the mining, analysis and utilization of the knowledge embedded in cyber threat intelligence. [ABSTRACT FROM AUTHOR]

Extracting relational triples from a piece of text is an essential task in knowledge graph construction. However, most existing methods either identify entities before predicting their relations, or detect relations before recognizing associated entities. This order may lead to error accumulation because once there is an error in the initial step, it will accumulate to subsequent steps. To solve this problem, we propose a parallel model for jointly extracting entities and relations, called PRE-Span, which consists of two mutually independent submodules. Specifically, candidate entities and relations are first generated by enumerating token sequences in sentences. Then, two independent submodules (Entity Extraction Module and Relation Detection Module) are designed to predict entities and relations. Finally, the predicted results of the two submodules are analyzed to select entities and relations, which are jointly decoded to obtain relational triples. The advantage of this method is that all triples can be extracted in just one step. Extensive experiments on the WebNLG*, NYT*, NYT and WebNLG datasets show that our model outperforms other baselines at 94.4%, 88.3%, 86.5% and 83.0%, respectively. [ABSTRACT FROM AUTHOR]

Background: Biomedical Relation Extraction (RE) is essential for uncovering complex relationships between biomedical entities within text. However, training RE classifiers is challenging in low-resource biomedical applications with few labeled examples. Methods: We explore the potential of Shortest Dependency Paths (SDPs) to aid biomedical RE, especially in situations with limited labeled examples. In this study, we suggest various approaches to employ SDPs when creating word and sentence representations under supervised, semi-supervised, and in-context-learning settings. Results: Through experiments on three benchmark biomedical text datasets, we find that incorporating SDP-based representations enhances the performance of RE classifiers. The improvement is especially notable when working with small amounts of labeled data. Conclusion: SDPs offer valuable insights into the complex sentence structure found in many biomedical text passages. Our study introduces several straightforward techniques that, as demonstrated experimentally, effectively enhance the accuracy of RE classifiers. [ABSTRACT FROM AUTHOR]

In view of the problem that the public data in the field of mine hoist fault is less and the fault knowledge is difficult to be effectively utilized, this paper proposes a method for constructing a mine hoist fault knowledge graph. This method firstly introduces a fault text classification process to deal with the information redundancy problem existing in the target corpus. Then it uses dictionary embedding BERT and BiLSTM-CRF combination for entity recognition, uses ERNIE for entity relation extraction, and stores the extracted triples in Neo4j graph database. On this basis, an intelligent question answering system based on mine hoist fault knowledge graph is realized. This knowledge graph can reveal the complex correlation between mine hoist faults, realize the root cause analysis of related faults, and provide support for mine hoist fault diagnosis. [ABSTRACT FROM AUTHOR]

Joint entity-relation extraction is a fundamental task in the construction of large-scale knowledge graphs. This task relies not only on the semantics of the text span but also on its intricate connections, including classification and structural details that most previous models overlook. In this paper, we propose the incorporation of this information into the learning process. Specifically, we design a novel two-dimensional word-pair tagging method to define the task of entity and relation extraction. This allows type markers to focus on text tokens, gathering information for their corresponding spans. Additionally, we introduce a multi-level attention neural network to enhance its capacity to perceive structure-aware features. Our experiments show that our approach can overcome the limitations of earlier tagging methods and yield more accurate results. We evaluate our model using three different datasets: SciERC, ADE, and CoNLL04. Our model demonstrates competitive performance compared to the state-of-the-art, surpassing other approaches across the majority of evaluated metrics. [ABSTRACT FROM AUTHOR]

The body of ecological literature, which informs much of our knowledge of the global loss of biodiversity, has been experiencing rapid growth in recent decades. The increasing difficulty of synthesising this literature manually has simultaneously resulted in a growing demand for automated text mining methods. Within the domain of deep learning, large language models (LLMs) have been the subject of considerable attention in recent years due to great leaps in progress and a wide range of potential applications; however, quantitative investigation into their potential in ecology has so far been lacking.In this work, we analyse the ability of GPT‐4 to extract information about invertebrate pests and pest controllers from abstracts of articles on biological pest control, using a bespoke, zero‐shot prompt.Our results show that the performance of GPT‐4 is highly competitive with other state‐of‐the‐art tools used for taxonomic named entity recognition and geographic location extraction tasks. On a held‐out test set, we show that species and geographic locations are extracted with F1‐scores of 99.8% and 95.3%, respectively, and highlight that the model can effectively distinguish between ecological roles of interest such as predators, parasitoids and pests. Moreover, we demonstrate the model's ability to effectively extract and predict taxonomic information across various taxonomic ranks. However, we do report a small number of cases of fabricated information (confabulations).Due to a lack of specialised, pre‐trained ecological language models, general‐purpose LLMs may provide a promising way forward in ecology. Combined with tailored prompt engineering, such models can be employed for a wide range of text mining tasks in ecology, with the potential to greatly reduce time spent on manual screening and labelling of the literature. [ABSTRACT FROM AUTHOR]

Relation extraction, as a key task in natural language processing, plays a significant role in deepening language understanding, constructing knowledge graphs, and optimizing information retrieval systems. However, traditional supervised learning methods are not well-suited for real-world scenarios due to the continuous emergence of new relations and the lack of large annotated datasets. Although the advent of large language models has significantly improved the performance of many natural language processing tasks, they still cannot effectively address the challenges of few-shot continual relation extraction. To fully leverage the semantic knowledge of large language models to mitigate catastrophic forgetting and overfitting issues, a novel few-shot continual relation extraction method, LAFA (large language model augmentation and feature alignment), is proposed. This method enhances representation alignment through various strategies such as relation instance rewriting, semantic expansion, and enhanced relation representation. It effectively improves the model adaptability to new relations and the retention of old knowledge while maintaining low data and computational costs. Experimental validation on two relation extraction datasets, FewRel and TACRED, demonstrates that LAFA outperforms existing methods in few-shot continual relation extraction tasks, particularly achieving the best results in incremental stages. Ablation experiments further reveal the significant contributions of each module to overall performance. Moreover, the inference efficiency and cost of LAFA are substantially lower than those of existing large language model-based methods, and it boasts strong scalability, being able to adapt to various language models.

Abstract Background Biomedical Relation Extraction (RE) is essential for uncovering complex relationships between biomedical entities within text. However, training RE classifiers is challenging in low-resource biomedical applications with few labeled examples. Methods We explore the potential of Shortest Dependency Paths (SDPs) to aid biomedical RE, especially in situations with limited labeled examples. In this study, we suggest various approaches to employ SDPs when creating word and sentence representations under supervised, semi-supervised, and in-context-learning settings. Results Through experiments on three benchmark biomedical text datasets, we find that incorporating SDP-based representations enhances the performance of RE classifiers. The improvement is especially notable when working with small amounts of labeled data. Conclusion SDPs offer valuable insights into the complex sentence structure found in many biomedical text passages. Our study introduces several straightforward techniques that, as demonstrated experimentally, effectively enhance the accuracy of RE classifiers.

Abstract The body of ecological literature, which informs much of our knowledge of the global loss of biodiversity, has been experiencing rapid growth in recent decades. The increasing difficulty of synthesising this literature manually has simultaneously resulted in a growing demand for automated text mining methods. Within the domain of deep learning, large language models (LLMs) have been the subject of considerable attention in recent years due to great leaps in progress and a wide range of potential applications; however, quantitative investigation into their potential in ecology has so far been lacking. In this work, we analyse the ability of GPT‐4 to extract information about invertebrate pests and pest controllers from abstracts of articles on biological pest control, using a bespoke, zero‐shot prompt. Our results show that the performance of GPT‐4 is highly competitive with other state‐of‐the‐art tools used for taxonomic named entity recognition and geographic location extraction tasks. On a held‐out test set, we show that species and geographic locations are extracted with F1‐scores of 99.8% and 95.3%, respectively, and highlight that the model can effectively distinguish between ecological roles of interest such as predators, parasitoids and pests. Moreover, we demonstrate the model's ability to effectively extract and predict taxonomic information across various taxonomic ranks. However, we do report a small number of cases of fabricated information (confabulations). Due to a lack of specialised, pre‐trained ecological language models, general‐purpose LLMs may provide a promising way forward in ecology. Combined with tailored prompt engineering, such models can be employed for a wide range of text mining tasks in ecology, with the potential to greatly reduce time spent on manual screening and labelling of the literature.

Abstract As the core competitiveness of the national industry, large-scale equipment such as ships, high-speed rail and nuclear power equipment, their production process involves in-depth personalization. It includes complex processes and long manufacturing cycles. In addition, the equipment’s supply chain management is extremely complex. Therefore, the development of a supply chain management knowledge graph is of significant strategic significance. It not only enhances the synergistic effect of the supply chain management but also upgrades the level of intelligent management. This paper proposes a distant supervision knowledge extraction and knowledge graph construction method in the supply chain management of large equipment manufacturing, which achieves digital and structured management and efficient use of supply chain management knowledge in the industry. This paper presents an approach to extract entity-relation knowledge using limited samples. We achieve this by establishing a distant supervision model. Furthermore, we introduce a fusion gate mechanism and integrate ontology information, thereby enhancing the model’s capability to effectively discern sentence-level semantics. Subsequently, we promptly modify the weights of input features using the gate mechanism to strengthen the model’s resilience and address the issue of vector noise diffusion. Finally, an inter-bag sentence attention mechanism is introduced to integrate different sentence bag information at the sentence bag level, which achieves more accurate entity-relation knowledge extraction. The experimental results prove that compared with the latest distant supervision method, the accuracy of relation extraction is improved by 2.8%, and the AUC value is increased by 3.9%, effectively improving the quality of knowledge graph in supply chain management.

The study of generic knowledge extraction methods in industrial manufacturing is of great significance for real- izing the automated construction of industrial knowledge bases. Aiming at the problem that industrial ontology definition requires a large amount of labour cost and expert experience for guidance, a semi-automatic ontology construction method is designed based on the sememe analysis, and finally the definition of ontology, ontology class, cascade and relationship is completed, using automobile manufacturing fault repair data as an example. Subsequently, in order to solve the problems of relationship nesting and cascade model error propagation that exist in relationship extraction in the industrial domain, a semantic- aware relationship extraction model is designed. In the three links of potential relationship mining, subject extraction and object extraction in this model, different leading interrogatives are spliced based on reading comprehension methods to obtain sentence codes applicable to different links.In order to make use of the a priori knowledge of the subject, the attention mechanism is incorporated into the object extraction module to improve the expressiveness of the codes in this module. Three sessions of joint optimised training to improve extraction results. Experiments are conducted on the automotive manufacturing fault repair dataset, the automotive industry fault pattern extraction evaluation dataset, and the equipment manufacturing dataset, and the model in this paper achieves better results than other relational extraction baseline models. [ABSTRACT FROM AUTHOR]

This paper describes NEREL—a Russian news dataset suited for three tasks: nested named entity recognition, relation extraction, and entity linking. Compared to flat entities, nested named entities provide a richer and more complete annotation while also increasing the coverage of relations annotation and entity linking. Relations between nested named entities may cross entity boundaries to connect to shorter entities nested within longer ones, which makes it harder to detect such relations. NEREL is currently the largest Russian dataset annotated with entities and relations: it comprises 29 named entity types and 49 relation types. At the time of writing, the dataset contains 56 K named entities and 39 K relations annotated in 933 person-oriented news articles. NEREL is annotated with relations at three levels: (1) within nested named entities, (2) within sentences, and (3) with relations crossing sentence boundaries. We provide benchmark evaluation of current state-of-the-art methods in all three tasks. The dataset is freely available at https://github.com/nerel-ds/NEREL. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Donghua University (English Edition) is the property of Journal of Donghua University Editorial Board and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Relation extraction is an important task in information extraction, which aims to identify the relation between two given entities. The algorithm based on distant supervision can automatically generate a large amount of annotated data, which becomes the main method to deal with the task of relation extraction. However, previous studies rely too much on the precision of supervision information and ignore the effective supervision information hidden in the case of mislabeling, which leads to the loss of supervision information. To solve this problem, we propose the distantly supervised relation extraction model based on residual attention and self-learning. The model uses residual attention to extract features, and then uses self-learning idea to generate corrected labels for training data, which are added into the training process as supervisory signals to prevent parameter error updates caused by noisy labels. The model can not only reduce the problem of mislabeling caused by distant supervision, but also makes full use of the available supervisory information in the data to improve data utilization. Experiments show that compared with the existing mainstream baseline methods, the proposed model has higher precision and recall. [ABSTRACT FROM AUTHOR]

Most existing relation extraction methods only determine the relation type after identifying all entities, thus not fully modeling the interaction between relation-type recognition and entity mention detection. This article introduces a novel paradigm for relation extraction by treating relevant entities as parameters of relations and harnessing the strong expressive capabilities and acceleration advantages of quantum computing to address the relation extraction task. In this article, we develop a quantum hierarchical reinforcement learning approach to enhance the interaction between relation-type recognition and entity mention detection. The entire relation extraction process is broken down into a hierarchical structure of two layers of quantum reinforcement learning strategies dedicated to relation detection and entity extraction, demonstrating greater feasibility and expressiveness, especially when dealing with superimposed relations. Our proposed method outperforms existing approaches through experimental evaluations on commonly used public datasets, mainly showcasing its significant advantages in extracting superimposed relationships. [ABSTRACT FROM AUTHOR]