Search

Annotating entities and relationships have been found with the high cost in the joint extraction tasks of entity relationship. There is the strong correlation with the professional fields. It is crucial to improve the extraction performance of the model in the scenarios of low resource. In this study, the reinforcement learning-based joint extraction model was proposed for the entity relationship. Two modules were consisted of: entity recognition and relationship extraction. The extraction efficiency and generalization were improved to jointly extract the entities and relationships in the low resource scenarios. The feature extractor was used to convert the text into feature representations with the richer semantics, which were shared by entity recognition and relationship extraction modules. Entity recognition was realized to utilize the CRF for sequence labeling. The output entity labeling sequence was traversed to locate the entity boundaries, and then connect the entity features for the entity embedding vectors. The limited labeled and abundant unlabeled data was considered in the low resource scenarios. Reinforcement learning was used to train the relationship extraction module. The input of the relationship extraction module included [sentence features, entity embeddings, and relationships] for the labeled data, and [sentence features, and entity embeddings] for the unlabeled data. The improved model was trained to simulate the pseudo labels that generated by unlabeled data in the gradient direction of labeled data, in order to maximize the similarity of the average gradients between them. There was the increase in the diversity and richness of the data, particularly for the better generalization with the less risk of overfitting. Meanwhile, the generated pseudo labels were reduced the dependence on a large amount of labeled data and lower annotation costs. More importantly, the gradient simulation was also balanced the sample distribution of different relationship categories in the dataset, especially in the cases of imbalanced relationship categories. The effectiveness of the model was verified to compare the mainstream models of low resource relationship extraction in the apple cultivation corpus (ATC). The results showed that the F1 score of the model was 88.71%, when the proportion of labeled data reached 30%, indicating the significantly improved model than the rest baselines. In addition, the entity relationships model was effectively extracted from the public dataset TACRED in the low resource scenarios. The proportion of unlabeled data was changed in the ATC and TACRED datasets. The experiments showed that the F1 performance varied on the fixed 10% labeled data and 10%, 30%, 50%, 70%, and 90% unlabeled data. The improved performance was achieved to add the unlabeled data for training. The optimal F1 performance was consistently achieved in the different proportions of unlabeled data. The effectiveness of the gradient simulation module was verified through ablation experiments. The relationship extraction model without gradient simulation module was basically the same as the Self-TrainedBERT model. There was an average F1 decrease of 6.12% in the Self-TrainedBERT model using different proportions of labeled data. The improved performance of the relationship extraction module was attributed to the gradient simulation module, which was improved the quality of pseudo labels. Finally, principal component analysis was used to demonstrate the gradient descent direction of the relationship extraction module for the labeled and pseudo labeled data, representing the quality of pseudo labeled data. The gradient simulation module was also added to gradually approach the ideal local minimum, although the optimization direction of pseudo label data initially fluctuated greatly. The effectiveness of the gradient simulation module was further proved to generate the high-quality pseudo labels. Therefore, the proposed model can effectively extract the entity relationships in the low resource scenarios, indicating the high generalization and performance. [ABSTRACT FROM AUTHOR]

In view of the problem that the public data in the field of mine hoist fault is less and the fault knowledge is difficult to be effectively utilized, this paper proposes a method for constructing a mine hoist fault knowledge graph. This method firstly introduces a fault text classification process to deal with the information redundancy problem existing in the target corpus. Then it uses dictionary embedding BERT and BiLSTM-CRF combination for entity recognition, uses ERNIE for entity relation extraction, and stores the extracted triples in Neo4j graph database. On this basis, an intelligent question answering system based on mine hoist fault knowledge graph is realized. This knowledge graph can reveal the complex correlation between mine hoist faults, realize the root cause analysis of related faults, and provide support for mine hoist fault diagnosis. [ABSTRACT FROM AUTHOR]

Using knowledge graph technology for data management can achieve effective representation of fully mechanized coal mining equipment. The information with deep mining value can be obtained. The imbalanced data of fully mechanized coal mining equipment and the limited number of entities in certain categories of equipment affect the precision of entity recognition models. In order to solve the above problems, a knowledge graph construction method for fully mechanized coal mining equipment based on joint coding is proposed. Firstly, the fully mechanized coal mining equipment ontology model is constructed, determining the concepts and relationships. Secondly, the entity recognition model is designed. The model uses Token Embedding, Position Embedding, Sentence Embedding, and Task Embedding 4-layer Embedding structures and Transformer Encoder to encode fully mechanized coal mining equipment data, extract dependency relationships and contextual information features between words. The model introduces a Chinese character library, using the Word2vec model for encoding, extracting semantic rules between characters, and solving the problem of rare characters in fully mechanized coal mining equipment data. The model uses the GRU model to jointly encode the data of fully mechanized coal mining equipment and the character vectors encoded in the font library, and fuse vector features. The model uses the Lattice-LSTM model for character decoding to obtain entity recognition results. Finally, the model uses graph database technology to store and organize extracted knowledge in the form of graphs, completing the construction of knowledge graphs. Experimental verification is conducted on the dataset of fully mechanized coal mining equipment. The results show that the method improves the recognition accuracy of fully mechanized coal mining equipment entities by more than 1.26% compared to existing methods, which to some extent alleviates the low accuracy problem caused by insufficient data when constructing a knowledge graph of fully mechanized coal mining equipment in a small sample situation.

Copyright of Journal of Mine Automation is the property of Industry & Mine Automation Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Purposes Event extraction is a prerequisite for building high-quality event knowledge graphs. The dependency of event elements exists in the process of clinical event extraction. Existing methods fail to accurately identify event elements and combine them into events, and the amount of available clinical event tagging data is limited. These problems bring great challenges to the event extraction task. Methods In this research, clinical event is extracted and modelled as an entity recognition model, and a Chinese medical event extraction method incorporating multiple features is proposed: BERT-MCRF. In this method, Bidirectional Encoder Representation from Transformers(BERT) is used to construct the embedding and feature extraction parts of the model, multiple word sliding window features in the Conditional Random Fields(CRF) layer are added, then BERT-MCRF is used as a base experiment for semi-supervised experiments, and a high confidence pseudo-labeled data is proposed. The selection algorithm is used as a condition to filter the data, and 300 data of higher quality are obtained and merged with the original data. Finally, 1 700 corpus are constructed and the model is retrained. Findings The overall F1 value of the BERT-MCRF model on the three attribute entities reaches 80.21%, which is 15.11% better than that of the classical Bi-directional Long Short Term Memory-Conditional Random Fields (BiLSTM-CRF) model; with the model retrained by the semi-supervised idea, the final F1 value reaches 81.56%, which is 1.35% higher than the original BERT-MCRF.

Copyright of Electric Power is the property of Electric Power Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In medical question answering systems, entity recognition plays a major role. Entity recognition based on deep learning has received more and more attention. However, in the medical question answering system, due to the lack of annotated training data, deep learning methods cannot well identify discontinuous and nested entities in medical text. Therefore, a large language model-based entity recognition application method is proposed, and it is applied to the medical problem system. Firstly, the dataset related to medical question answering is processed into text that can be analyzed and processed by a large language model. Secondly, the output of the large language model is classified, and different classifications are processed accordingly. Then, the input text is used for intent recognition, and finally the results of entity recognition and intent recognition are sent to the medical knowledge graph for query, and the answer to the medical question and answer is obtained. Experiments are performed on 3 typical datasets and compared with several typical correlation methods. The results show that the method proposed in this paper performs better. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Zhengzhou University (Natural Science Edition) is the property of Journal of Zhengzhou University (Natural Science Edition) Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In order to solve the problem of extracting machine tool design knowledge in the construction of computer numerical control( CNC) machine tool design knowledge graph, the classification standard and labeling strategy of knowledge in the field of CNC machine tools are formulated and a domain data set was constructed. The entity recognition method based on robustly optimized BERT pretraining approach(RoBERTa) for CNC machine tool design was proposed. Firstly, RoBERTa was fine-tuned by using the data set in the field of CNC machine tools, and then the text was encoded by RoBERTa to generate a vector representation. Secondly, the bidirectional long short-term memory (BiLSTM) network was used to extract the vector features. Finally, the conditional random field (CRF) was used to infer the optimal result. The experimental results show that the F1 value of the model on the test data set is 86. 139%, the F1 value of most key entity is greater than 85%. Compared with other models, it is improved by 2% ~18%. It can be seen that this method has obvious advantages in the recognition of CNC machine tool design entities, and can identify the key entities in the machine tool design knowledge, providing a data basis for the knowledge graph. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Dalian Ocean University is the property of Journal of Dalian Ocean University Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Railway Science & Engineering is the property of Journal of Railway Science & Engineering Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Objectives: In the face of the complex and changing network security environment, how to fight against Advanced Persistent Threat (APT) attacks has become an urgent problem for the entire security community. The massive APT attack analysis reports and threat intelligence generated by security companies have significant research value. They can effectively provide the information of APT organizations, thereby assisting in the traceability analysis of network attack events. Aiming at the problem that APT analysis reports have not been fully utilized, and there is a lack of automation methods to generate structured knowledge and construct feature portraits of the hacker organizations, an automatic knowledge extraction method of APT attacks combining entity recognition and entity alignment is proposed. The proposed method can automatically extract entities from APT analysis reports and construct structured knowledge of the APT organization. Methods: An automatic extraction method of APT attack knowledge that integrates entity recognition and entity alignment is designed. Firstly, 12 entity categories are designed according to the characteristics of APT attacks. Then, lowercase conversion, data cleaning, and data annotation are performed on the corpus through the preprocessing layer, and the preprocessed APT text sequence is represented as a vector. Secondly, the Bert model is built to pre-train the annotated corpus, encode each word, and generate the corresponding word vector. Also, the BiLSTM model is constructed to capture long-distance and contextual semantic features. The attention mechanism is built to highlight key features and convert the vector sequence into an annotation probability matrix. Thirdly, the CRF algorithm is utilized to decode the relationship between the output predicted labels and generate the optimal label sequence. Finally, the entity alignment method based on semantic similarity and Birch is constructed, which can improve the quality of the extracted APT attack knowledge through knowledge matching and merging into the infobox of each APT organization. Results: In terms of entity recognition, the proposed APT attack entity recognition method is superior to the existing entity recognition methods (i.e., CRF, LSTM-CRF, GRU-CRF, BiLSTMCRF, CNN-CRF, and Bert-CRF). The experimental results of our method have been improved to a certain extent, whose precision, recall, and F1-score are 0.929 6, 0.873 3, and 0.900 6. Compared with CRF, the F1-score of the proposed model is increased by 14.32%. Compared with CNN-CRF, which integrates convolutional neural networks, the F1-score of the proposed model is increased by 6.92%. Compared with LSTM-CRF and BiLSTM-CRF, the F1-score of the proposed model is increased by 8.43% and 5.30%, respectively. Compared with GRU-CRF, the F1-score of this model is increased by 8.74%. Compared with Bert-CRF, the F1-score of this model is increased by 7.03%. In addition, the accuracy of the proposed model is 0.9004, which is 9.85% higher than the average of the other six models. Also, the proposed model's training process is more stable, and the entire curve converges faster, which can achieve higher accuracy with fewer training batches. The model's error converges faster in the training period, and the curve is smoother. Moreover, the proposed model has the best prediction effect on the "attack method" entity category, whose F1-score is 0.927 5. On the one hand, a large number of entities exist in this category. On the other hand, this category of entities widely exists in semantic-rich APT attack events and has the action characteristics of attack behavior, which leads to a better recognition effect of this category. In terms of entity recognition with small sample annotation, the proposed method's precision, recall, and F1-score are 0.780 0, 0.589 4, and 0.671 4, respectively. Compared with the CRF model, LSTM-CRF model,GRU-CRF model, BiLSTM-CRF model, CNN-CRF model, and Bert-CRF model, the F1-score values of the proposed model are improved by 27.42%, 18.78%, 23.62%, 13.25%, 14.88%, and 14.46%. This experiment fully demonstrates that the proposed method can perform pre-training on a small sample corpus through the Bert model, thereby improving the effect of entity recognition. In terms of entity alignment and knowledge fusion, the experiment automatically extracts named entities with the high frequency of various entity categories, which often exist in APT attack events. For example, common APT organizations include "APT29", "APT32", "APT28", and "Turla";common attack equipment includes "PowerShell", "Cobalt Strike", and "Mimikatz"; common attack methods include "Spearphishing", "C2", "Watering Hole Attack", and "Backdoor"; common vulnerabilities include "CVE-2017-11882", "CVE-2017-0199", and "CVE-2012-0158", etc. The proposed method combines the corpus titles and keywords to carry out entity fusion of APT organization names. Finally, the infobox of common APT organizations in this dataset is constructed, and the structured knowledge of each APT organization is formed. Also, the attack domain knowledge of APT28 and APT32 is shown in detail. Conclusions: According to the characteristics of APT attacks, an automatic extraction method of APT attack knowledge based on entity recognition and entity alignment is designed and implemented. This method can effectively identify APT attack entities, automatically extract advanced persistent threat knowledge under the condition of few-sample annotation, and generate structured feature portraits of common APT organizations, which will provide support for subsequent APT attack knowledge graph construction and attack traceability analysis.

The widely researched sentence-level event extraction methods struggle to extract all arguments of the same event from a whole document. To solve this problem, this paper proposes a joint model for document-level event extraction based on deep learning. Firstly, an entity recognition module based on multi-head self-attention mechanism is used to identify entities and their types sentence by sentence. Then, an event type detection module trained by defining the importance of different argument roles, is used to locate the event mention sentence and predict the event type without the help of event triggers. Finally, an event argument extraction module embeds every entity??s semantic vector with its type information and its distance to the event mention sentence before feeding into a context-aware Transformer, in order to extract arguments within the document scope. In addition, by training the three modules mentioned above jointly, this paper realizes an end-to-end event extraction model and avoids error propagation problems in traditional pipeline models. The experimental results on a public dataset shows that, when each document contains only one event, the proposed model achieves a 86.3% [F1-score], which outperforms state-of-the-art methods, and the training process completes rather quickly.

Entity recognition is a sub task of information extraction.The traditional entity recognition model is used to identify entities of personnel,organization,location and name.In the real world,more types of entities must be considered,and fine-grained entity recognition is needed.At the same time,traditional entity recognition models such as BiGRU cannot make full use of the global features in a wider range.This paper presents an entity recognition model based on memory network and BERT.The pre-training language model of BERT is used for better semantic representation,and the memory network module can memorize a wider range of features.The results of entity recognition for cement clinker production corpus data show that this method can re-cognize entities and has some advantages over other traditional models.In order to further verify the model in this paper,experiments are carried out on the CLUENER2020 dataset.The results show that the optimization based on BiGRU-CRF model using BERT and memory network module can improve the effect of entity recognition.

Copyright of Journal on Communication / Tongxin Xuebao is the property of Journal on Communications Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Span-based joint extraction models have achieved excellent results in named entity recognition and relation extraction. These models regard text spans as candidate entities and span tuples as candidate relation tuples. span semantic representations are shared in both entity recognition and relation extraction, while existing models cannot well capture semantics of these candidate entities and relations. To address these problems, a span-based joint extraction framework with attention-based semantic presentations is proposed. Specially, attentions are utilized to calculate semantic representations, including span-specific and contextual ones. Experiments show that our model outperforms previous systems and achieves state-of-the-art results on ACE2005, CoNLL2004 and ADE. [ABSTRACT FROM AUTHOR]

In recent years,cybersecurity threats have increased,and data-driven security intelligence analysis has become a hot research topic in the field of cybersecurity.In particular,the artificial intelligence technology represented by the knowledge graph can provide support for complex cyberattack detection and unknown cyberattack detection in multi-source heterogeneous threat intelligence data.Cybersecurity entity recognition is the basis for the construction of threat intelligence knowledge graphs.The composition of security entities in open network text data is very complex,which makes traditional deep learning methods difficult to identify accurately.Based on the pre-training language model of BERT (pre-training of deep bidirectional transformers),a cybersecurity entity recognition model BERT-RDCNN-CRF based on residual dilation convolutional neural network and conditional random field was proposed.The BERT model was used to train the character-level feature vector representation.Combining the residual convolution and the dilation neural network model to effectively extract the important features of the security entity,and finally obtain the BIO annotation of each character through CRF.Experiments on the large-scale cybersecurity entity annotation dataset constructed show that the proposed method achieves better results than the LSTM-CRF model,the BiLSTM-CRF model and the traditional entity recognition model.

In recent years, cybersecurity threats have increased, and data-driven security intelligence analysis has become a hot research topic in the field of cybersecurity. In particular, the artificial intelligence technology represented by the knowledge graph can provide support for complex cyberattack detection and unknown cyberattack detection in multi-source heterogeneous threat intelligence data. Cybersecurity entity recognition is the basis for the construction of threat intelligence knowledge graphs. The composition of security entities in open network text data is very complex, which makes traditional deep learning methods difficult to identify accurately. Based on the pre-training language model of BERT (pre-training of deep bidirectional transformers), a cybersecurity entity recognition model BERT-RDCNN-CRF based on residual dilation convolutional neural network and conditional random field was proposed. The BERT model was used to train the character-level feature vector representation. Combining the residual convolution and the dilation neural network model to effectively extract the important features of the security entity, and finally obtain the BIO annotation of each character through CRF. Experiments on the large-scale cybersecurity entity annotation dataset constructed show that the proposed method achieves better results than the LSTM-CRF model, the BiLSTM-CRF model and the traditional entity recognition model.

Copyright of Journal of Heilongjiang University of Science & Technology is the property of Journal of Heilongjiang University of Science & Technology Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In order to solve the problems of insufficient modeling of entity and relationship dependence and the difficulty of extracting multiple relationships involved in existing j oint extraction methods of entities and relations hips, this paper designed a joint extraction framework based on deep learning. Firstly, for the problem of insufficient dependency modeling, the framework extracted entity co-occurrence features from the pre-trained corpus, and modeled the potential semantic relationship between entities and the dependency relationship between entities and relationships. Secondly, it included a novel pointer labeling method. This labeling method could represent the relationship category through a pointer. Since any entity could be pointed by multiple pointers, it was possible to mark overlapping entities in a piece of text and extracted multiple entity-relation triplets result. Finally, in ord er to effectively use the rich semantics of words and the information dependent on pointers, it d es igned a tag-aware attention mechanism was necessary, which incorporated word information from the coding layer and related co-occurrence semantic information. Compared with the joint extraction method at the forefront of research, the proposed method achieved an increase in F1 value on the Baidu DuIE test set. The experimental results show that the pointer labeling method can solve the problem of entity overlap to a certain extent. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Zhengzhou University: Engineering Science is the property of Editorial Office of Journal of Zhengzhou University: Engineering Science and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Aiming at the problems in the research methods of named entity recognition and entity relationship extraction in Chinese electronic medical records), this paper proposed an entity identification and entity relationship based on bidirectional long short-term memory and conditional random field (CRF). The method fir st used word embedding technology to convert text into numerical vector, as the input of neural network BiLSTM, combined with CRF chain structure for sequence labeling, output the maximum probability sequence, and mapping the recognition result knowledge graph by using the database tool Neo4j. Experiments show that the method can significantly improve the accuracy, recall rate and F value of entity identification and entity relationship extraction in Chinese electronic medical records. The experimental results meet the needs of clinical system applications, and have a guiding role in helping to study and construct clinical decision support systems and personalized medical recommendation services. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Chongqing University of Posts & Telecommunications (Natural Science Edition) is the property of Chongqing University of Posts & Telecommunications and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

It is very difficult to evaluate the effects of government governance.Without a good evaluation method and evaluation system,the effects of government governance cannot be guaranteed.Understanding the intention of web users in the topic of government governance from the perspective of natural language question-and-answering was proposed.By constructing a knowledge graph of intentions,equivalent questions and intentions were associated.The definition,construction framework and usage examples in government governance were illustrated,showing that knowledge graph of intentions is an effective way to evaluate the effects of government governance.In the context of government governance,by using the knowledge graphs of intentions,the intention fields between different governance subjects under the same governance topic were analyzed and compared,the effects of specific governance subjects on specific governance topics were analyzed,and the issues remained in government governance were found.

Copyright of Journal of Beijing University of Technology is the property of Journal of Beijing University of Technology, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)