1. A Few-shot Learning Method Based on Bidirectional Encoder Representation from Transformers for Relation Extraction
- Author
-
Rongzhi Qi, Yifei Gao, and Shuiyan Li
- Subjects
Relation (database) ,Computer science ,business.industry ,Feature extraction ,Pattern recognition ,computer.software_genre ,Convolutional neural network ,Relationship extraction ,Information extraction ,Feature (machine learning) ,Artificial intelligence ,Representation (mathematics) ,business ,Encoder ,computer - Abstract
Relation extraction is one of the fundamental subtasks of the information extraction. The purpose is to determine the implicit relation between two entities in a sentence. Therefore, Convolutional Neural Networks and Feature Attention-based Prototypical Networks (CNN-Proto-FATT), a typical few-shot learning method, is proposed and achieve competitive performance. However, convolutional neural networks suffer from the insufficient instances of relation in real scenes, leading to undesirable results. To extract long-distance features more comprehensively, the pre-trained model Bidirectional Encoder Representation from Transformers (BERT) is incorporated into CNN-Proto-FATT. In this model, named Bidirectional Encoder Representation from Transformers and Feature Attention-based Prototypical Networks (BERT-Proto-FATT), the multi-head attention helps the network extract semantic features cross long- and short-distance to enhance the encoded representations. Experimental results indicate that BERT-Proto-FATT demonstrates significant improvements on the FewRel dataset.
- Published
- 2021
- Full Text
- View/download PDF