1. Multi-view pre-trained transformer via hierarchical capsule network for answer sentence selection.
- Author
-
Li, Bing, Yang, Peng, Sun, Yuankang, Hu, Zhongjian, and Yi, Meng
- Subjects
CAPSULE neural networks ,CLASSIFICATION ,FORECASTING - Abstract
Answer selection requires technology that effectively captures in-depth semantic information between the question and the corresponding answer. Most existing studies focus on using linear or pooling operations to directly classify the output representation, resulting in the absence of critical information and the emergence of single-label predictions. To address these issues, we propose a novel Multi-view Pre-trained Transformer with Hierarchical Capsule Network (MPT-HCN). Specifically, we propose a Hierarchical Capsule Network composed of three capsule networks to independently process high-dimensional sparse information of words, semantic information of similar expressions, and feature classification information so that multiple attributes can be fully considered and accurately clustered. Moreover, we consider the impact of the intermediate encoder layer output information on the overall sequence semantic representation and propose a Multi-view Information Fusion that obtains the final semantic representation information by weighted fusion of the output information of all encoder layers, thereby avoiding the appearance of a single prediction label. Extensive experiments on five typical representative datasets, especially on the WikiQA dataset, show that our model MPT-HCN (RL) achieves an excellent performance of 0.939 on MAP and 0.942 on MRR, which is a significant improvement of 3.9% and 2.7% respectively, compared to the state-of-the-art baseline model. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF