1. 텍스트 요약을 위한 어텐션 기반 BART 모델 미세조정.
- Author
-
안영필 and 박현준
- Subjects
TEXT summarization ,LANGUAGE models - Abstract
Automatically summarizing long sentences is an important technique. The BART model is one of the widely used models in the summarization task. In general, in order to generate a summarization model of a specific domain, fine-tuning is performed by re-training a language model trained on a large dataset to fit the domain. The fine-tuning is usually done by changing the number of nodes in the last fully connected layer. However, in this paper, we propose a fine-tuning method by adding an attention layer, which has been recently applied to various models and shows good performance. In order to evaluate the performance of the proposed method, various experiments were conducted, such as accumulating layers deeper, fine-tuning without skip connections during the fine tuning process, and so on. As a result, the BART model using two attention layers with skip connection shows the best score. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF