Back to Search Start Over

Beyond [CLS] through Ranking by Generation

Authors :
Santos, Cicero Nogueira dos
Ma, Xiaofei
Nallapati, Ramesh
Huang, Zhiheng
Xiang, Bing
Publication Year :
2020

Abstract

Generative models for Information Retrieval, where ranking of documents is viewed as the task of generating a query from a document's language model, were very successful in various IR tasks in the past. However, with the advent of modern deep neural networks, attention has shifted to discriminative ranking functions that model the semantic similarity of documents and queries instead. Recently, deep generative models such as GPT2 and BART have been shown to be excellent text generators, but their effectiveness as rankers have not been demonstrated yet. In this work, we revisit the generative framework for information retrieval and show that our generative approaches are as effective as state-of-the-art semantic similarity-based discriminative models for the answer selection task. Additionally, we demonstrate the effectiveness of unlikelihood losses for IR.<br />Comment: EMNLP 2020

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2010.03073
Document Type :
Working Paper