Revisiting Bag of Words Document Representations for Efficient Ranking with Transformers.

The article focuses on enhancing efficiency in modern transformer-based information retrieval models, addressing the limitations posed by the self-attention mechanism for long input, such as in document retrieval, by exploring bag-of-words document representations that encode documents using only a fraction of their characteristic terms. It mentions through experiments on MS MARCO and Robust04 datasets.