Back to Search Start Over

Natural language processing with transformers: a review

Authors :
Georgiana Tucudean
Marian Bucos
Bogdan Dragulescu
Catalin Daniel Caleanu
Source :
PeerJ Computer Science, Vol 10, p e2222 (2024)
Publication Year :
2024
Publisher :
PeerJ Inc., 2024.

Abstract

Natural language processing (NLP) tasks can be addressed with several deep learning architectures, and many different approaches have proven to be efficient. This study aims to briefly summarize the use cases for NLP tasks along with the main architectures. This research presents transformer-based solutions for NLP tasks such as Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-Training (GPT) architectures. To achieve that, we conducted a step-by-step process in the review strategy: identify the recent studies that include Transformers, apply filters to extract the most consistent studies, identify and define inclusion and exclusion criteria, assess the strategy proposed in each study, and finally discuss the methods and architectures presented in the resulting articles. These steps facilitated the systematic summarization and comparative analysis of NLP applications based on Transformer architectures. The primary focus is the current state of the NLP domain, particularly regarding its applications, language models, and data set types. The results provide insights into the challenges encountered in this research domain.

Details

Language :
English
ISSN :
23765992
Volume :
10
Database :
Directory of Open Access Journals
Journal :
PeerJ Computer Science
Publication Type :
Academic Journal
Accession number :
edsdoj.631bdc5cfed74b27b6110dc8292d1781
Document Type :
article
Full Text :
https://doi.org/10.7717/peerj-cs.2222