Back to Search Start Over

An Eclectic Approach for Enhancing Language Models Through Rich Embedding Features

Authors :
Edwin Aldana-Bobadilla
Victor Jesus Sosa-Sosa
Alejandro Molina-Villegas
Karina Gazca-Hernandez
Jose Angel Olivas
Source :
IEEE Access, Vol 12, Pp 100921-100938 (2024)
Publication Year :
2024
Publisher :
IEEE, 2024.

Abstract

Text processing is a fundamental aspect of Natural Language Processing (NLP) and is crucial for various applications in fields such as artificial intelligence, data science, and information retrieval. It plays a core role in language models. Most text-processing approaches focus on describing and synthesizing, to a greater or lesser degree, lexical, syntactic, and semantic properties of text in the form of numerical vectors that induce a metric space, in which, it is possible to find underlying patterns and structures related to the original text. Since each approach has strengths and weaknesses, finding a single approach that perfectly extracts representative text properties for every task and application domain is hard. This paper proposes a novel approach capable of synthesizing information from heterogeneous state-of-the-art text processing approaches into a unified representation. Encouraging results demonstrate that using this representation in popular machine-learning tasks not only leads to superior performance but also offers notable advantages in memory efficiency and preservation of underlying information of the distinct sources involved in such a representation.

Details

Language :
English
ISSN :
21693536
Volume :
12
Database :
Directory of Open Access Journals
Journal :
IEEE Access
Publication Type :
Academic Journal
Accession number :
edsdoj.0b82df7867f488c8e282c76dd220662
Document Type :
article
Full Text :
https://doi.org/10.1109/ACCESS.2024.3422971