Search

Natural Language Processing has become an important issue with the rapid increase in textual data in the health sector recently. Especially with the effect of COVID-19, easy and fast analysis of health data is important for research. Traditional text representations such as BoW (bag of words), TF-IDF (term frequency-inverse document frequency), and modern word representation methods such as FastText and BERT are used to represent words. The BERT models are provided high performance recently. The BERT models are divided into pre-trained and fine-tuned BERT models. In order to get good results in the field of health, BioBERT models are obtained by fine-tuning the basic BERT models with datasets containing biomedical articles. In this study, semantic similarities in datasets are evaluated by the Pearson correlation method by using BoW, TF-IDF, FastText, BERT, and BioBERT models. As a result of the evaluations, it was observed that BioBERT models gave higher values compared to other models and methods used.

In Evidence-Based Medicine (EBM), medical literature is an essential resource used by clinicians and researchers. It contains research claims that summarize the critical findings of a study. However, research claims on the same topic can be contradicting. Given a clinical question, if two claims that answer the question have conflicting assertion values (Yes or No), they are considered contradictory claims. Hence, discovering the claim assertion value of a research claim is the key to detecting contradictory research claims in medical literature. In this study, we explored the usage of deep neural network (DNN) to recognize contradictory research claims in medical literature. The DNN model should determine the assertion value of a research claim against its clinical question. The model was evaluated using a publicly available corpus containing contradictory research claims from 24 systematic reviews on cardiovascular topic. Different DNN techniques such as the Global Vectors for Word Representation (GLoVe), bidirectional Long Short-Term Memory (LSTM) and Bidirectional Encoder Representations from Transformers (BERT) were implemented in building the claim assertion model. The evaluation results suggest that the BERT model performs better than LSTM and GloVe model and outperforms the previous studies’ models.

Making inference on clinical texts is a task which has not been fully studied. With the newly released, expert annotated MedNLI dataset, this task is being boosted. Compared with open domain data, clinical texts present unique linguistic phenomena, e.g., a large number of medical terms and abbreviations, different written forms for the same medical concept, which make inference much harder. Incorporating domain-specific knowledge is a way to eliminate this problem, in this paper, we assemble a new incorporating medical concept definitions module on the classic enhanced sequential inference model (ESIM), which first extracts the most relevant medical concept for each word, if it exists, then encodes the definition of this medical concept with a bidirectional long short-term network (BiLSTM) to obtain domain-specific definition representations, and attends these definition representations over vanilla word embeddings. The empirical evaluations are conducted to demonstrate that our model improves the prediction performance and achieves a high level of accuracy on the MedNLI dataset. Specifically, the knowledge enhanced word representations contribute significantly to entailment class. Institute of Electrical and Electronics Engineers

The main essence of word representation in a vector space is to preserve the similarity between the words. Traditional measures of word similarity retain the contextual or semantic affinity among the words. In this study, we propose an alternative word embedding scheme which considers the temporal relationships among the words. We employ the Google Trends search queries along with the respective time series information to represent words in a vector space. Our experiments show that the proposed representation is capable of incorporating temporal context that is otherwise unavailable in conventional word representations.

There is a large number of drugs introduced every year and a number of interactions between drugs also has quick growth. As a result, biomedical texts following new drugs and interactions expand [15]. Several published studies of drug safety have revealed that drug-drug interactions (DDIs) may be detected too late, when millions of patients have already been exposed [25]. Therefore, the management of drug drug interactions is critical issue since the importance of known drug drug interaction and the giant amount of available information around them [5]. Thus, the issue creates an imperative need for the development of high-reliable automatic DDI extraction methods while manual DDI extraction is time-consuming and could lead to out-of-date information. However, the accuracy of the current automatic DDI extraction method is still insufficient for the practical application. In this research, we explore the Relation Bidirectional Encoder Representations from Transformers (Relation BERT) architecture [32] to detect and classify DDIs from biomedical texts using the DDI extraction 2013 corpus [5] and present three proposed models namely R-BERT∗, R-BioBERT1, and R-BioBERT2. From our knowledge, we are the first to investigate the potential of Relation BERT for the aim of accuracy improvement in DDI extraction. By using the state-of-the-art word representation method, three models produce macro-average F1-score of over 79%. Moreover, the accuracy of extracting Advice and Mechanism interaction achieves 90.63% and 97% respectively in terms of F1-score. The high accuracy of the model in Advice and Mechanism interaction creates motivation for wide application of automatic DDI extraction to the practice with high-reliable and humanless.

We propose an approach to automatically predict the evaluation of the consultant for press conferences, using text only. The proposed approach includes a word representation model and a language model for automatic evaluation. The word representation model consists of token embedding using ELMo and type embedding. The language model we used is an LSTM with a self-attention mechanism. We collected seven publicly available press conference videos, and all the Q&A pairs between the journalists and the speakers were annotated by a professional consultant. As a result, we achieved an average accuracy of 57.6% for the prediction of 11 evaluation criterions.

While question answering (QA) systems have witnessed great breakthroughs in reading comprehension (RC) tasks, spoken question answering (SQA) is still a much less investigated area. Previous work shows that existing SQA systems are limited by catastrophic impact of automatic speech recognition (ASR) errors [1] and the lack of large-scale real SQA datasets [2]. In this paper, we propose using contextualized word representations to mitigate the effects of ASR errors and pretraining on existing textual QA datasets to mitigate the data scarcity issue. New state-of-the-art results have been achieved using contextualized word representations on both the artificially synthesised and real SQA benchmark data sets, with 21.5 EM/18.96 F1 score improvement over the sub-word unit based baseline on the Spoken-SQuAD [1] data, and 13.11 EM/10.99 F1 score improvement on the ODSQA data [2]. By further fine-tuning pre-trained models with existing large scaled textual QA data, we obtained 38.12 EM/34.1 F1 improvement over the baseline of fine-tuned only on small sized real SQA data.

Hate speeches are defined as utterances, writings, actions, or performances that are intended to incite violence or prejudice against a person on the basis of the characteristics of a particular group that he or she is representing, such as race, ethnicity. In this study, we built a hate speech classification model using word representation with continous bag of words (CBOW) and fastText algorithm. This algorithms was chosen, because it is able to achieve a good performance, specially in the case of rare words by making use of character level information. Based on this result, we can see that there is no single, universal variations that outperform other. But in general, models that use pre-trained vectors from Wiki outperform models that do not use pre-trained vectors.

Due to the large amounts of words usually present in documents, some of their appearances can complicate the classification process and make it less accurate. Accordingly, word representation methods have been employed to handle this issue through the use of a comparative study. In this study, we compare the effectiveness of both word embedding and TF-IDF weighting schema by applying four classifiers to assess the accuracy of the classification. To evaluate the effectiveness of our study, it was tested on the popular 20Newsgroup text document dataset. Following our experimentation, we found that using the TF-IDF method and ANN classifiers on the 20Newsgroup dataset greatly enhanced the text documents' classification compared against the use of word embedding and other classifiers.

In the area of Natural Language Processing, high-quality word representations play key roles in neural language processing tasks. Recently, various model designs and methods have blossomed in the domain of word representation. In this paper, we will explain the theories of two major language models such as autoencoding (AE) and autoregressive (AR), and illustrate the architectures of several notable examples of AE and AR including ELMo, GPT, BERT and XLnet. By comparing the pros and cons of these models theoretically and the performances of these models in experiments, we will deepen the understanding of various learning methods and realize the trend in the development of language models.

This paper proposes an answer ranking method used in Knowledge Base Question Answering (KBQA) system. This method first extracts the features of predicate sequence similarity based on sememe vector, predicates’ edit distances, predicates’ word co-occurrences and classification. Then the above features are used as inputs of the ranking learning algorithm Ranking SVM to rank the candidate answers. In this paper, the experimental results on the data set of KBQA system evaluation task in the 2016 Natural Language Processing & Chinese Computing (NLPCC 2016) show that, the method of word similarity calculation based on sememe vector has better results than the method based on word2vec. Its accuracy, recall rate and average F1 value respectively are 73.88%, 82.29% and 75.88%. The above results show that the word representation with knowledge has import effect on natural language processing.

Named Entity Recognition (NER) is a fundamental task in many NLP applications that seek to identify and classify expressions such as people, location, and organization names. Many NER systems have been developed, but the annotated data needed for learning is not available for low-resource languages, such as Cameroonian languages. In this paper we exploit the low frequency of named entities in text to define a new suitable word representation for named entity recognition. We build the first Ewondo (a Bantu language of Cameroon) named entities recognizer by projecting named entity tags from English using our word representation. In terms of Recall, Precision and F-score, the obtained results show the effectiveness of the proposed word representation

Bio-medical entity recognition extracts significant entities, for instance cells, proteins and genes, which is an arduous task in an automatic system that mine knowledge in bioinformatics texts. In this thesis, we utilized a bidirectional long short-term memory (Bi-LSTM) combined with conditional random fields (CRFs) approach to automatically obtain word representation, obliterated the need for a marvelous number of feature engineering tasks. The consequences of this experiment represent the word representation method can effectually acquire potential semantic information. Without relying on any artificial features, the result on the test dataset obtained 76.81 % F-score. Therefore, the proposed method is expected to advance biomedical text mining in bioinformatics entity recognition.

In this paper, we propose a model based on the bag of mixed-gram terms to deal with sentiment classification task and extracting sentimental features. We obtain a very short-dimensional vector to represent sentiment and use the sentimental representations to complete the task of sentiment classification. Furthermore, since the sentimental representations and some traditional word vectors have complementary advantages, we combine the sentimental representations with convolutional neural networks that use other word vectors and ultimately implement a more efficient classifier. Experimental results show that this combination method can use static word vectors to deal with sentimental classification tasks well, and the sentimental representations here play the role of fine-turned word vectors in previous research.

Extracting event information from Twitter is still promising since there are many Twitter accounts built just to spread the event information broadly. The most difficult part to extract event information is the Out of Vocabulary (OOV) problem, especially for event name. Here, we tried to enhance the features used for our supervised event extraction. These features include the word representation (skip-gram model and brown cluster), word list (event name and event location), word context and document level feature. By using CRF as the classification algorithm, 4 fold cross validation technique, and 1,300 tweets, the best F-Measure score achieved for OOV cases was 0.6 which is a significant improvement compared to the baseline of 0.445. The enhanced features also improved the F-Measure score for all vocabulary case from 0.693 (baseline) into 0.814 (proposed).

Word clouds have been widely used to present the contents and themes in the text for summary and visualization. In this paper, we propose a new semantic word cloud taking into account the word semantic meanings. Distributed word representation is applied to accurately describe the semantic meaning of words, and a word similarity graph is constructed based on the semantic distance between words to lay out words in a more compact and aesthetic manner. Word-related interactions are introduced to guide users fast read and understand the text. We apply the proposed word cloud to user generated reviews in different fields to demonstrate the effectiveness of our method.

International audience; This paper deals with the problem of high-quality transcription systems for very large vocabulary automatic speech recognition (ASR). We investigate the problem of automatic retrieval of out-of-vocabulary (OOV) proper names (PNs). We want to take into account the temporal, syntactic and semantic context of words. Nowadays, Artificial Neural Networks (NN) are widely used in natural language processing: continuous space representations of words is learned automatically from unstructured text data. To model the latent topics at document level, Latent Dirichlet Allocation (LDA) has been successful. In this paper, we propose OOV PN retrieval using (1) temporal versus topic context modeling; (2) different word representation spaces for word-level and document-level context modeling; (3) combinations of retrieval results. Experimental evaluation on broadcast news data shows that the proposed method combinations lead to better results. This confirms the complementarity of methods.