Search

The rapidly developing internet and other media have produced a tremendous amount of text data, making it a challenging and valuable task to find a more effective way to analyze text data by machine. Text representation is the first step for a machine to understand the text, and the commonly used text representation method is the Bag-of-Words (BoW) model. To form the vector representation of a document, the BoW model separately matches and counts each element in the document, neglecting much correlation information among words. In this paper, we propose a network-based bag-of-words model, which collects high-level structural and semantic meaning of the words. Because the structural and semantic information of a network reflects the relationship between nodes, the proposed model can distinguish the relation of words. We apply the proposed model to text classification and compare the performance of the proposed model with different text representation methods on four document datasets. The results show that the proposed method achieves the best performance with high efficiency. Using the Eccentricity property of the network as features can get the highest accuracy. We also investigate the influence of different network structures in the proposed method. Experimental results reveal that, for text classification, the dynamic network is more suitable than the static network and the hybrid network.

The Named Entity Recognition (NER) is an integrated task in many NLP applications such as machine translation, Information extraction and question answering. Arabic is one of the authorised spoken languages in the united nation. Currently, there is much Arabic information on the internet, so, nowadays the need for tools which process this information becomes significant. In this study, we have examined the impact of the conditional random field and the structured support vector machine in the task of Arabic NER. The structured support vector machine is the first time to be applied in the Arabic name entity recognition. Our proposed system has three stages: Preprocessing, extracting features and building model. We have used simple features like the bag of words in the [-1,1] window, the bag of part of speech tag in the [-1,1] window to enable our system to detect the multi-words entities. Also, we have tried to enhance the Stanford part of speech tagger to enhance the tagger output tags, which enabled our system to differentiate between the name entities from the non-entities. In addition, we have employed the binary features of: Is a person, is a prename, is a pre-location, is a location and is an organization. Our system has been trained and tested on part of ANER Crop. The results have proved that the conditional random field-based Arabic NER system outperforms the structured support vector machine-based Arabic NER using the same features set.

In near real-time photogrammetry, the first step in processing each new added image is determining the most relevant image in pre-sequence unordered images quickly and exactly, which is pivotal for accurate image matching and 3D reconstruction. This paper presents a hierarchical image retrieval algorithm based on multiple features and details the choice for representation of multiple features which is critical to the improvement of accuracy of this algorithm. First, we represent global features using AlexNet-FC7(fully connected layers) or ResNet101-Pool5(pooling layers) and local features using SIFT (scale-invariant feature transform) in two parallel threads with support of GPU (Graphics Processing Unit). Next, we obtain candidates based on cosine similarities between global features of each pre-sequence image and new added image. Finally, we determine the most relevant image from those candidates according to feature matching results for each candidate and new added image. The experimental results confirm that the second step is rather fast and the third step is necessary to tackle the problem that global features cannot distinguish objects from the same class. The total time our algorithm takes is about 83.6ms for determining the most relevant image in 5063 pre-sequence unordered images of size $1024\times 768$ , which outperforms exhaustive pairwise matching, Bag of Words and multi-vocabulary trees. Accuracy of our algorithm also perform better than the state-of-the-art methods on three benchmark datasets. SIFT matching results obtained in the third step after eliminating mismatches with RANSAC (Random Sample Consensus) can also be used for high-precision incremental SFM (Structure from Motion) reconstruction.

We categorize the user comments posted on YouTube video sharing website based on their relevance to the video content given by the description associated with the video posted. Comments are analysed for polarity and are further segregated as positive or negative. A comparative analysis of classifier using the Bag of Words and Association List approaches is presented.

To eliminate the false alarms in the ship target detection effectively for synthetic aperture radar (SAR) images in complex scenes, this article present a novel ship target discrimination algorithm based on bag of words (BOW) model with multiple features and spatial pyramid matching (SPM), which is named MF-SPM-BOW. The proposed discrimination method mainly contains three stages. First, the SAR scale-invariant feature transform (SAR-SIFT) descriptors and gray-level co-occurrence matrix (GLCM) descriptors are extracted as local features to describe the gradient information and texture information of local regions of an image chip. Then, the SPM technique considering its spatial location information-keeping capability is employed to generate global features with excellent discrimination ability. Finally, the support vector machine (SVM) discriminator based on multiple kernel learning is applied to realize feature fusion in image layer and thus identify targets and clutter. Experimental results show that compared with the traditional discrimination methods and the BOW model discrimination methods, the proposed SAR ship target discrimination algorithm achieves better discrimination performance, which can eliminate most of the false alarms in candidate ship target chips effectively.

This paper proposed a novel multi-view interactive behavior recognition method based on local self-similarity descriptors and graph shared multi-task learning. First, we proposed the composite interactive feature representation which encodes both the spatial distribution of local motion of interest points and their contexts. Furthermore, local self-similarity descriptor represented by temporal-pyramid bag of words (BOW) was applied to decreasing the influence of observation angle change on recognition and retaining the temporal information. For the purpose of exploring latent correlation between different interactive behaviors from different views and retaining specific information of each behaviors, graph shared multi-task learning was used to learn the corresponding interactive behavior recognition model. Experiment results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA, i3Dpose dataset and self-built database for interactive behavior recognition.

Sentiment analysis has become a state-of-the-art to make products market adaptive and understand the market gaps to launch updates and variants of a product entity. The contributions of this paper are providing prior point of sale based key features to generate rule based algorithms to predict the attitude and opinion implied. Amazon food reviews data has been used in the paper. We have done a comparative analysis of Random Forest and Extreme Gradient Boosting model using Bag of Words(BoW) and Term Frequency-Inverse Document Frequency(TFIDF) featurizations.

This study examines the relation between non-financial information and companies' financial results, especially stock returns and weighted average cost of capital (WACC) in the Russian Federation. It selected a sample as a source of nonfinancial information containing Annual reports, Sustainable development reports, ESG reports (environmental, social, governance) and Global Reporting Initiative (GRI) reports of public firms listed in the Russian stock market (MOEX) from 2010 to 2018. To evaluate the role of information disclosure there were applied Bloomberg ESG disclosure indexes. The methodology of text analysis was based on latent semantic analysis (LSA) technique. Latent semantic analysis is a method of textual associative patterns recognition and themes of text formalization with rich mathematical background. Besides, we applied classical text mining procedure - bag of words with corporate social performance dictionary. The results demonstrate both significant influence of some themes or information disclosure importance for stakeholders and independent market agents. Firstly, this study serves top-management purposes and expands understanding of non-financial information influence. Second, the results may be used by external agents to detect the core idea vanishing in text massive.

Context encompasses the classification of a certain environment by its key attributes that take the role of semantic markers. It is an abstract representation of a certain data environment. In texts, the context classifies and represents a piece of text in a generalized form. Context can be a recursive construct when summarizing information on a more coarse-grained level. This paper presents identification and standardization of context on different levels of granularity that finally supports faster and more precise information retrieval. The prototypical system presented here applies supervised learning for a semi-automatic approach to extract, distil, and standardize data from text. The approach is based on named-entity recognition and simple ontologies for identification and disambiguation of context. Even though the prototype shown here still represents work in progress, it already demonstrates its potential for mining texts on different levels of context granularity. The paper presents the design of the Contexter system that supports identification and classification of misinformation and fake news around the topic Covid-19.

In order to accelerate the performance of various Natural Language Processing tasks for Roman Urdu, this article for the very first time provides 3 neural word embeddings prepared using most widely used approaches namely Word2vec, FastText, and Glove. The integrity of generated neural word embeddings is evaluated using intrinsic and extrinsic evaluation approaches. Considering the lack of publicly available benchmark datasets, it provides a first-ever Roman Urdu public dataset which consists of 3241 sentiments annotated against positive, negative, and neutral classes. To provide benchmark baseline performance over the presented dataset for Roman Urdu sentiment analysis, we adapt diverse machine learning (Support Vector Machine, Logistic Regression, Naive Bayes), deep learning (convolutional neural network, recurrent neural network), and hybrid deep learning approaches. Performance impact of generated neural word embeddings based representation is compared with other most widely used bag of words based feature representation approaches using diverse machine and deep learning classifiers. In order to improve the performance of Roman Urdu sentiment analysis, it proposes a novel precisely extreme multi-channel hybrid methodology which makes use of convolutional and recurrent neural networks along with pre-trained neural word embeddings. The proposed hybrid approach outperforms adapted machine learning approaches by the significant figure of 9% and deep learning approaches by the figure of 4% in terms of F1-score.

Object classification model used in point clouds requires to be retrained when applying on heterogeneous point clouds that are collected from different kinds of laser scanning systems. The model training procedure needs supervised information from the corresponding laser scanning systems. However, the acquisition of supervised information is labor-intensive and time-consuming. This letter proposes a new framework to exploit objects from point clouds with supervised information (source domain) to directly classify objects from heterogeneous point clouds with no supervised information (target domain). More specifically, to alleviate occlusions and point density variations, intraclass variations in object classification, the proposed framework integrates a bag-of-words model to effectively describe 3-D objects of point clouds. To alleviate the difference of point clouds collected from various devices, a joint distribution adaption model is exploited to build an effective feature transformation to accomplish the adaption in the source and target domains. The proposed framework is validated on data sets, which contain three kinds of point clouds. Extensive experiments show the effectiveness of the proposed framework on classifying 3-D objects in heterogeneous point clouds.

We consider the following problem: given neural language models (embeddings) each of which is trained on an unknown data set, how can we determine which model would provide a better result when used for feature representation in a downstream task such as text classification or entity recognition? In this paper, we assess the word similarity measure through analyzing its impact on word embeddings learned from various datasets and how they perform in a simple classification task. Word representations were learned and assessed under the same conditions. For training word vectors, we used the implementation of Continuous Bag of Words described in [1]. To assess the quality of the vectors, we applied the analogy questions test for word similarity described in the same paper. Further, to measure the retrieval rate of an embedding model, we introduced a new metric (Average Retrieval Error) which measures the percentage of missing words in the model. We observe that scoring a high accuracy of syntactic and semantic similarities between word pairs is not an indicator of better classification results. This observation can be justified by the fact that a domain-specific corpus contributes to the performance better than a general-purpose corpus. For reproducibility, we release our experiments scripts and results.

Detection of early changes in rotational speed is highly required in on-line process monitoring of industrial manufacturing and numerical controlled machining. This paper proposes a Bag-of-Words (BoW) based feature extraction method that uses vibration signal with such a motivation. Initially, the BoW model is adopted to cluster a prior collection of vibration signals. Then, for a new vibration signal, two strategies: histogram-based encoding (HBE) and embedding-based encoding (EBE), are investigated respectively and comprehensively to encode it based on the BoW model, in order to extract its dynamic characteristic. The entropy is subsequently computed, supporting continuous analysis of dynamic machine status over time. Distance metric is finally adopted to make decision by hypothesis testing. The method is validated with both simulated and real-engineering signals. Results reveal excellent performance by using EBE, coupled with Kolmogorov distance, in the proposed method. Comparison with state-of-the-art competitors demonstrate the priority and robustness of the method.

A family of algorithms for time series classification (TSC) involve running a sliding window across each series, discretising the window to form a word, forming a histogram of word counts over the dictionary, then constructing a classifier on the histograms. A recent evaluation of two of this type of algorithm, Bag of Patterns (BOP) and Bag of Symbolic Fourier Approximation Symbols (BOSS) found a significant difference in accuracy between these seemingly similar algorithms. We investigate this phenomenon by deconstructing the classifiers and measuring the relative importance of the four key components between BOP and BOSS. We find that whilst ensembling is a key component for both algorithms, the effect of the other components is mixed and more complex. We conclude that BOSS represents the state of the art for dictionary-based TSC. Both BOP and BOSS can be classed as bag of words approaches. These are particularly popular in Computer Vision for tasks such as image classification. We adapt three techniques used in Computer Vision for TSC: Scale Invariant Feature Transform; Spatial Pyramids; and Histogram Intersection. We find that using Spatial Pyramids in conjunction with BOSS (SP) produces a significantly more accurate classifier. SP is significantly more accurate than standard benchmarks and the original BOSS algorithm. It is not significantly worse than the best shapelet-based or deep learning approaches, and is only outperformed by an ensemble that includes BOSS as a constituent module.

Because the fault features of planetary gear are very weak, the feature extraction process of planetary gear fault under traditional thinking is becoming more and more complex, and the amount of extracted features is very small. To solve these problems, a new fault diagnosis method for planetary gear based on image feature extraction and bag-of-words (BoW) model is proposed, which does not need complex noises reduction and redundant features elimination. In order to extract a large number of features quickly, the FAST-Unoriented-SIFT feature extraction algorithm is proposed, which combines the advantages of FAST and SIFT algorithms, and ignores the orientations of features. After converting the original vibration signals to gray-scale images, the features are extracted directly by FAST-Unoriented-SIFT, and finally the BoW model is built on the extracted features to realize fault classification. The overall recognition rate reaches 98.67%. The experimental results certify the effectiveness of the proposed method.

We implement an efficient methodology for extracting themes from Securities Exchange Commission 13D filings using aspects of human‐assisted active learning and long short‐term memory (LSTM) neural networks. Sentences from the ‘Purpose of Transaction’ section of each filing are extracted and a randomly chosen subset is labelled based on six filing themes that the existing literature on shareholder activism has shown to have an impact on stock returns. We find that an LSTM neural network that accepts sentences as input performs significantly better, with precision of 77%, than an alternately specified neural network that uses the common bag of words approach. This indicates that both sentence structure and vocabulary are important in classifying SEC 13D filings. Our study has important implications, as it addresses the recent cautions raised in the literature that analysis of finance and accounting‐related text sources should move beyond bag‐of‐words approaches to alternatives that incorporate the analysis of word sense and meaning reflecting context.

Semantic Textual Similarity and Natural Language Inference are two popular natural language understanding tasks used to benchmark sentence representation models where two sentences are paired. In such tasks sentences are represented as bag of words, sequences, trees or convolutions, but the attention model is based on word pairs. In this article we introduce the use of word n-grams in the attention model. Our results on five datasets show an error reduction of up to 41% with respect to the word-based attention model. The improvements are especially relevant with low data regimes and, in the case of natural language inference, on the recently released hard subset of Natural Language Inference datasets.

Natural language processing (NLP) is the task of converting unstructured human language data into structured data that a machine can understand. While its applications are far and wide in healthcare, and are growing considerably every day, this chapter will focus on one particularly relevant application for healthcare professionals-reducing the burden of clinical documentation. More specifically, the chapter will discuss two studies (Gopinath et al., Fast, structured clinical documentation via contextual autocomplete. arXiv: 2007.15153, 2020; Greenbaum et al., Contextual autocomplete: a novel user interface using machine learning to improve ontology usage and structured data capture for presenting problems in the emergency department, 2017) that have implemented contextual autocompletion in electronic medical records and their promising results with regards to time saved for clinicians. The goals of this chapter are to introduce to the curious healthcare provider the basics of natural language processing, zoom into the use case of contextual autocomplete for electronic medical records, and provide a hands-on tutorial that introduces the basic NLP concepts required to build a model for predictive suggestions.

Automatically selecting a set of representative views of a 3D virtual cultural relic is crucial for constructing wisdom museums. There is no consensus regarding the definition of a good view in computer graphics; the same is true of multiple views. View-based methods play an important role in the field of 3D shape retrieval and classification. However, it is still difficult to select views that not only conform to subjective human preferences but also have a good feature description. In this study, we define two novel measures based on information entropy, named depth variation entropy and depth distribution entropy. These measures were used to determine the amount of information about the depth swings and different depth quantities of each view. Firstly, a canonical pose 3D cultural relic was generated using principal component analysis. A set of depth maps obtained by orthographic cameras was then captured on the dense vertices of a geodesic unit-sphere by subdividing the regular unit-octahedron. Afterwards, the two measures were calculated separately on the depth maps gained from the vertices and the results on each one-eighth sphere form a group. The views with maximum entropy of depth variation and depth distribution were selected, and further scattered viewpoints were selected. Finally, the threshold word histogram derived from the vector quantization of salient local descriptors on the selected depth maps represented the 3D cultural relic. The viewpoints obtained by the proposed method coincided with an arbitrary pose of the 3D model. The latter eliminated the steps of manually adjusting the model’s pose and provided acceptable display views for people. In addition, it was verified on several datasets that the proposed method, which uses the Bag-of-Words mechanism and a deep convolution neural network, also has good performance regarding retrieval and classification when dealing with only four views.

Covid-19 leads to public fear. People are afraid to exist in crowded places like public transportation, airports, and hotels. Covid-19 has affected the field of tourism negatively. Many factors affect the travelers’ intentions to travel, including safety and security, space accessibility, travel costs, quality issues, sanitation risks, hygiene, and destination trust. The research aims to propose a sentiment classifier model that analyses travelers’ sentiments intention. The TravelerIntention Sentiment model uses three classification techniques: Support Vector Machine, Naive Bayes and Decision Tree. One thousand sentiments were collected and analyzed. The safety and security factor was the highest important factor based on 326 sentiments. Results have shown that Naive Bayes has the highest accuracy when using the Term Frequency Inverse Document Frequency feature selection method, and Support Vector Machine has the highest accuracy level when using the Bag of Words feature selection method.

Depression is a severe mental health problem. Due to its relevance, the development of computational tools for its detection has attracted increasing attention in recent years. In this context, several research works have addressed the problem using word-based approaches (e.g., a bag of words). This type of representation has shown to be useful, indicating that words act as linguistic markers of depression. However, we believe that in addition to words, their contexts contain implicitly valuable information that could be inferred and exploited to enhance the detection of signs of depression. Specifically, we explore the use of user’s characteristics and the expressed sentiments in the messages as context insights. The main idea is that the words’ discriminative value depends on the characteristics of the person who is writing and on the polarity of the messages where they occur. Hence, this paper introduces a new approach based on specializing the framework of classification to profiles of users (e.g., males or women) and considering the sentiments expressed in the messages through a new text representation that captures their polarity (e.g., positive or negative). The proposed approach was evaluated on benchmark datasets from social media; the results achieved are encouraging, since they outperform those of state-of-the-art corresponding to computationally more expensive methods.

In this paper, we consider the task of assigning relevant labels to studies in the social science domain. Manual labelling is an expensive process and prone to human error. Various multi-label text classification machine learning approaches have been proposed to resolve this problem. We introduce a dataset obtained from the Finnish Social Science Archive and comprised of 2968 research studies’ metadata. The metadata of each study includes attributes, such as the “abstract” and the “set of labels”. We used the Bag of Words (BoW), TF-IDF term weighting and pretrained word embeddings obtained from FastText and BERT models to generate the text representations for each study’s abstract field. Our selection of multi-label classification methods includes a Naive approach, Multi-label k Nearest Neighbours (ML-kNN), Multi-Label Random Forest (ML-RF), X-BERT and Parabel. The methods were combined with the text representation techniques and their performance was evaluated on our dataset. We measured the classification accuracy of the combinations using Precision, Recall and F1 metrics. In addition, we used the Normalized Discounted Cumulative Gain to measure the label ranking performance of the selected methods combined with the text representation techniques. The results showed that the ML-RF model achieved a higher classification accuracy with the TF-IDF features and, based on the ranking score, the Parabel model outperformed the other methods.

Human activity recognition based on sensors (e.g., accelerometer and gyroscope) embedded in smartphones is of great significance for many applications under uncontrolled environments. Although significant progress has been noticed in this field, one of the challenges limiting its real-life applications lies in robust feature extraction for efficient activity recognition on smartphones. This study addresses this challenge by proposing an improved bag-of-words representation for activity signal characterization. Specifically, raw activity signals are processed by discrete wavelet transformation to extract local features, which will be clustered using K-means to form a bag-of-words dictionary. The vocabularies in the dictionary are regarded as bin centers for histogram feature construction. For each local feature of an activity signal, its distance from all the bin centers will be measured. To capture higher-order information for feature representation, the frequency for the bin centers corresponding to the minimum n distances will be updated. Moreover, the frequency is increased by a trigonometry constraint cosine value of the corresponding distances to account for activity signals’ structural information. The proposed feature representation has been verified with three well-established classifiers, namely SVM, ANN, and KNN on the UCI-HAR dataset. The consistently good performance validates the effectiveness and robustness of the proposed feature representation. Compared with the state-of-the-art, the experimental results also demonstrate the advantage of the proposed method in terms of accuracy and computational cost.

Many efforts have been deployed by the IR community to extend free-text query processing toward semi-structured XML search. Most methods rely on the concept of Lowest Comment Ancestor (LCA) between two or multiple structural nodes to identify the most specific XML elements containing query keywords posted by the user. Yet, few of the existing approaches consider XML semantics, and the methods that process semantics generally rely on computationally expensive word sense disambiguation (WSD) techniques, or apply semantic analysis in one stage only: performing query relaxation/refinement over the bag of words retrieval model, to reduce processing time. In this paper, we describe the building blocks of a new approach for XML keyword search aiming to solve the limitations mentioned above. Our solution first transforms the XML document collection (offline) and the keyword query (on-the-fly) into meaningful semantic representations using context-based and global disambiguation methods, specially designed to allow almost linear computation efficiency. Consequently, the semantically augmented XML data tree is processed for structural node clustering, based on semantic query concepts (i.e., key-concepts), in order to identify and rank candidate answer sub-trees containing related occurrences of query key-concepts. Preliminary experiments highlight the quality and potential of our approach.

The task of adhoc search is undergoing a renaissance, sparked by advances in natural language processing. In particular, pre-trained contextualized language models (such as BERT and T5) have consistently shown to be a highly-effective foundation upon which to build ranking models. These models are equipped with a far deeper understanding of language than the capabilities of bag-of-words (BoW) models. Applying these techniques to new tasks can be tricky, however, as they require knowledge of deep learning frameworks, and significant scripting and data munging. In this full-day tutorial, we build up from foundational retrieval principles to the latest neural ranking techniques. We first provide foundational background on classical bag-of-words methods. We then show how feature-based Learning to Rank methods can be used to re-rank these results. Finally, we cover contemporary approaches, such as BERT, doc2query, and dense retrieval. Throughout the process, we demonstrate how these can be easily experimentally applied to new search tasks in a declarative style of conducting experiments exemplified by the PyTerrier and OpenNIR search toolkits. This tutorial is interactive in nature for participants. It is broken into sessions, each of which mixes explanatory presentation with hands-on activities using prepared Jupyter notebooks running on the Google Colab platform. These activities give participants experience applying the techniques covered in the tutorial on the TREC COVID benchmark test collection. The tutorial is broken into four sessions. In the first session, we cover foundational retrieval concepts, including inverted indexing, retrieval, and scoring. We also demonstrate how evaluation can be conducted in a declarative fashion within PyTerrier, encapsulating ideas such as significance testing, and multiple correction, as promoted as IR best practices. In the second session, we build upon the core retrieval concepts to demonstrate how to re-write queries (e.g., using RM3) and re-rank documents (e.g., using learning-to-rank). In the third session, we introduce contextualized language models, such as BERT and show how they can be utilized for document re-ranking (e.g, using Vanilla/monoBERT and EPIC). Finally, in session four, we move beyond re-ranking and cover how approaches that modify documents (e.g., DeepCT) as well as efforts to replace the traditional inverted index with an embedding-based index (e.g., ANCE, ColBERT, and ColBERT-PRF). By the end of the tutorial, participants will have experience conducting IR experiments from classical bag-of-words models to contemporary BERT models and beyond.

To detect unknown malware, several methods with machine learning techniques have been proposed. These methods extract the features of malware, and thereby classify samples as malicious or benign. The previous method extracts words from the source code and detects macro malware. This method constructs a language model latent semantic indexing (LSI) to extract the features. On the other hand, several methods to avoid the detection have been proposed. These methods add benign features to the source code for Android malware. These methods can be applied to the macro malware. In this study, we discuss the risk of macro malware that evades detection. This paper attempts to imitate benign macros by adding benign features to macromalware. Our method extracts the variables from macro malware and replaces them to frequent words of benign macros. Furthermore, our method inserts the frequent words that appear in benign macros or the LSI topics. These words are inserted as the arguments of some functions, which do not vary the main behavior. The target detection model is our previous method, which detects macromalware with language models: Bag of words (BoW) and LSI. The detection rate is evaluated under two conditions that the attacker can access inside the model or not. As a result, the detection rate with BoW decreases to 1.5% under the situation that the attacker can access inside the model. Even if the attacker cannot access inside the model, the detection rate with LSI decreases by 73%.

Democracies rely on the capability of a country to conduct fair elections. And, fair elections rely on an open participation of candidates and general public. In particular, we explore a way to compare campaign proposals aiding general public to make an informed decision while choosing candidates. This document explores a way to compare campaign proposals through each candidate manifest using natural language processing techniques (i.e. Doc2Vec algorithm). As a linguistic corpus we used all the articles written in Spanish from Wikipedia and we used two models of neural networks, Distributed Bag of Words (DBOW) and Distributed Memory Model (DM). We chose the 2021 Ecuadorian Presidential Elections in its second round and tagged each manifesto paragraph (from the runoff candidates) into the seven domains according to the Manifesto Project. Finally, we compute manifesto comparisons by topic and also as a whole for different vector configurations. Our results indicate that Doc2Vec produces reasonable results while comparing documents but the DBOW model provides better results while dealing with larger documents and the DM model while dealing with smaller ones.

The problem of fake deceptive reviews has become a threatening aspect for online users in recent years. With the evolution of the online markets, the trend towards fake reviews has increased, mainly to attract or distract customers. Fake reviews have affected both customers and sellers. These reviews consist of writings and spreading misleading information and beliefs. Sentiment analysis was first introduced a few years ago in the e-commerce sector. It is an emerging research area today due to the rapid growth in the e-commerce industry. The biggest challenge in detecting fake reviews is the lack of an effective way to distinguish fake reviews from legitimate reviews. The difference cannot be seen with the naked eye and is, therefore, a severe concern. In this paper, we have applied the bag of words model and glove embedding matrix with a focus on fake reviews. We have used two different feature extraction techniques and three new deep-learning algorithms on text classifications. The experimental analysis with an existing public dataset showed good and better results compared to the traditional machine-learning models.

In this paper we perform a comparative analysis of three models for a feature representation of text documents in the context of document classification. In particular, we consider the most often used family of bag-of-words models, the recently proposed continuous space models word2vec and doc2vec, and the model based on the representation of text documents as language networks. While the bag-of-word models have been extensively used for the document classification task, the performance of the other two models for the same task have not been well understood. This is especially true for the network-based models that have been rarely considered for the representation of text documents for classification. In this study, we measure the performance of the document classifiers trained using the method of random forests for features generated with the three models and their variants. Multi-objective rankings are proposed as the framework for multi-criteria comparative analysis of the results. Finally, the results of the empirical comparison show that the commonly used bag-of-words model has a performance comparable to the one obtained by the emerging continuous-space model of doc2vec. In particular, the low-dimensional variants of doc2vec generating up to 75 features are among the top-performing document representation models. The results finally point out that doc2vec shows a superior performance in the tasks of classifying large documents.

Online media have flourished in modern years to connect with the world. Most of those stuff users share on blogs like facebook, twitter and many other are pessimistic or just middle spirited. Further, an increasingly professional anti - spyware technologies are dependent on Machine Learning(ML) technology to secure malicious consumers. Over the past few years, revolutionary learning approaches have yielded remarkable outcomes and have immediately generated photos, characters and text interpretations of dynamic weak points. The Purple consumer frequency makes the troll and attacker aim an enticing one. The users will learn the controversial topics and techniques used by malware from articles with ties to harmful material and bogus applications. It is essential to build and customize a lot of potential functionality in vulnerability and application developers around the world. To represent a public web firmware assault with deep logistic inference using Extreme Spontaneous Tree (FAI-DLB). A corresponding output device is named harmful or benign by training an FAI-DLB with different modulation clustered with such a normal or anomalous API. It was therefore equipped to locate a suspicious sequence in unidentified firmware of FAI Deep LB. The outcome demonstrates a good actual meaning of 96.25% and a low spyware assault of 0.03%.

Over the past few years, social media platforms like Twitter, Facebook, Reddit, Instagram, etc. have become a medium for one to share their feelings and experiences. This stands as a significant opportunity for the research community to utilize such data for early diagnosis of psychological issues like “depression” (a mood disorder marked by a sense of despair and detachment). This paper emphasizes comparing the performance of various machine learning models on well-labeled twitter data and deals with a class imbalance problem. This paper outlines various baseline feature engineering methods with different machine learning techniques. Evaluation metrics such as f1-score, precision, and recall have been used to compare the performance of these models. This research work yields better results using Bag of Words with AdaBoost Classifier compared to other machine learning classifiers.

Smoking is an important variable for clinical research, but there are few studies regarding automatic obtainment of smoking classification from unstructured bilingual electronic health records (EHR). We aim to develop an algorithm to classify smoking status based on unstructured EHRs using natural language processing (NLP). With acronym replacement and Python package Soynlp, we normalize 4711 bilingual clinical notes. Each EHR notes was classified into 4 categories: current smokers, past smokers, never smokers, and unknown. Subsequently, SPPMI (Shifted Positive Point Mutual Information) is used to vectorize words in the notes. By calculating cosine similarity between these word vectors, keywords denoting the same smoking status are identified. Compared to other keyword extraction methods (word co-occurrence-, PMI-, and NPMI-based methods), our proposed approach improves keyword extraction precision by as much as 20.0%. These extracted keywords are used in classifying 4 smoking statuses from our bilingual EHRs. Given an identical SVM classifier, the F1 score is improved by as much as 1.8% compared to those of the unigram and bigram Bag of Words. Our study shows the potential of SPPMI in classifying smoking status from bilingual, unstructured EHRs. Our current findings show how smoking information can be easily acquired for clinical practice and research.

Natural Language Processing has become an important issue with the rapid increase in textual data in the health sector recently. Especially with the effect of COVID-19, easy and fast analysis of health data is important for research. Traditional text representations such as BoW (bag of words), TF-IDF (term frequency-inverse document frequency), and modern word representation methods such as FastText and BERT are used to represent words. The BERT models are provided high performance recently. The BERT models are divided into pre-trained and fine-tuned BERT models. In order to get good results in the field of health, BioBERT models are obtained by fine-tuning the basic BERT models with datasets containing biomedical articles. In this study, semantic similarities in datasets are evaluated by the Pearson correlation method by using BoW, TF-IDF, FastText, BERT, and BioBERT models. As a result of the evaluations, it was observed that BioBERT models gave higher values compared to other models and methods used.

In the last years, the environment in which we develop has had various changes, most of them due to major technological changes, and the constant development of Information and Communication Technologies (ICTs). As a result of the advances in ICTs, today it is common to interact through social networks and make constant use of them. For this reason, this paper presents an analysis of the 10 most influential Twitter accounts in Ecuador; the objective of this analysis is to detect what the topics or topics addressed in these accounts are. The Latent Dirichlet Allocation (LDA) algorithm using bag of words (BoW) model and also the Term Frequency–Inverse Document Frequency (TF-IDF) model were used for the analysis, finally finding, if the topics provided by both models are similar.

Sentiment analysis is needed because the analysis of the emotion is somehow complicated due to slang phrases, misspellings, short-forms, recurring characters, the use of dialects, and modern emoticons. The aim of this project is to develop a system to analyze the sentiment of movie reviews and visualize the result of sentiment analysis. The dataset was captured from IMDb movie reviews. This dataset contains 50,000 instances with two columns which are the reviews and their sentiments. The data set undergoes a cleaning process to extract meaningful data. Two types of features have been investigated, namely, the Bag of Words (BoW) and the TF-IDF modelling with Naive Bayes classifiers. The result of accuracy is 89% which indicates that the Naive Bayes can be used for sentiment analysis in any movie review task.

Loop closure detection is an important component of visual simultaneous localization and mapping (SLAM). However, most existing loop closure detection methods are vulnerable to complex environments and use limited information from images. As higher-level image information and multi-information fusion can improve the robustness of place recognition, a semantic–visual–geometric information-based loop closure detection algorithm (SVG-Loop) is proposed in this paper. In detail, to reduce the interference of dynamic features, a semantic bag-of-words model was firstly constructed by connecting visual features with semantic labels. Secondly, in order to improve detection robustness in different scenes, a semantic landmark vector model was designed by encoding the geometric relationship of the semantic graph. Finally, semantic, visual, and geometric information was integrated by fuse calculation of the two modules. Compared with art-of-the-state methods, experiments on the TUM RBG-D dataset, KITTI odometry dataset, and practical environment show that SVG-Loop has advantages in complex environments with varying light, changeable weather, and dynamic interference.

False or misleading information, created deliberately to misinform and deceive the reader, is referred to as Fake News. As the most extensively utilized network for disseminating information, social media has become a highly popular platform for individuals to propagate incorrect information. Fake news can be pernicious when it comes to specific individuals and their opinions and perceptions, influencing and misleading their actions. Several initiatives have been taken in the past to reduce the spread of fake news and detect inaccurate information by devoting a significant amount of time and effort. In this paper, we have compared the performance of various Machine Learning and Deep Learning models for Fake News Detection. For our study, we used four datasets. From our experimentation, we realized that Random Forest and Bag of Words on the FARN Dataset outperformed the rest with an accuracy of 98.8%. In addition, we discovered that TF-IDF outperforms other feature extraction methods.

Covid-19 is a disease caused by a virus and has become a pandemic in many countries around the world. The disease not only affects public health, but also affects other aspects of life. People tend to write comments about things happening during the pandemic on social media, one of which is Twitter. Sentiment analysis on Twitter data is not an easy task due to the characteristics of the tweeter text which is user generated content. Therefore, in this paper, a sentiment analysis study is carried out on Twitter data using three schemes, namely the vector space model (Bag of Words and TF-IDF) with Support Vector Machine, word embedding (word2vec and Glove) with Long Short-Term Memory, and BERT (Bidirectional Encoder Representations from Transformers). Based on the conducted experiments, BERT achieved the best performance compared to the other two schemes, reaching 0.85 (weighted F1-score) and 0.83 (macro F1-score) for the classification of three sentiment classes on Kaggle competition data (Coronavirus tweets NLP – Text Classification).

The buying decisions of consumers across the world are increasingly influenced by online reviews. With this in mind, there exists increased opportunity for unscrupulous business owners or spammers to create false reviews to promote their brand or smear those of their competitors. As a result, there have been many studies of ways to identify review spam. However, a common limitation in these studies is the lack of large amounts of labeled training data. This is problematic because Deep Learning is susceptible to overfitting when trained on small datasets. In this paper, we empirically explore the use of inductive transfer learning to help with the problem by using a language model trained on Wikipedia and fine-tuning it for the task of hotel review spam detection. We show that it is possible to use inductive transfer learning to create a deep learning model that outperforms traditional bag of words based approaches on datasets with limited labeled data.

Sarcasm emerges as a common phenomenon across social networking sites because people express their negative thoughts, hatred and opinions using positive vocabulary which makes it a challenging task to detect sarcasm. Although various studies have investigated the sarcasm detection on baseline datasets, this work is the first to detect sarcasm from a multi-domain dataset that is constructed by combining Twitter and News Headlines datasets. This study proposes a hybrid approach where the convolutional neural networks (CNN) are used for feature extraction while the long short-term memory (LSTM) is trained and tested on those features. For performance analysis, several machine learning algorithms such as random forest, support vector classifier, extra tree classifier and decision tree are used. The performance of both the proposed model and machine learning algorithms is analyzed using the term frequency-inverse document frequency, bag of words approach, and global vectors for word representations. Experimental results indicate that the proposed model surpasses the performance of the traditional machine learning algorithms with an accuracy of 91.60%. Several state-of-the-art approaches for sarcasm detection are compared with the proposed model and results suggest that the proposed model outperforms these approaches concerning the precision, recall and F1 scores. The proposed model is accurate, robust, and performs sarcasm detection on a multi-domain dataset.

open access article Sentiment analysis of social media textual posts can provide information and knowledge that is applicable in social settings, business intelligence, evaluation of citizens' opinions in governance, and in mood triggered devices in the Internet of Things. Feature extraction and selection is a key determinant of accuracy and computational cost of machine learning models for such analysis. Most feature extraction and selection techniques utilize bag of words, N‐grams, and frequency‐based algorithms especially Term Frequency‐Inverse Document Frequency. However, these approaches do not consider relationships between words, they ignore words' characteristics and they suffer high feature dimensionality. In this paper we propose and evaluate a feature extraction and selection approach that utilizes a fixed hybrid N‐gram window for feature extraction and minimum redundancy maximum relevance feature selection algorithm for sentence level sentiment analysis. The approach improves the existing features extraction techniques, specifically the N‐gram by generating a hybrid vector from words, Part of Speech (POS) tags, and word semantic orientation. The vector is extracted by using a static trigram window identified by a lexicon where a sentiment word appears in a sentence. A blend of the words, POS tags, and the sentiment orientations of the static trigram are used to build the feature vector. The optimal features from the vector are then selected using minimum redundancy maximum relevance (MRMR) algorithm. Experiments were carried out using the public Yelp dataset to compare the performance of the proposed model and existing feature extraction models (BOW, normal N‐grams and lexicon‐based bag of words semantic orientations). Using supervised machine learning classifiers the experimental results showed that the proposed model had the highest F‐measure (88.64%) compared to the highest (83.55%) from baseline approaches. Wilcoxon test carried out ascertained that the proposed approach performed significantly better than the baseline approaches. Comparative performance analysis with other datasets further affirmed that the proposed approach is generalizable.

In this paper, document feature representations coupled with K-Means clustering-based methods are investigated to extract the public perceptions towards the COVID-19 vaccine from the social media platform Twitter. First, Distributed Bag of Words (PV-DBOW) model was adopted to find an effective fixed-length document vector representation from all the downloaded tweets. Second, the extracted document embedding was fed into several K-means clustering-based methods for partitioning into different topics by measuring their distances to the centroids of the clusters. Different internal validation measurements were further performed to evaluate the effectiveness of the identified clustering structure and the optimal number of clusters. Our analysis identified four topics from the downloaded tweets: (1) national vaccine rollout, (2) vaccine and its relation to death, (3) vaccine approval and (4) vaccine hesitancy.

Aiming at the problem that the visual SLAM algorithm is easily disturbed by complex environmental factors, resulting in poor closed-loop accuracy and real-time performance. An improved closed-loop detection algorithm based on a dynamic increase or decrease mechanism is proposed. The algorithm can dynamically increase or decrease the number of words in the visual bag of words in real time according to the complexity of the scene; and optimize the selected keyframes through a double-threshold keyframe selection strategy to reduce the redundancy of keyframes, thus improving the system's real-time performance. The algorithm in this paper can achieve localization and position correction in complex environments. Test results in public datasets and real scenes demonstrate the advantages of the proposed algorithm in terms of accuracy, real-time performance, and adaptability to the environment.

The increasing amount of data on the Internet has been a constant challenge when text cleaning and relevant-text extraction are of interest. One of the areas of focus on the internet is the Dark Web; the data here is much more volatile and dynamic. With more researchers looking for data and extracting information, the algorithms have always been in a state of constant improvement. The solutions currently offered, all work based on text feature extraction algorithms like TF-IDF, Bag of Words, Word2Vec. Discussion on these extraction methods is well documented but a critical evaluation among these algorithms is amiss from standard literature. This paper discusses a balanced approach for tagging extracted data; ELEMENT (Effective Lemmatization, Efficient Management of Extracting Noteworthy Tags) which is a modified form of TF-IDF. Having a balanced approach like ELEMENT will benefit from being able to perform well under any given circumstance. The paper discusses and compares the proposed approach with existing strategies of text feature extraction. This comparison spans across accuracy of feature extraction, efficiency concerning Time and Space, concluding with a simplistic view of the strengths and weaknesses of each algorithm.

The capital investment for automated analysis of electronic documents has enhanced rapidly since the growth of text categorization and classification. In recent times, various works have been done in the context of text mining and information retrieval. This paper proposes a new method for information retrieval using latent semantic analysis (LSA) and tf-idf also known as term frequency—inverse document frequency. The proposed method helps to identify the most and least significant words from sentences of the paragraph obtained from a document by splitting the sentences to make the most significant words a query of each sentence. If the word is of the highest priority, then the document will be saved in the drive as a star file. The star file consists of the highlighted less significant words in that document and an array of higher probability words for the respective word at the top of its sentence. The proposed system has been implemented by using a machine learning pipeline, Kubeflow. Based on the analysis of the estimated parameters and calculations of the output graph slope in the system, the model gave a hearty probability of 0.9427 that refers to the accuracy of the system.

Although the number of labeled datasets in Earth Observation (EO) is increasing, there is still a major gap between the Deep Learning (DL) classifiers designed in this field versus the models in Computer Vision. This gap is produced mainly by the number of datasets available, but also by the diversity of data. In EO, there are different sensors acquiring images, from multispectral (MS) or hyperspectral data, to SAR imagery. In this paper, we want to demonstrate how to reduce the divergence created by the diversity of data. We trained several DL architectures on Bag-of-Words from large-scale MS and SAR datasets, and then we used transfer learning on smaller ones and evaluated the results. With this method, we demonstrate that a DL architecture can be trained with any type of large-scale data, transformed into Bag-of-Words, and the trained model can be used further on other types of data, without regard on the number of channels.

We present the Game Walkthrough Corpus (GWTC), which contains 12,295 unique walkthrough documents covering 6,117 games. For each game walkthrough, we provide frequencies of unigrams and bigrams, treating the walkthrough document as a Bag of Words. In addition, we provide word frequencies at the sentence level. Furthermore, the GWTC contains a number of game-related metadata, including title, publisher, developer, year, and genre. All the language statistics and metadata are stored in separate plain text files and can be referenced through uniform resource names (URN). These URNs can also be used to derive any combination of statistics and metadata. Researchers, for instance, can investigate the most frequent unigrams for games in the “Adventure” genre. This way, the GWTC can be reused for different kinds of research questions on gaming language.

Standardized approaches to relevance classification in information retrieval use generative statistical models to identify the presence or absence of certain topics that might make a document relevant to the searcher. These approaches have been used to better predict relevance on the basis of what the document is “about”, rather than a simple-minded analysis of the bag of words contained within the document. In more recent times, this idea has been extended by using pre-trained deep learning models and text representations, such as GloVe or BERT. These use an external corpus as a knowledge-base that conditions the model to help predict what a document is about. This paper adopts a hybrid approach that leverages the structure of knowledge embedded in a corpus. In particular, the paper reports on experiments where linked data triples (subject-predicate-object), constructed from natural language elements are derived from deep learning. These are evaluated as additional latent semantic features for a relevant document classifier in a customized news-feed website. The research is a synthesis of current thinking in deep learning models in NLP and information retrieval and the predicate structure used in semantic web research. Our experiments indicate that linked data triples increased the F-score of the baseline GloVe representations by 6% and show significant improvement over state-of-the art models, like BERT. The findings are tested and empirically validated on an experimental dataset and on two standardized pre-classified news sources, namely the Reuters and 20 News groups datasets.