Search

This work presents an alternative method to represent documents based on LDA (Latent Dirichlet Allocation) and how it affects to classification algorithms, in comparison to common text representation. LDA assumes that each document deals with a set of predefined topics, which are distributions over an entire vocabulary. Our main objective is to use the probability of a document belonging to each topic to implement a new text representation model. This proposed technique is deployed as an extension of the Weka software as a new filter. To demonstrate its performance, the created filter is tested with different classifiers such as a Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), and Naive Bayes in different documental corpora (OHSUMED, Reuters-21578, 20Newsgroup, Yahoo! Answers, YELP Polarity, and TREC Genomics 2015). Then, it is compared with the Bag of Words (BoW) representation technique. Results suggest that the application of our proposed filter achieves similar accuracy as BoW but greatly improves classification processing times. Xunta de Galicia | Ref. ED431C2018/55

Machine learning techniques are widely used to take intelligent decisions in industrial and educational domains. In the educational domain, when a research scholar submits a dissertation, then it has to be indexed and classified. The number of dissertations that are submitted in an educational institute is usually high and if done manually, it becomes difficult to index and classify correctly. This study applies machine learning techniques to automate the indexing and categorization of dissertations. We have focused on dissertations from the Engineering, Medical, Social Science, and General Science fields. We used the Bag of Words (BoW) method to extract features and K-means, Density-based spatial clustering of applications with noise (DBSCAN) and Expectation-Maximisation (EM) to train our model. Our experimental results reveal that the proposed K- means technique for indexing and categorization leads to higher accuracy and significant reduction in negative predictions as compared to DBSCAN and Expectation-Maximisation (EM).

Data mining is the primary method of gathering large volumes of knowledge. To make use of such data to implementation requires the use of effective machine learning strategies. Semantic Textual Similarity is one of the primary machine learning strategies. At its core, semantic textual similarity is the identification of words with similar context. Initial work in STS involved text reuse, word search among others. The proposed research work uses a specific method of determining textual similarity using Google’s Word2Vec framework and the Continuous-bag-of-words algorithm for identifying word similarity in rap records. A large data set consisting of over 50,000 rap records is used. The key aspect of proposed methodology is to determine the words with similar context and cluster them into different word clusters also called bags. To achieve the desired result, the dataset is first processed to obtain the features. Once the features are selected, a model is generated by passing the data onto the Word2Vec framework. The research work on semantic textual similarity was repeated across three different runs, with the data set size changing in every run. At the end of each the accuracy of similarity obtained by the model was determined. The current research work has achieved average accuracy as 85%.

Arabic is one of the most semantically and syntactically complex languages in the world. A key challenging issue in text mining is text summarization, so we propose an unsupervised score-based method which combines the vector space model, continuous bag of words (CBOW), clustering, and a statistically-based method. The problems with multidocument text summarization are the noisy data, redundancy, diminished readability, and sentence incoherency. In this study, we adopt a preprocessing strategy to solve the noise problem and use the word2vec model for two purposes, first, to map the words to fixed-length vectors and, second, to obtain the semantic relationship between each vector based on the dimensions. Similarly, we use a k-means algorithm for two purposes: (1) Selecting the distinctive documents and tokenizing these documents to sentences, and (2) using another iteration of the k-means algorithm to select the key sentences based on the similarity metric to overcome the redundancy problem and generate the initial summary. Lastly, we use weighted principal component analysis (W-PCA) to map the sentences&rsquo, encoded weights based on a list of features. This selects the highest set of weights, which relates to important sentences for solving incoherency and readability problems. We adopted Recall-Oriented Understudy for Gisting Evaluation (ROUGE) as an evaluation measure to examine our proposed technique and compare it with state-of-the-art methods. Finally, an experiment on the Essex Arabic Summaries Corpus (EASC) using the ROUGE-1 and ROUGE-2 metrics showed promising results in comparison with existing methods.

In this paper we introduce a framework for annotating a social media text corpora for various categories. Since, social media data is generated via individuals, it is important to annotate the text for the individuals demographic attributes to enable a socio-technical analysis of the corpora. Furthermore, when analyzing a large data-set we can often annotate a small sample of data and then train a prediction model using this sample to annotate the full data for the relevant categories. We use a case study of a Facebook comment corpora on student loan discussion which was annotated for gender, military affiliation, age-group, political leaning, race, stance, topicalilty, neoliberlistic views and civility of the comment.

Companies have gained important advantages by the development of electronic commerce. Consumer evaluations in electronic environment offer great possibilities for analysis. The fact that the consumer opinions are comprised of textual data, analyzes have complicated and challenging process. In recent years, it is seen that text mining methods are used in analyzes in the literature. However, the evaluations of consumers which are formed by short texts make it necessary to realize the analysis with insufficient data. The weighting methods such as Term Frequency and Term Frequency-Inverse Document Frequency as well as common used classification algorithms such as Naïve Bayes and Support Vector Machine have some inadequacies in short text analysis. In this study, a grey relational classification model based on Vector Space Model and Bag of Words has been developed. The model was first applied to the positive-negative categorization of the evaluations, then, applied to the classification of negative evaluations. It was determined that the accuracy level of the model is higher than the classification algorithms commonly used in short text. According to the results of the research, 9637 negative evaluations in 24479 consumer opinion were determined, and 50.4% of the negative evaluations were found to have the most problems related to product.

Methods for bilingual lexicon induction are often based on word embeddings (WE) similarity. These methods must be able to project the WE to the same space. Uncontextualized WE proved to be useful for this task. We compare them to contextualized WE and Bag of Words, using specialized and general datasets. We also evaluate the impact of seed lexicons and check the existing reference lists validity, claiming that extracting the translation of some words in those lists is not useful and confirming the need to have more fine-grained reference lists.

Writers tend to follow a certain style that can be detected or at least sketched by an appropriate algorithm. Columnists in newspapers, being also writers, follow their specific style. The style tends to be stable once writers reach maturity, but it is subject to change when internal or external circumstances differ. Here, we apply a bag-of-words approach to approximate the style of several journalists working in Mexican newspapers, and we track their style for a long period of time with the aim of detecting changes when external circumstances, in particular political ones, change. This provided us with an environment for detecting variations in stylomics, which is the closest we can get to an experiment. In particular, we collected hundreds of writings of ten Mexican columnists from different newspapers, both previous to the Presidential Mexican elections of 2018 and posterior to it. We processed these documents on different supervised and not supervised learning algorithms, such as random forest, principal component analysis, and k-means. Likewise, we implemented different validation procedures. As a result, we detected that the style in all studied columnists suffered tangible changes in the frequency of use of some particular words, particularly at specific times, some of which may be related to the 2018 Mexican presidential elections.

Categorization of text documents has become a prime task with plenty of applications. It is a process that assigns the text documents to their individual categories based on the contents. In this paper, a long-short-term memory (LSTM) recurrent neural network (RNN) based system along with handcrafted graph-based features is proposed for text categorization. In comparison to the traditional bag of words model for representation of documents, this graph-based model represents each document by a weighted graph for determining the relationships among the tokens. The significance of a token to a text document is determined by the graph-based feature. Experiments were tested on 14,373 Bangla text documents (around 57,22,569 tokens) and a maximum accuracy of 99.21% has been obtained.

We explore the task of sentiment analysis on Hinglish (code-mixed Hindi-English) tweets as participants of Task 9 of the SemEval-2020 competition, known as the SentiMix task. We had two main approaches: 1) applying transfer learning by fine-tuning pre-trained BERT models and 2) training feedforward neural networks on bag-of-words representations. During the evaluation phase of the competition, we obtained an F-score of 71.3% with our best model, which placed $4^{th}$ out of 62 entries in the official system rankings.

Traditionally, structured or coded data fields from a crash report are the basis for identifying crashes involving different types of vehicles, such as farm equipment. However, using only the structured data can lead to misclassification of vehicle or crash type. The objective of the current article is to examine the use of machine learning methods for identifying agricultural crashes based on the crash narrative and to transfer the application of models to different settings (e.g., future years of data, other states). Different data representations (e.g., bag-of-words [BoW], bag-of-keywords [BoK]) and document classification algorithms (e.g., support vector machine [SVM], multinomial naïve Bayes classifier [MNB]) were explored using Texas and Louisiana crash narratives across different time periods. The BoK-support vector classifier (SVC), BoK-MNB, and BoW-SVC models trained with Texas data were better predictive models than the baseline rule-based algorithm on the future year test data, with F1 scores of 0.88, 0.89, 0.85 vs. 0.84. The BoK-MNB trained with Louisiana data performed the closest to the baseline rule-based algorithm on the future year test data (F1 scores, 0.91 baseline rule-based algorithm vs. 0.89 BoK-MNB). The BoK-SVC and BoK-MNB models trained with Texas and Louisiana data were better productive models for Texas future year test data with F1 scores 0.89 and 0.90 vs. 0.84. The BoK-MNB model trained with both states’ data was a better predictive model for the Louisiana future year test data, F1 score 0.94 vs. 0.91. The findings of this study support that machine learning methodologies can potentially reduce the amount of human power required to develop key word lists and manually review narratives.

Human action recognition has gathered significant attention in recent years due to its high demand in various application domains. In this work, we propose a novel codebook generation and hybrid encoding scheme for classification of action videos. The proposed scheme develops a discriminative codebook and a hybrid feature vector by encoding the features extracted from CNNs (convolutional neural networks). We explore different CNN architectures for extracting spatio-temporal features. We employ an agglomerative clustering approach for codebook generation, which intends to combine the advantages of global and class-specific codebooks. We propose a Residual Vector of Locally Aggregated Descriptors (R-VLAD) and fuse it with locality-based coding to form a hybrid feature vector. It provides a compact representation along with high order statistics. We evaluated our work on two publicly available standard benchmark datasets HMDB-51 and UCF-101. The proposed method achieves 72.6% and 96.2% on HMDB51 and UCF101, respectively. We conclude that the proposed scheme is able to boost recognition accuracy for human action recognition.

With the exponential growth in the enhancement of modes of information exchange, the spread of text has become not only substantially faster, but also widespread. Due to this, text has become an indispensable part of all kinds of decision-making. Hence, it has become imperative to analyse the methods that can help make sense of this text as efficiently as possible. We shall make an attempt at the same by discussing various tools to make this very task increasingly productive. We shall try to analyse the relationship between the way an algorithm works and how it performs on various sets of data having different types of featurization. We shall analyse featurization techniques such as bag of words/N-grams, Tf-Idf vectorization, average Word2Vec and Tf-Idf Word2Vec.

In this work, we investigate several machine learning methods to tackle the problem of intent classification for dialogue utterances. We start with bag-of-words in combination with Naive Bayes. After that, we employ continuous bag-of-words coupled with support vector machines (SVM). Then, we follow long short-term memory (LSTM) networks, which are made bidirectional. The best performing model is hierarchical, such that it can take advantage of the natural taxonomy within classes. The main experiments are a comparison between these methods on an open sourced academic dataset. In the first experiment, we consider the full dataset. We also consider the given subsets of data separately, in order to compare our results with state-of-the-art vendor solutions. In general we find that the SVM models outperform the LSTM models. The former models achieve the highest macro-F1 for the full dataset, and in most of the individual datasets. We also found out that the incorporation of the hierarchical structure in the intents improves the performance.

Texture information allows characterizing the regions of interest in a scene. It refers to the spatial organization of the fundamental microstructures in natural images. Texture extraction has been a challenging problem in the field of image processing for decades. In this paper, different techniques based on the classic Bag of Words (BoW) approach for solving the texture extraction problem in the case of hyperspectral images of the Earth surface are proposed. In all cases the texture extraction is performed inside regions of the scene called superpixels and the algorithms profit from the information available in all the bands of the image. The main contribution is the use of superpixel segmentation to obtain irregular patches from the images prior to texture extraction. Texture descriptors are extracted from each superpixel. Three schemes for texture extraction are proposed: codebook-based, descriptor-based, and spectral-enhanced descriptor-based. The first one is based on a codebook generator algorithm, while the other two include additional stages of keypoint detection and description. The evaluation is performed by analyzing the results of a supervised classification using Support Vector Machines (SVM), Random Forest (RF), and Extreme Learning Machines (ELM) after the texture extraction. The results show that the extraction of textures inside superpixels increases the accuracy of the obtained classification map. The proposed techniques are analyzed over different multi and hyperspectral datasets focusing on vegetation species identification. The best classification results for each image in terms of Overall Accuracy (OA) range from 81.07% to 93.77% for images taken at a river area in Galicia (Spain), and from 79.63% to 95.79% for a vast rural region in China with reasonable computation times This work was supported in part by the Civil Program UAVs Initiative, promoted by the Xunta de Galicia and developed in partnership with the Babcock Company to promote the use of unmanned technologies in civil services. We also have to acknowledge the support by Ministerio de Ciencia e Innovación, Government of Spain (grant number PID2019-104834GB-I00), and Consellería de Educación, Universidade e Formación Profesional (ED431C 2018/19, and accreditation 2019-2022 ED431G-2019/04). All are cofunded by the European Regional Development Fund (ERDF) SI

With the increasing use of natural language processing, how to build a word vector which contains more semantic information becomes a top priority. Word vector is used to represent the most basic unit in the language-word, and is the basis of the neural natural language processing model. Therefore the quality of word vectors directly affects the performance of various applications. In continuous bag-of-words model, limited by the frequency of occurrence, some words do not get enough training. At the same time, based on the set of minimum frequency, some low-frequency words are ignored by the model. In this paper, we build similar clusters from the semantic dictionary and integrate it into CBOW model with the help of multi-classifier. We improve word vectors and use it to complete a semantic similarity comparison task. Compared with the original word vectors built by CBOW, the method we proposed got higher accuracy. It shows some semantic information are integrated and the word vectors of low–frequency words are improved.

Most successful approaches on image classification apply the Bag-of-Words (BoW) approach in the context of category-level image classification. To incorporate spatial image information in the BoW model, Spatial Pyramids (SPs) are used. However, spatial pyramids are rigid in nature and are based on predefined grid configurations. As a consequence, they often fail to coincide with the underlying spatial structure of images from different categories which may negatively affect the classification accuracy.

For successful real-time action recognition in videos, the compromise between computational cost and accuracy must be carefully considered. To explore this balance, we focus on the popular Bag-of-Words (BoW) framework. Although computationally efficient, the BoW has weak classification power. Thus, many variants have been developed. These variants aim to increase classification power whilst maintaining computational efficiency; achieving the ideal cost-to-accuracy balance. Four factors affecting the computational cost vs accuracy balance were identified: ‘Sampling’ strategy, ‘Optical Flow’ algorithm, ‘Saliency’ of extracted features and overall algorithm ‘Flexibility’. The practical effects of these factors were experimentally evaluated using the Dense Trajectories feature framework and the KTH and HMDB51 (a reduced version) datasets. The ‘Saliency’ of extracted information is found to be the most vital factor - spending computational resources to process large amounts of non-salient information can decrease accuracy.

As for the topic representation in standard topic models, the words that appear in a document are considered with the same weight under the assumption of ‘bag of words’. The word-topic assignment will lean to the high-frequency words and ignore the influence of the low-frequency words. As a result, it will ultimately impact on the performance of topic representation. Generally, the statistical information obtained from the whole document collection can be used to improve this situation. In addition, headlines of some kind of documents, such as news articles, usually summarize the important elements in the document, and the words in headlines are more appropriate to represent the topics. However, few previous studies consider the headline rich information, which is significant for topic modeling. In this paper, we propose a new headline-based topic model in order to accomplish a well-formed topic description. Experimental results on three widely used datasets show that the proposed headline-based modeling scheme achieves lower perplexity.

In the present era, most of the data on the internet is in the form of raw text. These gold mines of data are invaluable since it contains lots of underlying information which can be extracted using natural language processing or text analytics techniques. The data from these text-based documents disclose users’ sentiments and opinions about a particular subject. In this paper, customer reviews from Amazon.com are pre-processed, analyzed using our proposed framework, and how these textual reviews justify the star ratings is studied. Features derived from textual reviews are used to predict its corresponding star ratings. To accomplish it, the prediction problem is transformed into a multi-class classification task to classify reviews to one of the five classes corresponding to its star rating. The performances of various classifiers used are evaluated and compared. Evaluation results on ground-truth data set show that Logistic Regression Classifier outperformed other models. Our study also reveals that among the various factors, polarity of the review and length of the review showed a significant effect on its rating.

Word embeddings are a useful tool for extracting knowledge from the free-form text contained in electronic health records, but it has become commonplace to train such word embeddings on data that do not accurately reflect how language is used in a healthcare context. We use prediction of medical codes as an example application to compare the accuracy of word embeddings trained on health corpora to those trained on more general collections of text. It is shown that both an increase in embedding dimensionality and an increase in the volume of health-related training data improves prediction accuracy. We also present a comparison to the traditional bag-of-words feature representation, demonstrating that in many cases, this conceptually simple method for representing text results in superior accuracy to that of word embeddings.

We develop and test a model of high quality annual report discourse. The model is trained and evaluated on reports published between 2007 and 2018 by London Stock Exchange-listed firms shortlisted for an award by corporate reporting experts. We use methods from computational linguistics to identify an initial set of 19 features that distinguish quality according to what management say (i.e.: content) and how they say it (i.e.: language structure). We supplement these features with popular bag-of words proxies drawn from extant research (document length, reading ease, net tone, forward-looking content, and uncertainty). Stepwise regression yields a parsimonious quality model comprising 10 features that suggest more strategy-related commentary, less focus on growth, and greater language accessibility that promotes cognitive processing (evidenced by more relevancy markers, greater connectivity, more exclusive forms of language, and fewer grammatical words). The model predicts over 70% of shortlisting cases in out-of-sample tests and outperforms a baseline model comprising popular bag-of-words features.

As the first step in automated natural language processing, representing words and sentences is of central importance and has attracted significant research attention. Despite the successful results that have been achieved in the recent distributional dense and sparse vector representations, such vectors face nontrivial challenge in both memory and computational requirement in practical applications. In this paper, we designed a novel representation model that projects dense vectors into a higher dimensional space and favors a highly sparse and binary representation of vectors, while trying to maintain pairwise inner products between original vectors as much as possible. Our model can be relaxed as a symmetric non-negative matrix factorization problem which admits a fast yet effective solution. In a series of empirical evaluations, the proposed model reported consistent improvement in both accuracy and running speed in downstream applications and exhibited high potential in practical applications.

Indian movie reviews, sentiments play an important role in knowing the feedback of a movie comment that will identify whether the movie is good, bad or average. This research paper collects dataset of online movie review comments from a popular movie booking website BookMyShow for purely research purpose where the dataset of the movie reviews are based on the five genres such as: Animation, Comedy, Action, Romance, and Epics for a varied age group of Kids, Teens, Young adults, Adults and Family. This research study analyses live movie review comments of five movies to classify the sentiment of each movie review comment, emoticons and rating stars to calculate an overall polarity percentage. In our experimental approach, we have taken the unstructured data of movie reviews from a popular booking website BookMyShow using a Python script called Grabber.py which is later converted into structured form with a defined bag of words. Further, it goes for data preprocessing by extracting the corpus using syntactic feature selection method and using phrase patterns for positive and negative sentiment orientation, a supervised learning approach is applied on the movie comments for text processing and emoticons with SVM classification and Naive Bayes classifier for classifying the polarity of the movie review comments.

There are different methods or techniques used for identifying violence from video, such as hitting some object, kicking, fighting, and punching someone but still there is a big challenge for us to identify violence. However, some of the earlier mechanism generally extract descriptors around the spatiotemporal interesting points (STIP) or extract statistic features but there is limited effectiveness in detecting video-based violence. Therefore, the objective is to develop a better violence identification system that identifies the violence and triggers an alarm so that prompt assistance will be provided. This paper helps researchers who wish to study violent activity recognition and gather different insights on the main challenges and issues to solve in this emerging field.

After many sentiment analysis as well as many types of methods classify the reviews that is based on test data and reviewer’s ratings which uses training., after reading reviews it is seen that star rating of reviewer do not always give a precise measure of his sentiment. This paper primarily focuses on analyzing customer reviews from the e-commerce space. Upon surveying popular e-commerce websites it can be observed that in several instances the product rating given by a customer is not consistent with the product review written by him/her. The problem is made complex by the fact that there is no standard scale to measure the rating that the user gives and the rating of the product are instinctive to the customers’ view. In several cases it is seen that a product is rated 4 out of 5. However, the reviews detail that the customer’s experience with the product is not favourable. Indeed, text reviews are a true picture of the product. To get rid of this problem, the stated system will give a boolean result i.e. whether the product is good or bad and the user does not need to read all the reviews to analyze the product.

Sentiment analysis is used to derive the emotion/opinion that is being conveyed in a text. This helps in determining whether the author’s intent is positive or negative. Its applications are vast and help in analyzing product reviews, popularity of a brand, and in determining people’s opinions on any subject. Due to the complexities involved in the human language such as subjectivity, metaphors, sarcasm, and multiple sentiments, it becomes difficult to categorize our opinions, computationally. The goal of this project is to conduct sentiment analysis on Amazon product reviews using various natural language processing (NLP) techniques and classification algorithms. The dataset consists of 400,000 reviews of unlocked mobile phones sold on Amazon.com. We will achieve the result by preprocessing the reviews and converting them to clean reviews, after which using word embedding, the word reviews were converted into numerical representations. Then, we finally fit the numerical representations of reviews to the Naive Bayes, logistic regression, and random forest algorithm. The results and accuracy of all these classifiers are compared in this paper. This will be helpful for a brand/company to understand the general opinion toward their product which in turn will help them in evaluating the improvements required.

With the growing information on web, online movie review is becoming a significant information resource for Internet users. However, online users post thousands of movie reviews on daily basis and it is hard for them to manually summarize the reviews. Movie review mining and summarization is one of the challenging tasks in natural language processing. Therefore, an automatic approach is desirable to summarize the lengthy movie reviews, and it will allow users to quickly recognize the positive and negative aspects of a movie. This study employs a feature extraction technique called bag of words (BoW) to extract features from movie reviews and represent the reviews as a vector space model or feature vector. The next phase uses Naïve Bayes machine learning algorithm to classify the movie reviews (represented as feature vector) into positive and negative. Next, an undirected weighted graph is constructed from the pairwise semantic similarities between classified review sentences in such a way that the graph nodes represent review sentences, while the edges of graph indicate semantic similarity weight. The weighted graph-based ranking algorithm (WGRA) is applied to compute the rank score for each review sentence in the graph. Finally, the top ranked sentences (graph nodes) are chosen based on highest rank scores to produce the extractive summary. Experimental results reveal that the proposed approach is superior to other state-of-the-art approaches.

Youngsters during their adolescent experience various changes in their life. Some youngsters could not handle pressure created due to these changes and tend to feel isolated and distressed. Social media is a popular medium of communication amongst youngsters to remain connected with their friends. Facebook is one of the most preferred social media sites which stores the gigantic amount of data which can be explored for sentiment analysis. We have studied Facebook status updates in English and Hindi to determine the degree of emotional distress using bag-of-words model enhanced with weighted affect words, negation, discourse relation, emoticons, and punctuation marks. SVM classifier has given a promising result with an accuracy of 90.49%.

The recognition and prediction of human interaction based on video has a broad application prospect in intelligent video monitoring and other fields, but the current integration algorithm of it is not mature, which greatly limits the application of the algorithm. A prediction method of human interaction based on the statistical characteristics of interest points is proposed, which established an integrated framework to the recognition and prediction of human interaction initially. First, the spatio-temporal interest points are extracted and performed as 3D-SIFT feature description, then the bag of words is used to represent the action video. In the training stage, Gaussian models are used to establish the action model for each action at different time scales. In the prediction stage, the bag of words representation is extracted and compared with the established prediction models of different time lengths to obtain similar prediction probabilities between the models for an action video of unknown length. Finally, the predictive probability of each action at different time scales is fused by weighted probabilities completing the recognition and prediction of interaction prediction. The experimental result on UT-interaction dataset demonstrated that the proposed approach is easy to implement and has a good predictive effect.

SOM-based visualization is a promising approach for revealing potential technical solutions varied in patent documents. In this paper, we try to improve the quality of visual representation by constructing Fuzzy Bag-of-Words (FBoW) matrices through utilization of multi-view information. F-term is the special theme code given by the examiners of Japan Patent Office (JPO) for Japanese patent documents and is expected to be a potential candidate of second-view information. Document \(\times \) F-term co-occurrence information is utilized for improving the quality of FBoW representation such that semantical similarities among words are measured by considering F-term semantics. The advantage of utilizing F-term semantics in constructing FBoW is demonstrated through analysis of patent document data.

From the past decade, there has been drastic development and deployment of digital data warehoused in electronic health record (EHR). Initially, it is intended for getting patient general info and accomplishment healthcare tasks like billing, but researchers focused on secondary and most important use of these data for innumerable clinical solicitations. In this paper, we addressed the use of deep learning-based clinical note multi-label multi-class approach using ensemble approach based on CNN and bag-of-words approach. And we map those classes for multi-classes. And we perform experiments with Python, and we used libraries of Keras, TensorFlow, NumPy, matplotlib, and we use MIMIC-III data set. And we made comparison with existing works CNN, skip-gram, n-gram and bag of words. The performance results show that proposed framework performed good while classifying the text notes.

Social media has extensive content volunteered by drug users, who also mentions drug side-effects. Such mentions may indicate side-effects that are heretofore unknown to the medical professionals and researchers, pharmaceutical companies, and regulators. To study and monitor the drug and its potential symptoms and side-effects for epidemiological surveillance of drug trends, the research needed a higher level of knowledge extraction and improvement in domain knowledge. Our aim is to methodically classify the tweets that mention drug side-effects, and by applying our model we improved the medical knowledge extraction from our data-set. We crawled 7 million tweets related to marijuana concentrate and cannabis (known as marijuana) the tweet text mentioned about marijuana usage, marijuana consumption, the medicinal value of marijuana and symptoms or side-effects of marijuana. It is essential to identify the drug side-effects in the tweets in order to create a bag of words relating to each category of marijuana. This will assist us in the classification of tweets under different categories of marijuana-based on side-effects. We manually created multi-domain knowledge, modeled in DsOn(Drug and Symptom Otology), to facilitate the extraction of semantic parsing from people opinion on Social Media, through a combination of medical taxonomy dictionary, and semantics relationship based techniques from UMLS, PubMed, and DBpedia. Experiment results showed an improvement in recall from 0.14 (without ontology model) to 0.23 (with ontology model).

Short message service (SMS) is considered as one of the most popular means of communication, it allows to the mobile phone users to exchange a short text message with a low cost. Its growing popularity and its dependence on mobile phone has increased the number of attacks, caused by sending an unsolicited message like SMS spam. In this paper, we address a comparative study, between multilayer perceptron (MLP), support vector machine (SVM), random forest and k-nearest neighbors (KNN). For extracting the feature vectors the bag-of-words (BOW) and the TF-IDF methods are applied. These feature vectors are used as input for training and testing the different machine-learning classifiers mentioned above. The results of different machine-learning classifiers, based on their accuracy, precision, recall, F-measure, and ROC (receiver operating characteristic) curve have shown that the MLP outperforms SVM, random forest and KNN in SMS spam detection. Although the MLP has achieved the highest accuracy by using the BOW, than by using the TF-IDF method.

In the field of process mining, one of the challenges of the trace representation problem is to exploit a lot of potentially useful information within the traces while keeping a low dimension of the corresponding vector space. Motivated by the initial results of applying the deep neural networks for producing trace representation, in this paper, we continue to study and apply two more advanced models of deep learning, i.e., Continuous Bag of Words and Long short-term memory, for generating the trace representation. The experimental results have achieved significant improvement, i.e., not only showing the close relationship between the activities in a trace but also helping to reduce the dimension of trace representation.

Authorship Attribution (AA) also known as Authorship Identification is the problem of identifying the author of an anonymous text based on its characteristics or features. Among notable features extraction methods used to this end, one can cite, the bag of words methods (BOW) and the semantic and syntactic methods (SSM). BOW methods consider the text as a sequence of tokens and disregard the semantics of the language, whereas SSM rely on advanced natural language processing (NLP) techniques. The features extracted from an anonymous text are compared to features extracted from a corpus of texts written by known authors using several similarity measures. In this paper, we combine multiple results generated using conventional methods (chosen from the literature) and we use a genetic algorithm (GA) to find the optimal weighting distribution. The optimal combination obtained by the GA is then applied, and the author attributed to the anonymous text is selected among a set of known authors based on the highest similarity. The fitness of our GA is the resulting accuracy of the authorship attribution task. A numerical application on a corpus consisting of 3036 books written by 142 authors shows that the proposed method has higher accuracy than conventional methods and achieved satisfying performance.

In this paper, an efficient approach for text-independent writer identification using bag of words model and the combination of multiple classifiers is proposed. First of all, a bag of words model is established by extracting sub-images from the original handwriting image. Then, features are extracted by moment method, direction index histogram method and simplified Wigner method respectively to calculate the distance between the sub images having the same labels. Finally, the handwriting classification task is completed by means of feature fusion and multi-classifier combination. To evaluate this approach, writer identification is conducted on IAM English database. Experimental results revealed that the proposed writer identification algorithm with small number of characters and unconstrained contents achieves interesting results as compared to those reported by the existing writer recognition systems.

We present a new approach and set of dictionaries for the analysis of sentiments in financial and business news headlines at the firm level. The proposed Polarity Pattern Approach (PPA) captures the deep structure of context and syntax and reduces measurement errors evident in previous approaches for classifying the tone of words and sentences in news headlines. We demonstrate the superior predictive validity of PPA using an out-of-sample dataset. As expected, negative news headlines are negatively related to stock returns and positively related to trading volumes, while positive news headlines are positively related to both, especially for finance news. Prior studies report a non-significant or, counter intuitively, negative relationship between positive news and stock returns, which we attribute to measurement error in previous approaches. Additionally, using market activities, we jointly estimate cross-sectional regression to compare the explanatory power of the PPA approach with commonly used methods. We show how PPA’s tone assignment significantly outperforms other dictionaries and approaches, including the commercial package (RavenPack) that uses both machine learning and deep learning. The PPA, including dictionaries, provides finance researchers with a new, more valid method of sentiment analysis for media news. Link to PPA tone assignment dataset: https://www.whensentimentisnews.com

This chapter introduces the representation of texts as elements of feature spaces, as well as various exploratory tools to study such representations. It investigates how students of humanities can discover groups of topically similar texts in a large textual collection and how recurring themes giving rise to similarity can be detected. Concepts including feature space, bag of words, cosine similarity, document collection, document vector, and document–term matrix are explained through a number of simplified examples; key engineering procedures (feature selection, feature scoring) in the construction of feature spaces are also introduced. The more complex application example of the study of the Anglo-Saxon Chronicle demonstrates how to detect similar records and recurring themes using explorative methods such as dimensionality reduction, clustering and topic modelling. The chapter concludes by pointing out the limitation of feature space representation and by defining what topical similarity means in the context of language technology.

Recommender systems aim to effectively recommend items to the user based on their profile. An online recruitment system recommends jobs for a candidate according to his profile and can also act in reverse, recommending more qualified candidates for a particular job. Defining which variables will be used impacts directly the recommendation quality so that, when using the most important variables, we have a better assertiveness in the process. The goal of this work is to select the most important features of an online recruitment database using feature selection techniques. More specifically, we used the algorithms of Mitra, SUD and ACA to perform feature selection. The datasets used were derived from the original dataset assuming three distinct scenarios: the dataset containing the attributes related with the jobs’ features; the dataset containing the bag of words of the description feature of the jobs; and the dataset resulting from the union of the two previous ones. The features’ subsets selected in each of the above scenarios had their performance evaluated in a clustering task. The results obtained in each scenario show a performance gain of the clustering process when feature selection is made over the original data. Also, it was observed that the jobs’ features result in better performance than the other two cases.

The traditional feature description methods of human interaction action based on RGB video are greatly affected by illumination change, object occlusion and environmental change. In this paper, depth and color information of the image are fused in the processing of feature extraction, and a novel human interaction recognition algorithm based on interest point of depth information fusion is proposed. Firstly, the spatio-temporal interest points (STIP) are extracted from the video, and the spatio-temporal interest points are processed hierarchically by the corresponding depth information. Secondly, three-dimensional scale invariant feature transform (3D SIFT) are used to obtain the feature description of hierarchical interest points, the model of the bag of words is constructed by using K-means clustering and Gaussian mixture clustering, and the representation of the training video is obtained based on dictionary projection. Finally, the support vector machine is used to classify different layers of video features respectively, then human interaction recognition is achieved by decision-level fusion of recognition probability for different layers of a video. The experimental results show that the accuracy of the human interaction recognition is improved by combing the depth information, and the proposed algorithm achieves 88.75% recognition accuracy in the SBU Kinect interaction dataset. The validity of the proposed method is proved.

With the advent of deep learning paradigms, many new algorithms have been flourishing which can be leveraged for the task of sentiment analysis. In this paper, we present an experimental comparison of traditional classification algorithms—Naive Bayes, Logistic Regression, Random Forest using bag of words model along with modern deep learning algorithms like Recurrent Neural Networks with Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) on reviews and tweets using three different datasets to classify a binary output, based on positive or negative polarity of the sentiment. We compare the results to find which model provides the best accuracy and analyze reason behind the differences in accuracies. The goal is to explore natural language processing (NLP) models trained on a certain dataset and then benchmark their respective performances, thereby ultimately demonstrating that deep learning algorithms prove to be more effective. We also state how the deep learning models outperform the traditional models and analyze them in detail.

This work is dedicated to Bangla Crime Type Classification. As very few works had been done for Bangla crime classifier. To carry out this research, first we have developed a Bangla crime dataset which contains around 24,295 news articles and made most of them publicly available at github. Then we have built our crime classifier model and trained the classifier with our own dataset. We have analyzed word vectors like bag of words, TF-IDF in state-of-art machine learning algorithms as well as most promising semantic and syntactic word embeddings like Word2Vec, GloVe, fast-Text in both shallow and deep CNN and RNN to select best word embeddings for our classifier module. Finally we have summarized the experimental result in tabular form. We can see that significant improved accuracy can be achieved using deep learning algorithms over state-of-art machine learning algorithms in classifying Bangla crime data. The final experimental result shows that using shallow CNN with fastText,proposed model is able to achieve 93.70% accuracy.

In this article we examine gender differences in the spoken usage using a selection of files from the British National Corpus (BNC). Our aim is twofold. First, to report on some similarities and differences in the words and phrases used by men and women in conversation. Secondly, we address some methodological issues related to the study of gender and to corpus linguistics research in general. In particular, we aim to address what we call the “bag of words, bag of people’ problem. In many studies a corpus is treated as a bag of words in common techniques such as a keyword analysis. Such frequency-based analyses have led to many discoveries about the nature of language, but the backgrounding of discourse and text structure is problematic in obscuring some patterns of language usage. In addition, corpora are necessarily compiled using the language output of many individuals---a bag of people---and the individual contributions, and hence variation in usage, are often overlooked. These issues are explored with reference to some linguistic elements known to potentially sensitive to gender variation.

With an increasing trend of publishing news online on website, automatic text processing becomes more and more important. Automatic text classification has been a focus of many researchers in different languages for decades. There is a huge amount of research repository on features of English language and their uses on automated text processing. This research implements Nepali language key features for automatic text classification of Nepali news. In particular, the study on impact of Nepali language based features, which are extremely different than English language is more challenging because of the higher level of complexity to be resolved. The research experiment using vector space model, n-gram model and key feature based processing specific to Nepali language shows promising result compared to bag-of-words model for the task of automated Nepali news classification.

we present a dynamic Web document Classification using semantic knowledge (DBpedia). We present a method for a dynamic Web document Classification and automatic classification. The proposed approach required only a domain ontology and a set of user predefined categories. Currently, most approaches to text classification represent document as (bag of words) and training the large set of documents to train the classifier. Our approach doesn't require a training set of documents. In our proposed method, we use DBpedia ontology as the main classifier, representing documents as (bag of concepts). We extract the terms from the document, extract their resources from DBpedia Spotlight, use Sparqle query to determine class ontology and map them to their concepts then we determine the best category.

Word embedding models are able to accurately model the semantic content of words. The process of extracting a set of word embedding vectors from a text document is similar to the feature extraction step of the Bag-of-Features (BoF) model, which is usually used in computer vision tasks. This gives rise to the proposed Bag-of-Embedded Words (BoEW) model that can efficiently represent text documents overcoming the limitations of previously predominantly used techniques, such as the textual Bag-of-Words model. The proposed method extends the regular BoF model by a) incorporating a weighting mask that allows for altering the importance of each learned codeword and b) by optimizing the model end-to-end (from the word embeddings to the weighting mask). Furthermore, the BoEW model also provides a fast way to fine-tune the learned representation towards the information need of the user using relevance feedback techniques. Finally, a novel spherical entropy objective function is proposed to optimize the learned representation for retrieval using the cosine similarity metric.

This chapter reviews some of the policy measures taken by the Government of Bangladesh (GoB) to generate employment opportunities for youth. The National Youth Policy (NYP) 2017 is reviewed, and its text is analysed using a simple bag of words model. Other government documents such as the 7th Five Year Plan (7FYP) of Bangladesh are also reviewed with a focus on youth. Some brief discussion on budget allocation for youth is included.