Search

The topic model has been widely applied to various research domains such as information retrieval, data mining, and so on. It can discover topics of texts in an unsupervised way. In the early years, most researches mainly focused on long texts. With the emergence of the Internet, the number of short texts is growing rapidly. Most existing schemes to solve the sparsity problems of short texts, are mainly based on data aggregation or model improvements. Among them, the Biterm Topic Model is one of the most representative models. It proposed a new way to model topics based on document-level word pairs and has shown creativity and effectiveness. However, this strategy ignores those semantically similar and rarely co-occurrent word pairs. What’s more, most researches ignore the multi-sense phenomenon in natural languages. In this paper, we utilize multi-sense word vectors to extract similar word pairs from the whole corpus by considering multiple senses. Based on this idea, we introduce a novel short-text topic model, which disambiguates multiple senses of words and generates more reasonable global biterms. Experimental results on two open-source English datasets have shown superiority to state-of-the-art topic models.

The rapid growth of COVID-19 publications has driven clinical researchers and healthcare professionals in pursuit to reduce the knowledge gap on reliable information for effective pandemic solutions. The manual task of retrieving high-quality publications based on the evidence pyramid levels, however, presents a major bottleneck in researchers’ workflows. In this paper, we propose an “evidence-based” recommender system namely, KnowCOVID-19 that utilizes an edge computing service to integrate recommender modules for data analytics using end-user thin-clients. The edge computing service features chatbot-based web interface that handles a given COVID-19 publication dataset using two recommender system modules: (i) evidence-based filtering that observes domain specific topics across the literature and classifies the filtered information according to a clinical category, and (ii) social filtering that allows diverse experts with similar objectives to collaborate via a “social plane” to jointly find answers to critical clinical questions to fight the pandemic. We compare the Domain-specific Topic Model (DSTM) used in our evidence-based filtering with state-of-the-art models considering the CORD-19 dataset (a COVID-19 publication archive) and show improved generalization effectiveness as well as knowledge pattern query effectiveness. In addition, we conduct a comparison study between a manual literature review process and the KnowCOVID-19 augmented process, and evaluate the benefits of our information retrieval techniques over important queries provided by COVID-19 clinical experts.

In this paper, we propose a coherent framework for multi-machine analysis, using a group clustering model, which can be utilized for predictive maintenance (PdM). The framework benefits from the repetitive structure posed by multiple machines and enables for assessment of health condition, degradation modeling and comparison of machines. It is based on a hierarchical probabilistic model, denoted Gaussian topic model (GTM), where cluster patterns are shared over machines and therefore it allows one to directly obtain proportions of patterns over the machines. This is then used as a basis for cross comparison between machines where identified similarities and differences can lead to important insights about their degradation behaviors. The framework is based on aggregation of data over multiple streams by a predefined set of features extracted over a time window. Moreover, the framework contains a clustering schema which takes uncertainty of cluster assignments into account and where one can specify a desirable degree of reliability of the assignments. By using a multi-machine simulation example, we highlight how the framework can be utilized in order to obtain cluster patterns and inherent variations of such patterns over machines. Furthermore, a comparative study with the commonly used Gaussian mixture model (GMM) demonstrates that GTM is able to identify inherent patterns in the data while the GMM fails. Such result is a consequence of the group level being modeled by the GTM while being absent in the GMM. Hence, the GTM are trained with a view on the data that is not available to the GMM with the consequence that the GMM can miss important, possibly even key cluster patterns. Therefore, we argue that more advanced cluster models, like the GTM, can be key for interpreting and understanding degradation behavior across machines and ultimately for obtaining more efficient and reliable PdM systems. CC BY 4.0Corresponding author: Alexander Karlsson (alexander.karlsson@his.se)This work was supported by grant 2019-00789 at Vinnova, Project: Predictive Maintenance using Advanced Cluster Analysis (PACA). Predictive Maintenance using Advanced Cluster Analysis (PACA)

Topic modeling is a probabilistic graphical model for discovering latent topics in text corpora by using multinomial distributions of topics over words. Topic labeling is used to assign meaningful labels for the discovered topics. In this paper, we present a new topic labeling method that uses automatic term recognition to discover and assign relevant labels for each topic, i.e., TLATR (Topic Labeling using Automatic Term Recognition). TLATR uses domain-specific multi-terms that appear in the set of documents belonging to a topic. The multi-term having the highest score as determined by the automatic term recognition algorithm is chosen as the label for that topic. To evaluate TLATR, we use two real, publicly available datasets that contain scientific articles’ abstracts. The topic label evaluation is done both automatically and using human annotators. For the automatic evaluation, we use Pointwise Mutual Information, Normalized Pointwise Mutual Information, and document similarity. For human evaluation, we employ the average rating method. Furthermore, we also evaluate the quality of the topic models using the Adjusted Rand Index. To prove that our novel method extracts relevant topic labels, we compare TLATR with two state-of-the-art methods, one supervised and one unsupervised, provided by the NETL Automatic Topic Labelling system. The experimental results show that our method outperforms or provides similar results with both NETL’s supervised and unsupervised approaches.

In recent years, with the rapid growth of social media, short texts have been very prevalent on the internet. Due to the limited length of each short text, word co-occurrence information in this type of documents is sparse. Conventional topic models based on word co-occurrence are unable to distill coherent topics on short texts. A state-of-the-art strategy is self-aggregated topic models which implicitly aggregate short texts into latent long documents. But these models have two problems. One problem is that the number of long documents should be defined explicitly and the inappropriate number leads to poor performance. Another problem is that latent long documents may bring non-semantic word co-occurrence which brings incoherent topics. In this article, we firstly apply the Chinese restaurant process to automatically generate the number of long documents according to the scale of short texts. Then to exclude non-semantic word co-occurrence, we propose a novel probabilistic model generating latent long documents in a more semantically way. Specifically, our model employs a pitman-yor process to aggregate short texts into long documents. This stochastic process can guarantee that the distribution between short texts and long documents following a power-law distribution which can be found in social media like Twitter. Finally, we compared our method with several state-of-the-art methods on four real short texts corpus. The experiment results show that our model performs superior to other methods with the metrics of topic coherence and text classification.

The understanding and analyzing of available content on Social media Platforms such as Twitter and Facebook, through various topic modeling methods is not supervised. However, despite several existing conventional techniques, they have had limited success when applied directly for filtering and quick comprehension of short-text contents due to text sparseness and noise. Thus, it always has been challenging to discover reliable latent topics from online discussion texts that prevail with low words co-occurrence and availability of large size social media benchmark datasets, even for resource-rich languages. The existing literature lacks such work for Urdu text to unveil niche topics even with conventional topic models, mainly due to the lack of benchmark datasets, limited availability of pre-processing tools/ algorithms, and time and compute limitations on large-sized datasets. This work presents experiments with multiple approaches of topic modeling like Latent Semantic Analysis (LSA), Probabilistic Latent Semantic Analysis (PLSA), Latent Dirichlet Allocation (LDA), and Non-negative Matrix Factorization (NMF) on 0.8 million Urdu tweets. These tweets are collected through Twitter API by giving various hashtags as a query to avoid dominance of single topic in the dataset. In addition, we have pre-processed the text of the tweets, prepared the three variants of the collected dataset, and extracted multiple features to represent documents on different n-grams. Furthermore, all these techniques are compared and evaluated on the dataset variants, using both qualitative and quantitative measures. We have also demonstrated the results of these approaches through visualization methods, graphs depicting tweets size per topic, word clouds, and hashtags analysis, giving insights about algorithms performances on finalized topics. Observed results reveal that NMF outperformed the techniques with TF-IDF feature vectors in Urdu tweets text, while LDA performed best with merging short-text strategy into long pseudo documents.

Tracking social media sentiment on a desired target is certainly an important query for many decision-makers in fields like services, politics, entertainment, manufacturing, etc. As a result, there has been a lot of focus on Sentiment Analysis. Moreover, some studies took one step ahead by analyzing subjective texts further to understand possible motives behind extracted sentiments. Few other studies took several steps ahead by attempting to automatically interpret sentiment variations. Learning reasons from sentiment variations is indeed valuable, to either take necessary actions in a timely manner or learn lessons from archived data. However, machines are still immature to carry out the full Sentiment Variations’ Reasoning task perfectly due to various technical hurdles. This paper attempts to explore main approaches to Opinion Reason Mining, with focus on Interpreting Sentiment Variations. Our objectives are investigating various methods for solving the Sentiment Variations’ Reasoning problem and identifying some empirical research gaps. To identify these gaps, a real-life Twitter dataset is analyzed, and key hypothesis for interpreting public sentiment variations are examined.

Urban functional zones are considered significant components for understanding urban landscape patterns in the socioeconomic environment. Although the spatial configuration of road networks contributes to urban function delineation at the block level, the morphological uncertainties caused by the road network structure in fine-scale urban function retrieval are ignored. This paper proposes an adaptive network-constrained clustering (ANCC) model to map urban function distributions at a finer level. By utilizing points of interest (POIs) to indicate independent functional places, the adaptive road configuration with a multilevel bandwidth selection strategy is proposed. On this basis, a term frequency–inverse document frequency-weighted latent Dirichlet allocation (TW-LDA) topic model is designed to delineate urban functions from semantic information. Taking Futian District, Shenzhen, as a case study, the results show an overall accuracy of approximately 77.10% in urban function mapping. A comparison of a block-level mapping model, a non-adaptive network-based model and the ANCC model reveals accuracies of 53.10%, 59.20% and 77.10%, respectively, indicating the advantages of the ANCC model for improving urban function mapping accuracy. The proposed ANCC model shows potential application prospects in monitoring urban land use for sustainable city planning.

Service search in IoT’s large-scale, heterogeneous and multi-domain services space is a challenging task. It can take the time that may not be acceptable for many IoT applications and requires resources that may not be available in many IoT devices. A categorisation of these services into their application domains can reduce the search space and offer an efficient and scalable service search. Recently, in many fields, such as short text messages and categorisation of IoT service specifications, generative probabilistic models, like Topic modelling, are being used. Generally, IoT service descriptions are short and sparse. Existing work on IoT services categorisation is based on Latent Dirichlet Allocation (LDA), but it does not perform well in short and sparse texts. Also, IoT services categorisation has few specific issues, which are not well addressed by existing short texts-specific topic modelling approaches. In this paper, we identify these issues and quantitatively and qualitatively evaluate how well a set of selected short texts-specific topic modelling approaches perform as IoT service categorisers against these issues. The results show that these approaches do not perform well in a corpus of noisy APIs descriptions and heterogeneous service descriptions. Also, they do not support domain identification of services, which is essential in domain-based service search. We conclude that integrating an appropriate and comprehensive knowledge base (i.e., domain ontology) could minimise noise and address IoT’s APIs and service descriptions’ heterogeneity. More importantly, it can identify the domains of those APIs and services.

In microblog networks, when a user posts a microblog, other users may forward the post, and then the forwarding process will bring about the rapid dissemination and diffusion of information. In this paper, we propose a comprehensive and novel approach to predict user forwarding behavior. Firstly, we build the feature sets that affect the microblog forwarding, such as interest topic,geographic location, user aggregation coefficient,neighborhood overlap and so on. These features are classified into four categories: user characteristics, microblog features, network structure features, and interactive behavior characteristics. Secondly, we establish a feature selection model based on Filtering and Wrapping for predicting the forwarding behavior of users. The model includes three aspects: (1)ANOVA(Analysis of variance): The value of each feature is analyzed by variance analysis. If the feature variance is small, the feature provides less information. (2)$\chi ^{2}$ test and point-two-column correlation analysis: They filter discrete and continuous features, respectively. (3)Wrapper analysis: In order to solve the strong correlations between the features, we use LVW(Las vagas wrapper) algorithm to analyze the above feature sets, and then obtain the optimal feature combination. Finally, we propose the forwarding prediction model based on AdaBoost(Adaptive boosting) algorithm. Experimental results demonstrate that the model has the highest precision and F1 score than Naive Bayes, Logistic Regression, Random Forest and SVM(Support vector machine), and the F1 score reached 0.885. Among different topics, our proposed AdaBoost prediction model has good recall and F1 scores for different topics. In addition, by using different feature sets for comparison experiments, it is found that the optimal features selected in this paper are very effective.

An increasing number of people are choosing to shop online; hence, online reviews are an increasingly influential factor in consumer purchasing decisions. However, extracting useful information from online reviews is a challenge in the analysis of consumer sentiment. In this paper, we focus on the automatic discovery of the features evaluated in online reviews and the expression of sentiment. We propose a novel fine-grained topic model called the “review unit topic model” (RUTM) to extract semantic meanings and polarities. In this model, a review unit rather than a review sentence is treated as the representational model, and prior knowledge of sentiment is further exploited to identify aspect-aware sentiment polarities. We evaluate RUTM extensively using real-world review data. Experimental results demonstrate that the proposed model outperforms well-established baseline models in sentiment analysis tasks.

Topic Modelling has been successfully applied in many text mining applications such as natural language processing, information retrieval, information filtering, etc. In information filtering systems (IFs), user interest representation is the core part which determines the success of the system. Topics in a topic model generated from a user's documents can be used to represent the user's information interest. However, the quality of a topic model generated from a document collection is not always accurate because the topics of the topic model might contain meaningless or ambiguous words. This ambiguity problem can affect the performance of IFs which use a topic model to represent user information interest. Hence, a topic evaluation method to assess the quality of topics in a topic model is important for ensuring the effectiveness of utilizing the topic model in text mining applications. One method in measuring the quality of a topic model is to match the topical words of the model to concepts in an ontology. However, a limitation of this method is that some topical words in an examined topic cannot be found in the mapping ontology. In this study, we propose a new model to evaluate the quality of topics by matching concepts in an ontology. In particular, word embedding technique is applied to dealing with the ambiguity problem by finding similar concept words based on word embeddings. The assessed topics are then used in an information filtering system for filtering relevant documents for a user. The proposed model was evaluated against some state-of-the-art baseline models in terms of term-based, phrase-based, and topic-based user interest representations, and also some topic evaluation models. The result of the evaluation shows that the new proposed model outperforms the state-of-the-art baseline models.

Subnetwork identification plays a significant role in analyzing, managing, and comprehending the structure and functions in big networks. Numerous approaches have been proposed to solve the problem of subnetwork identification as well as community detection. Most of the methods focus on detecting communities by considering node attributes, edge information, or both. This study focuses on discovering subnetworks containing researchers with similar or related areas of interest or research topics. A topic-aware subnetwork identification is essential to discover potential researchers on particular research topics and provide quality work. Thus, we propose a topic-based optimal subnetwork identification approach (TOSNet). Based on some fundamental characteristics, this paper addresses the following problems: 1)How to discover topic-based subnetworks with a vigorous collaboration intensity? 2) How to rank the discovered subnetworks and single out one optimal subnetwork? We evaluate the performance of the proposed method against baseline methods by adopting the modularity measure, assess the accuracy based on the size of the identified subnetworks, and check the scalability for different sizes of benchmark networks. The experimental findings indicate that our approach shows excellent performance in identifying contextual subnetworks that maintain intensive collaboration amongst researchers for a particular research topic.

Automatic related work generation is a new challenge in multi-document scientific summarization focusing on refining a related work section for a given scientific paper. In this paper, we propose a brand new framework ToC-RWG for related work generation by incorporating topic model and citation information. We present an unsupervised generative probabilistic model, called QueryTopicSum, which utilizes a LDA-style model to characterize the generative process of both the scientific paper and its reference papers. We also take advantage of citations of reference papers to identify Cited Text Spans (CTS) from reference papers. This approach provides us with a perspective of annotating the importance of the reference papers from the academic community. With QueryTopicSum and the identified CTS as candidate sentences, an optimization framework based on minimizing KL divergence is exerted to select the most representative sentences for related work generation. Our evaluation results on a set of 50 scientific papers along with their corresponding reference papers show that ToC-RWG achieves a considerable improvement over generic multi-document summarization and scientific summarization baselines.

Detecting community structures is an important research topic in social network analysis. Unfortunately, the fundamental factors that drive the generation of social networks (i.e., the network topology and content) and community structures have not been well investigated. In this paper, according to the natural characteristics of social networks, we reveal that individual topics play a core role in community generation. If two individuals are in the same community and are interested in similar topics, it is more likely that a link will form between them. Otherwise, the probability of generating a link depends on the relationships between their communities and the topics they talk about. Based on the above observations, a novel generative community detection model is proposed that simulates the generation of the network topology and network content by considering individual topics. Moreover, our model utilizes a topic model to generate network content. The model is evaluated on two real-world datasets. The experimental results show that the community detection results outperform all the state-of-the-art baselines. In addition to accurate community detection results, we identify each individual topic distribution and the most popular users corresponding to different topics in each community.

Over the past two decades, a number of advances in topic modeling have produced sophisticated models that are capable of generating topic hierarchies. In particular, hierarchical Latent Dirichlet Allocation (hLDA) builds a topic tree based on the nested Chinese Restaurant Process (nCRP) or other sampling processes to generate a topic hierarchy that allows arbitrarily large branch structures and adaptive dataset growth. In addition, hierarchical topic models based on the latent tree model, such as Hierarchical Latent Tree Analysis (HLTA), have been developed over the last five years. However, these models do not work well in cases with millions of documents and hundreds of thousands of terms. In addition, the topic trees generated by these models are always poorly interpretable, and the relationships among topics in different levels are relatively simple. The biomedical literature, including Medline abstracts, has large-scale documents in two major categories: biological laboratory research and medical clinical research. We propose a top-down binary hierarchical topic model (biHTM) for biomedical literature by iteratively applying a flat topic model and adaptively processing subtrees of the hierarchy. The biHTM topic hierarchy of complete Medline abstracts with more than 14 topic node levels shows good bimodality and interpretability. Compared to hLDA and HLTA, biHTM shows promising results in experiments assessed in terms of runtime and quality.

With the prevalence of smart terminal devices and the rapid development of the mobile Internet, mobile application markets become increasingly prosperous. Third-party libraries have played an essential role in mobile application development. These libraries can shorten development time, increase development efficiency, and improve development quality. Currently, a large number of third-party libraries have been published, which puts a heavy burden on developers in selecting appropriate libraries. Towards this issue, in this paper, we propose a novel third-party library recommendation approach by integrating topic modeling and knowledge graph techniques. In the topic modeling component, we extract topics from textual application descriptions and make recommendations based on libraries used by applications that share similar topics with the new application to develop. In the knowledge graph component, we leverage knowledge graph to incorporate structured information of third-party libraries and applications, as well as the interaction information of applications and libraries for the recommendation. Experiments conducted on a real-world dataset show that our proposed approach outperforms several state-of-the-art approaches in terms of recommendation performance.

Recommender systems are efficient tools for online applications; these systems exploit historical user ratings on items to make recommendations of items to users. This paper aims to enhance dynamic collaborative filtering on recommender systems under volatile conditions in which both users’ preferences and item properties dynamically change over time. Moreover, existing collaborative filtering models mainly rely on solving data sparsity by adding side information to improve performance. We propose a model to capture the user preference dynamics in the rating matrix by using a joint decomposition method to extract user latent transition patterns and combine latent factors together with the associated topic evolution of review texts by using topic modeling based on the dynamic environment. We evaluate the accuracy on real datasets, and the experimental results show that the model leads to a significant improvement compared with the state-of-the-art dynamic CF models.

Topic modeling is a widely studied foundational and interesting problem in the text mining domains. Conventional topic models based on word co-occurrences infer the hidden semantic structure from a corpus of documents. However, due to the limited length of short text, data sparsity impedes the inference process of conventional topic models and causes unsatisfactory results on short texts. In fact, each short text usually contains a limited number of topics, and understanding semantic content of short text needs to the relevant background knowledge. Inspired by the observed information, we propose a regularized non-negative matrix factorization topic model for short texts, named TRNMF. The proposed model leverages pre-trained distributional vector representation of words to overcome the data sparsity problem of short texts. Meanwhile, the method employs the clustering mechanism under document-to-topic distributions during the topic inference by using Gibbs Sampling Dirichlet Multinomial Mixture model. TRNMF integrates successfully both word co-occurrence regularization and sentence similarity regularization into topic modeling for short texts. Through extensive experiments on constructed real-world short text corpus, experimental results show that TRNMF can achieve better results than the state-of-the-art methods in term of topic coherence measure and text classification task.

As an essential way to ensure success of construction projects, on-site inspection involves intensive paperwork, while generating large amounts of textual data. Lack of understanding of information hidden in text-based inspection records always leads to overlooking of important issues and deferred decisions. Therefore, a novel text mining approach based on keyword extraction and topic modeling is introduced to identify key concerns and their dynamics of on-site issues for better decision-making process. Then, the proposed approach was demonstrated in a real world project and tested with 7250 issue records. Results showed that the proposed method could successfully extract key concerns hidden in texts and identify their changes with time, thereby enabling a more efficient on-site inspection and data-centric decision-making process. This research contributes: (1) to the body of knowledge a new framework for discovering key concerns and their changes with time in texts, and (2) to the state of practice by providing insights on hot topics and their changes with time to reduce on-site issues and make decisions efficiently.

Using pre-trained topic information to assist in training neural networks can effectively support the completion of the rating prediction task. However, existing neural-topic methods consider only the use of topic information corresponding to current users and items without neighbors, whereas existing memory-based neighborhood approaches are inappropriate for the direct modeling of neighbors with topics. To address the limitations, we argue that memory networks have the ability to organize neighbors with corresponding topics well and can provide a general solution to this problem. To confirm our hypothesis, we propose two approaches. One is an augmented memory network to couple with and enhance existing neural-topic models. The other is a symmetric memory network activated by a memory reorganization mechanism, which is a compact and generalized method for rating prediction. The experimental results demonstrate the effectiveness of the memory reorganization mechanism and show that the two proposed methods have advantages over existing state-of-the-art topic modeling approaches.

Allocation of courses and research students based on faculty's subject specialization and area of interest has always remained a challenging task for university administration due to the presence of academics' cross-domain interests, stale faculty resumes at university portals and changing the skill set demands from the industry. Collaborative filtering and content-based recommender systems have already been in use by the industry for recommending things, such as movies, news, restaurants, and shopping items to the users, and however, no one has utilized these off-the-shelf models for enhancing the student experience and improving the quality of higher education in academia. This paper presents a case study showcasing the use of probabilistic topic models for generating recommendations to users in academia through appropriate course allocation and supervisor assignment. The proposed system coined as ScholarLite harnesses the power of machine learning to extract research themes from faculty members' past publications, mines research interests from their resumes, and combines it with their educational background to generate recommendations for course teaching, research supervision, and industry-academia collaboration. We have shown the recommendation results on real-world data gathered from the higher education commission of the country and demonstrated that the proposed techniques are scalable across various programs offered by the universities and could be deployed in a small budget by universities for automating course and supervisor allocation procedures. The experiments confirm our performance expectation by showing good relevance and objectivity in results, thus making this decision management system more appealing for large-scale deployment and use by academia.

Customer-agent conversations (i.e. call transcripts) are invaluable source for companies as they convey direct information from their customers implicit and explicit behaviour. Identifying customer-related events is an important task in customer services which is possible from the call transcripts. However, call centers produces a vast amount of transcripts which makes the manual or semi-manual processing of such raw datasets quite challenging. Furthermore, customer-agent call transcripts tend not to explicitly denote events that might be beneficial to customer services. Albeit being highly researched across multiple domains in the literature, event detection, especially implicit life event detection have not been well examined from call transcripts due to a lack of proper large-scale dataset. In this research, we propose a novel deep learning approach based on latent topic modeling and deep recurrent neural networks with memory units to automatically detect implicit events from a customer's history of call transcripts. These implicit events are detected prior to the report date of that event thereby not containing any explicit topic/feature. We provide a case study on a real-life, large-scale data of more than 800K call transcripts from a large financial services company in the U.S. to examine the practical features and challenges of this problem. The evaluation results demonstrate the potential applicability of our implicit life event detection as it achieves a macro-recall score of 53 (macro-f1 of 47.5) on a highly imbalanced test set, negative samples are 95% of the data. Our model beats the the state-of-the-art text classification benchmarks by macro-f1 score of 5.6 and macro-recall of 8.8 on average, and performs better than the ensemble of all single-document and sequential classification benchmarks albeit being significantly less complex. The comparison results show the importance as well as our model's capability of capturing the mutual information of a sequence of call transcripts in detecting the implicit life events.

Text data plays an imperative role in the biomedical domain. As patient’s data comprises of a huge amount of text documents in a non-standardized format. In order to obtain the relevant data, the text documents pose a lot of challenging issues for data processing. Topic modeling is one of the popular techniques for information retrieval based on themes from the biomedical documents. In topic modeling discovering the precise topics from the biomedical documents is a challenging task. Furthermore, in biomedical text documents, the redundancy puts a negative impact on the quality of text mining as well. Therefore, the rapid growth of unstructured documents entails machine learning techniques for topic modeling capable of discovering precise topics. In this paper, we proposed a topic modeling technique for text mining through hybrid inverse document frequency and machine learning fuzzy k-means clustering algorithm. The proposed technique ameliorates the redundancy issue and discovers precise topics from the biomedical text documents. The proposed technique generates local and global term frequencies through the bag-of-words (BOW) model. The global term weighting is calculated through the proposed hybrid inverse documents frequency and Local term weighting is computed with term frequency. The robust principal component analysis is used to remove the negative impact of higher dimensionality on the global term weights. Afterward, the classification and clustering for text mining are performed with a probability of topics in the documents. The classification is performed through discriminant analysis classifier whereas the clustering is done through the k-means clustering. The performance of clustering is evaluated with Calinsiki-Har-abasz (CH) index internal validation method. The proposed toping modeling technique is evaluated on six standard datasets namely Ohsumed, MuchMore Springer Corpus, GENIA corpus, Bioxtext, tweets and WSJ redundant corpus for experimentation. The proposed topic modeling technique exhibits high performance on classification and clustering in text mining compared to baseline topic models like FLSA, LDA, and LSA. Moreover, the execution time of the proposed topic modeling technique remains stable for different numbers of topics.

Topic models have been widely utilized in Topic Detection and Tracking tasks, which aim to detect, track, and describe topics from a stream of broadcast news reports. However, most existing topic models neglect semantic or syntactic information and lack readable topic descriptions. To exploit semantic and syntactic information, Language Models (LMs) have been applied in many supervised NLP tasks. However, there are still no extensions of LMs for unsupervised topic clustering. Moreover, it is difficult to employ general LMs (e.g., BERT) to produce readable topic summaries due to the mismatch between the pretraining method and the summarization task. In this paper, noticing the similarity between content and summary, first we propose a Language Model-based Topic Model (LMTM) for Topic Clustering by using an LM to generate a deep contextualized word representation. Then, a new method of training a Topic Summarization Model is introduced, where it is not only able to produce brief topic summaries but also used as an LM in LMTM for topic clustering. Empirical evaluations of two different datasets show that the proposed LMTM method achieves better performance over four baselines for JC, FMI, precision, recall and F1-score. Additionally, the generated readable and reasonable summaries also validate the rationality of our model components.

In order to improve the edge caching efficiency of the fog radio access network (F-RAN), this paper put forward a distributed deep Q-learning-based content caching scheme based on user preference prediction and content popularity prediction. Given that the constraint that the storage capacity of each device is limited, and the optimization problem is formulated so as to maximize the caching hit rate. Specifically, by taking users’ selfishness into consideration, user preference is predicted in an offline manner by applying popular topic models. Then, the online predicted content popularity is achieved by combining the network topology relationship together with the obtained user preference. Finally, with the predicted user preference and content popularity, the deep Q-learning network (DQN)-based content caching algorithm is proposed to achieve the optimal content caching strategy. Moreover, we further present a content update policy with user preference and content popularity prediction, so that the proposed algorithm can handle the variations of contents popularity in a timely manner. Simulation results demonstrate that the proposed scheme achieves better caching hit rate compared with existing algorithms.

With the prevalence of social media and online forum, opinion mining, aiming at analyzing and discovering the latent opinion in user-generated reviews on the Internet, has become a hot research topic. This survey focuses on two important subtasks in this field, stance detection and product aspect mining, both of which can be formalized as the problem of the triple (target, aspect, opinion) extraction. In this paper, we first introduce the general framework of opinion mining and describe the evaluation metrics. Then, the methodologies for stance detection on different sources, such as online forum and social media are discussed. After that, approaches for product aspect mining are categorized into three main groups which are corpus level aspect extraction, corpus level aspect, and opinion mining, and document level aspect and opinion mining based on the processing units and tasks. And then we discuss the challenges and possible solutions. Finally, we summarize the evolving trend of the reviewed methodologies and conclude the survey.

With the popularity of social media, including micro-blog, mining effective information in short texts has become an increasingly important issue. However, due to the sparseness, high dimensionality and large amount of data, mining this information is a very challenging task. In this paper, we propose a method to extend the Dirichlet multinomial mixture (DMM) topic model by combining the user-LDA topic model based on internal data expansion with the potential feature vector representation of words trained on a very large external corpus (we refer to it as ULW-DMM). The experimental results show that the ULW-DMM model produces a relatively large improvement in topic consistency and classification tasks for topic modeling of microblog short texts.

Smart campus builds on characteristic learning and feedback evaluation of diverse students and aims to enable intelligent, accurate, and customized education. Mining social media data, especially topic modeling, from students, provides a non-intrusive method to know the instantaneous thoughts and willings of them. However, it is challenging to deal with multi-modal data (i.e., text, images, and videos contained in the social media data) as well as the modality dependence and missing modality. In this paper, we present a novel deep topical correlation analysis (DTCA) approach, which achieves robust and accurate topic detection for microblogs and simultaneously handles the two challenges aforementioned. In particular, bidirectional recurrent neural networks and convolutional neural networks are used to learn deep textual and visual features, respectively. Then, a canonical correlation analysis-based fusion scheme is proposed, which has two innovations to deal with both modality independence and modality missing, i.e., a filter gate to capture the modality dependency and a matrix-projection based component to handle the missing modality. DTCA is trained in an end-to-end manner, in which the parameters of visual, textual, and cross-modal prediction parts are trained jointly. We further release a large-scale cross-modal twitter dataset for topic detection, denoted as TM-Twitter. On this dataset, extensive and quantitative evaluations are conducted with comparisons to several state-of-the-art and alternative approaches. Significant performance gains are reported to demonstrate the merits of the proposed DTCA.

User satisfaction is an important aspect to consider in any public transport system, and as such, regular and sound measurements of its levels are fundamental. However, typical evaluation schemes involve costly and time-consuming surveys. As a consequence, their frequency is not enough to properly and timely characterize the satisfaction of the users. In this paper, we propose a methodology, based on Twitter data, to capture the satisfaction of a large mass of users of public transport, allowing us to improve the characterization and location of their satisfaction level. We analyzed a massive volume of tweets referring to the public transport system in Santiago, Chile (Transantiago) using text mining techniques, such as sentiment analysis and topic modeling, in order to capture and group bus users' expressions. Results show that, although the level of detail and variety of answers obtained from surveys are higher than the ones obtained by our method, the amount of bus stops and bus services covered by the proposed scheme is larger. Moreover, the proposed methodology can be effectively used to diagnose problems in a timely manner, as it is able to identify and locate trends, and issues related to bus operating firms, whereas surveys tend to produce average answers. Based on the consistency and logic of the results, we argue that the proposed methodology can be used as a valuable complement to surveys, as both present different, but compatible characteristics.

News has been an important source for many financial time series predictions based on fundamental analysis. However, digesting a massive amount of news and data published on the Internet to predict a market can be burdensome. This paper introduces a topic model based on latent Dirichlet allocation (LDA) to discover features from a combination of text, especially news articles and financial time series, denoted as Financial LDA (FinLDA). The features from FinLDA are served as additional input features for any machine learning algorithm to improve the prediction of the financial time series. We provide posterior distributions used in Gibbs sampling for two variants of the FinLDA and propose a framework for applying the FinLDA in a text and data mining for financial time series prediction. The experimental results show that the features from the FinLDA empirically add value to the prediction and give better results than the comparative features including topic distributions from the common LDA.

Topic extraction is an essential task in bibliometric data analysis, data mining and knowledge discovery, which seeks to identify significant topics from text collections. The conventional topic extraction schemes require human intervention and involve also comprehensive pre-processing tasks to represent text collections in an appropriate way. In this paper, we present a two-stage framework for topic extraction from scientific literature. The presented scheme employs a two-staged procedure, where word embedding schemes have been utilized in conjunction with cluster analysis. To extract significant topics from text collections, we propose an improved word embedding scheme, which incorporates word vectors obtained by word2vec, POS2vec, word-position2vec and LDA2vec schemes. In the clustering phase, an improved clustering ensemble framework, which incorporates conventional clustering methods (i.e., k-means, k-modes, k-means++, self-organizing maps and DIANA algorithm) by means of the iterative voting consensus, has been presented. In the empirical analysis, we analyze a corpus containing 160,424 abstracts of articles from various disciplines, including agricultural engineering, economics, engineering and computer science. In the experimental analysis, performance of the proposed scheme has been compared to conventional baseline clustering methods (such as, k-means, k-modes, and k-means++), LDA-based topic modelling and conventional word embedding schemes. The empirical analysis reveals that ensemble word embedding scheme yields better predictive performance compared to the baseline word vectors for topic extraction. Ensemble clustering framework outperforms the baseline clustering methods. The results obtained by the proposed framework show an improvement in Jaccard coefficient, Folkes & Mallows measure and F1 score.

Digital libraries, journals and conference proceedings repositories are a great source of information. These sources are very useful for the purpose of research and development. This paper presents an overview of text mining and its application towards information extraction from literature. In this study, we used word cloud, term frequency analysis, similarity analysis, cluster analysis, and topic modeling to extract information from multi-domain research articles. Cloud computing and big data are new emerging trends. So it is important to extract useful patterns and knowledge from published articles in these domains and discover the relationship between them. Therefore, a total of two hundred research articles published from 2010 to 2018 in these domains, were selected. The source of these articles is high impact factor journals from reputed publishers namely IEEE, Springer, Wiley, Elsevier, and ACM. It is a cross-domain analysis in cloud computing and big data domains to find the latest trends, related topics, tools, terms, and author affiliation from extracted data. This study identifies the ten major areas of big data using cloud computing, fourteen factors towards cloud adoption, and hurdles in adoption. Moreover finding shows that IEEE has more sources for subject cloud computing application towards big data, then comes Springer, Wiley, and Elsevier. Furthermore, it has been observed in the analysis that the number of articles in these domains increased from 2013 onward.

The large amount of text that is generated daily on the web through comments on social networks, blog posts and open-ended question surveys, among others, demonstrates that text data is used frequently, and therefore; its processing becomes a challenge for researchers. The topic modeling is one of the emerging techniques in text mining; it is based on the discovery of latent data and the search for relationships among text documents. In this paper, the objective of the research is to evaluate a generic methodology based on topic modeling and text network modeling, that allows researchers to gather valuable information from surveys that use open-ended questions. To achieve this, this methodology has been evaluated through the use of a case study in which the responses to a teacher self-assessment survey in an Ecuadorian university have been studied. The main contribution of the article is the inclusion of clustering algorithms in order to complement the results obtained when executing topic modeling. The proposed methodology is based on four phases: (a) Construction of a text database, (b) Text mining and topic modeling, (c) Topic network modeling and (d) The relevance of the identified topics. In previous works, it has been observed that the human interpretative contribution plays an important role in the process, especially in phases (a) and (d). For this reason, the visualization interfaces, such as graphs and dendograms, are of critical importance for researchers in order allow topic to efficiently analyze the results of the topic modeling. As a result of this case study, a compendium of the main strategies that teachers carry out in their classes with the aim of improving student retention is presented. In addition, the proposed methodology can be extended to the analysis of the unstructured textual information found in blogs, social networks, forums, etc.

Community plays an important role in shaping a network. Quantitatively interpreting a community is necessary for graph generalization which is used for privacy preserving, summarization, and dimensionality reduction in social network mining. However, there are few works in community detection focusing on making sense of the identified communities. In this paper, we study communities in the social media context and investigate structure-based communities from the perspective of community topical homophily. We train Latent Dirichlet Allocation topic model to capture the topics in the aggregated tweets of each user in a community and propose new distance metrics to quantify the topic similarity of individual users, cliques, and communities. By building a Twitter topic modeling system to interpret the communities identified by two community detection algorithms in a large scale Twitter topology, we discover evidence that Twitter users in a community show common interests in general. The major contributions of this paper lie in that it verifies the topical homophily in structure-based social media communities and proposes new metrics to quantitatively label the degree of the homophily and describe the theme of the communities.

Online product reviews sentiment classification plays an important role on service recommendation, yet most of current researches on it only focus on single-modal information ignoring the complementary information, that results in unsatisfied accuracy of sentiment classification. This paper proposes a cross-modal hypergraph model to capture textual information and sentimental information simultaneously for sentiment classification of reviews. Furthermore, a mixture model by coupling the latent Dirichlet allocation topic model with the proposed cross-modal hypergraph is designed to mitigate the ambiguity of some specific words, which may express opposite polarity in different contexts. Experiments are carried out on four-domain data sets (books, DVD, electronics, and kitchen) to evaluate the proposed approaches by comparison with lexicon-based method, Naïve Bayes, maximum entropy, and support vector machine. Results demonstrate that our schemes outperform the baseline methods in sentiment classification accuracy.

Next-app prediction is the task of predicting the next app that a user will choose to use on the smartphone. It helps to establish a variety of intelligent personalized services, such as fast-launch UI app, intelligent user-phone interactions, and so on. Since app names only provide limited semantic information, the intrinsic relation among apps cannot be fully exploited. Meanwhile, next-app to be used is largely determined by a sequence of apps that a user used recently. To address these challenging problems, this paper first enriches the semantic information of apps by extracting descriptive text of each app from the app store and thus proposes a topic model to transform apps as well as user preferences into latent vectors. Then, a set of nearest neighbors can be constructed based on the similarity of latent vectors and it is employed for training the prediction model. Furthermore, our prediction scheme is built on the temporal sequential data and is modeled by using the chain-augmented Naive Bayes model. Experimental results with a real smartphone application log data have demonstrated that our method achieves higher recall and DCG values compared with several baseline next-app prediction methods.

People have their own tastes on visual appearances of products from various categories. For many of them, the tastes are affected by the current fashion trend. Studying visual appearances and fashion trend makes us understand the composition of users’ preferences and their purchase choices. However, since the fashion is changing over time, it is complex to model time-aware and non-time-aware variables simultaneously. In this paper, we present VIsually-aware Temporal rAting modeL with topics using review text to help mine visual dynamics and non-visual features for rating prediction task. Understanding the reviews will help the Recommender Systems (RSs) know whether a user is attracted by the appearance of an item, and which aspect of an item’s appearance contributes most to its ratings. To achieve this, we incorporate the visual information into the rating predicting function and introduce a topic model that can automatically classify words in an item’s reviews into non-visual words that explain the coherent feature, and visual words that are associated with its visual appearances in each time period, respectively. We run experiments on eleven real-world public datasets and the results show that our model performs better on predicting ratings than many of the state-of-the-art RSs, such as PMF, timeSVD++, HFT, JMARS, ETDR, and TVBPR+.

Short texts have become a kind of prevalent source of information, and discovering topical information from short text collections is valuable for many applications. Due to the length limitation, conventional topic models based on document-level word co-occurrence information often fail to distill semantically coherent topics from short text collections. On the other hand, word embeddings as a powerful tool have been successfully applied in natural language processing. Word embeddings trained on large corpus are encoded with general semantic and syntactic information of words, and hence they can be leveraged to guide topic modeling for short text collections as supplementary information for sparse co-occurrence patterns. However, word embeddings are trained on large external corpus and the encoded information is not necessarily suitable for training data set of topic models, which is ignored by most existing models. In this article, we propose a novel global and local word embedding-based topic model (GLTM) for short texts. In the GLTM, we train global word embeddings from large external corpus and employ the continuous skip-gram model with negative sampling (SGNS) to obtain local word embeddings. Utilizing both the global and local word embeddings, the GLTM can distill semantic relatedness information between words which can be further leveraged by Gibbs sampler in the inference process to strengthen semantic coherence of topics. Compared with five state-of-the-art short text topic models on four real-world short text collections, the proposed GLTM exhibits the superiority in most cases.

We define behavior as a set of actions performed by some actor during a period of time. We consider the problem of analyzing a large collection of behaviors by multiple actors, more specifically, identifying typical behaviors and spotting anomalous behaviors. We propose an approach leveraging topic modeling techniques – LDA (Latent Dirichlet Allocation) Ensembles – to represent categories of typical behaviors by topics that are obtained through topic modeling a behavior collection. When such methods are applied to text in natural languages, the quality of the extracted topics are usually judged based on the semantic relatedness of the terms pertinent to the topics. This criterion, however, is not necessarily applicable to topics extracted from non-textual data, such as action sets, since relationships between actions may not be obvious. We have developed a suite of visual and interactive techniques supporting the construction of an appropriate combination of topics based on other criteria, such as distinctiveness and coverage of the behavior set. Two case studies on analyzing operation behaviors in the security management system and visiting behaviors in an amusement park, and the expert evaluation of the first case study demonstrate the effectiveness of our approach.

Patients in hospitals, particularly in critical care, are susceptible to many complications affecting morbidity and mortality. Digitized clinical data in electronic medical records can be effectively used to develop machine learning models to identify patients at risk of complications early and provide prioritized care to prevent complications. However, clinical data from heterogeneous sources within hospitals pose signi ficant modeling challenges. In particular, unstructured clinical notes are a valuable source of information containing regular assessments of the patient's condition but contain inconsistent abbreviations and lack the structure of formal documents. Our contributions in this paper are twofold. First, we present a new preprocessing technique for extracting features from informal clinical notes that can be used in a classifi cation model to identify patients at risk of developing complications. Second, we explore the use of collective matrix factorization, a multi-view learning technique, to model heterogeneous clinical data text-based features in combination with other measurements, such as clinical investigations, comorbidities, and demographic data. We present a detailed case study on postoperative respiratory failure using more than 700 patient records from the MIMIC II database. Our experiments demonstrate the ef ficacy of our preprocessing technique in extracting discriminatory features from clinical notes as well as the bene fits of multi-view learning to combine clinical measurements with text data for predicting complications.

This paper presents a novel approach for spectral unmixing of remotely sensed hyperspectral data. It exploits probabilistic latent topics in order to take advantage of the semantics pervading the latent topic space when identifying spectral signatures and estimating fractional abundances from hyperspectral images. Despite the contrasted potential of topic models to uncover image semantics, they have been merely used in hyperspectral unmixing as a straightforward data decomposition process. This limits their actual capabilities to provide semantic representations of the spectral data. The proposed model, called dual-depth sparse probabilistic latent semantic analysis (DEpLSA), makes use of two different levels of topics to exploit the semantic patterns extracted from the initial spectral space in order to relieve the ill-posed nature of the unmixing problem. In other words, DEpLSA defines a first level of deep topics to capture the semantic representations of the spectra, and a second level of restricted topics to estimate endmembers and abundances over this semantic space. An experimental comparison in conducted using the two standard topic models and the seven state-of-the-art unmixing methods available in the literature. Our experiments, conducted using four different hyperspectral images, reveal that the proposed approach is able to provide competitive advantages over available unmixing approaches.

TToPIC (Tuning of Parameters for Inference of Concepts) is a distributed self-tuning engine whose aim is to cluster collections of textual data into correlated groups of documents through a topic modeling methodology (i.e., LDA). ToPIC includes automatic strategies to relieve the end-user of the burden of selecting proper values for the overall analytics process. ToPIC's current implementation runs on Apache Spark, a state-of-the-art distributed computing framework. As a case study, ToPIC has been validated on three real collections of textual documents characterized by different distributions. The experimental results show the effectiveness and efficiency of the proposed solution in analyzing collections of documents without tuning algorithm parameters and in discovering cohesive and well-separated groups of documents with a similar topic.

Previous studies have found that a significant number of bug reports are misclassified between bugs and non-bugs, and that manually classifying bug reports is a time-consuming task. To address this problem, we propose a bug reports classification model with N-gram IDF, a theoretical extension of Inverse Document Frequency (IDF) for handling words and phrases of any length. N-gram IDF enables us to extract key terms of any length from texts, these key terms can be used as the features to classify bug reports. We build classification models with logistic regression and random forest using features from N-gram IDF and topic modeling, which is widely used in various software engineering tasks. With a publicly available dataset, our results show that our N-gram IDF-based models have a superior performance than the topic-based models on all of the evaluated cases. Our models show promising results and have a potential to be extended to other software engineering tasks., 5 pages, ICSME 2017

The growing need to analyze large collections of documents has led to great developments in topic modeling. Since documents are frequently associated with other related variables, such as labels or ratings, much interest has been placed on supervised topic models. However, the nature of most annotation tasks, prone to ambiguity and noise, often with high volumes of documents, deem learning under a single-annotator assumption unrealistic or unpractical for most real-world applications. In this article, we propose two supervised topic models, one for classification and another for regression problems, which account for the heterogeneity and biases among different annotators that are encountered in practice when learning from crowds. We develop an efficient stochastic variational inference algorithm that is able to scale to very large datasets, and we empirically demonstrate the advantages of the proposed model over state-of-the-art approaches., Comment: 14 pages

The topic model based on latent Dirichlet allocation relies on the prior statistics of topic proportionals for multinomial words. The words in a document are modeled as a random mixture of latent topics which are drawn from a single Dirichlet prior. However, a single Dirichlet distribution may not sufficiently characterize the variations of topic proportionals estimated from the heterogeneous documents. To deal with this concern, we present a Dirichlet mixture allocation (DMA) model which learns latent topics and their proportionals for topic and document clustering by using the prior based on a Dirichlet mixture model. Multiple Dirichlets pave a way to capture the structure of latent variables in learning representation from real-world documents covering a variety of topics. This paper builds a new latent variable model and develops a variational Bayesian inference procedure to learn model parameters consisting of mixture weights, Dirichlet parameters and word multinomials. Experiments on document representation show the merit of the proposed structural learning by increasing the number of Dirichlets in a DMA topic model.

Bug reports play essential roles in many software engineering tasks. Since validity and performance of these tasks definitely rely on the quality of bug reports, accurate information from bug reports is very important. However, as found in previous study, significant numbers of reports classified as bug are not really a bug. Recent studies proposed techniques to automatically classify bug reports into binary classes, yet there is still more to desire. These bug reports can be classified into multiple classes, which could help to identify what these reports are actually about. Moreover, previous study only looks into one possibility of topic modeling, that is, Latent Dirichlet Allocation (LDA). While LDA has its advantage, parameter tuning is required. In this paper, we propose a nonparametric approach to automatically classify bug reports with, another topic modeling method, Hierarchical Dirichlet Process (HDP). The result indicates that our nonparametric approach performance is comparable to the parametric one. We also examine various aspects of LDA to provide more thoroughly understanding of this process., SNPD 2014 : 15th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, 30 June-2 July 2014, Las Vegas, NV, USA

It is well known that one way to improve the accuracy of a text retrieval system is to expand the original query with additional knowledge coded through topic-related terms. In the case of an interactive environment, the expansion, which is usually represented as a list of words, is extracted from documents whose relevance is known thanks to the feedback of the user. In this paper we argue that the accuracy of a text retrieval system can be improved if we employ a query expansion method based on a mixed Graph of Terms representation instead of a method based on a simple list of words. The graph, that is composed of a directed and an undirected subgraph, can be automatically extracted from a small set of only relevant documents (namely the user feedback) using a method for term extraction based on the probabilistic Topic Model. The evaluation of the proposed method has been carried out by performing a comparison with two less complex structures: one represented as a set of pairs of words and another that is a simple list of words.