Search

Identifying the development trends of emerging technologies with disruptive potential as early as possible is crucial for enterprise research and development (R&D) investment planning and government R&D strategic planning. Many researchers have used academic papers and patent data to identify trends in emerging technologies, although they rarely have made use of web news data focused on emerging technologies. Web news contains a large amount of social awareness data that represents the public's sense of and response to emerging technologies, and the social awareness data is of great significance for identifying the development trends of emerging technologies. Therefore, this article presented a research framework to identify the development trends of emerging technologies using patent analysis and web news data mining. In the research framework, first, we have used patent analysis to analyze the evolution topics of these emerging technologies, and have applied the topic model to study the changing patterns of publics’ topics of concern regarding emerging technologies contained in web news. Second, we have made a comparison analysis of the results of patent analysis and the public’ sense of and response to emerging technologies contained in web news. Then, we have applied an improved sentiment analysis to study the changing patterns of publics’ expectations for emerging technologies contained in web news. Finally, by studying the changes in publics’ topics of concern and publics’ expectations for emerging technologies over time, we have identified the development trends of emerging technologies. The perovskite solar cells technology is used as a case study to analyze the effectiveness and feasibility of the framework. This article provides a new research perspective to identify technology trends, and contribute to the technology forecasting methodology.

CyberPhysicalSocial System (CPSS) technology closely integrates and coordinates computing resources, physical resources, and social information to provide humans with efficient and convenient services. Service recommendation technology satisfies the individual needs of users by collecting, processing, and analyzing user characteristic data. In recent years, service recommendation algorithms based on social networks and collaborative filtering have been widely used in CPSS. However, this type of service recommendation algorithm cannot effectively use the aliasing and auxiliary information in the service, nor can it generate different service recommendation schemes according to different scenarios. We propose a collaborative filtering service recommendation algorithm that combines heterogeneous information networks and topic models for CPSS. The proposed algorithm designs a more effective functional similarity measurement method through the word vectors learned by Word2vec. Then, we use the topic model to cluster the scenario-based service (SBS), thereby greatly reducing the time required for service recommendation. Experimental analysis results show that we finally determine the best combination of Mashup similarity between the two scenarios with better recommendation efficiency and accuracy. Thus, the proposed recommendation algorithm can effectively improve the quality of service of CPSS, and provide a basis for the personalized service of CPSS.

For multimodal representation learning, traditional black-box approaches often fall short of extracting interpretable multilayer hidden structures, which contribute to visualize the connections between different modalities at multiple semantic levels. To extract interpretable multimodal latent representations and visualize the hierarchial semantic relationships between different modalities, based on deep topic models, we develop a novel multimodal Poisson gamma belief network (mPGBN) that tightly couples the observations of different modalities via imposing sparse connections between their modality-specific hidden layers. To alleviate the time-consuming Gibbs sampler adopted by traditional topic models in the testing stage, we construct a Weibull-based variational inference network (encoder) to directly map the observations to their latent representations, and further combine it with the mPGBN (decoder), resulting in a novel multimodal Weibull variational autoencoder (MWVAE), which is fast in out-of-sample prediction and can handle large-scale multimodal datasets. Qualitative evaluations on bimodal data consisting of image-text pairs show that the developed MWVAE can successfully extract expressive multimodal latent representations for downstream tasks like missing modality imputation and multimodal retrieval. Further extensive quantitative results demonstrate that both MWVAE and its supervised extension sMWVAE achieve state-of-the-art performance on various multimodal benchmarks.

Topic modeling is a significant branch of natural language processing and machine learning focused on inferring the generative process of text. Traditionally, algorithms for estimating topic models have relied on Bayesian inference and Gibbs sampling. This paper proposes a novel “acceptable set” framework for formulating topic modeling problems inspired by ideas from discrete component analysis and data-driven robust optimization. Our approach not only simplifies the design and inference of topic models, but also allows for extensions and generalizations that are challenging to integrate into traditional approaches. Different restrictions (e.g., sparsity) and assumptions (e.g., alternative generative processes) can be easily incorporated into our formulations through additional or modified constraints. Our formulations also naturally control a widely used metric of model quality, perplexity. We adapt state-of-the-art stochastic gradient methods to find good local optima for the optimization formulations. The algorithms are efficient, scaling to realistic problem sizes with runtimes comparable to existing methods. Through extensive computational experiments, we show that our methods have improved solution quality compared to baseline methods and reconstruct more reliably the underlying generative models. Our framework overcomes known vulnerabilities of traditional topic modeling algorithms: our methods are effective in low-data settings, register good out-of-sample performance, and perform well for a variety of initial assumptions on input parameter values.

In the era of social networks, the rapid growth of data mining in information retrieval and natural language processing makes automatic text summarization necessary. Currently, pretrained word embedding and sequence to sequence models can be effectively adapted in social network summarization to extract significant information with strong encoding capability. However, how to tackle the long text dependence and utilize the latent topic mapping has become an increasingly crucial challenge for these models. In this article, we propose a topic-aware extractive and abstractive summarization model named T-BERTSum, based on Bidirectional Encoder Representations from Transformers (BERTs). This is an improvement over previous models, in which the proposed approach can simultaneously infer topics and generate summarization from social texts. First, the encoded latent topic representation, through the neural topic model (NTM), is matched with the embedded representation of BERT, to guide the generation with the topic. Second, the long-term dependencies are learned through the transformer network to jointly explore topic inference and text summarization in an end-to-end manner. Third, the long short-term memory (LSTM) network layers are stacked on the extractive model to capture sequence timing information, and the effective information is further filtered on the abstractive model through a gated network. In addition, a two-stage extractive-abstractive model is constructed to share the information. Compared with the previous work, the proposed model T-BERTSum focuses on pretrained external knowledge and topic mining to capture more accurate contextual representations. Experimental results on the CNN/Daily mail and XSum datasets demonstrate that our proposed model achieves new state-of-the-art results while generating consistent topics compared with the most advanced method.

Document clustering allows a user to group semantically similar documents. It has been an interesting research area for the past many years and various methods and techniques have been developed. However, the research has primarily been limited to English and other high resource languages. For low-resource languages, such as Urdu, the area of document clustering is open to contributions. This work presents an experimental evaluation of clustering techniques on Urdu tweets. It is a challenging task to semantically cluster tweets due to their very short length. In this paper, various features, including sentence and phrase-level embeddings, TF-IDF features and document embeddings are extracted from tweets and clustering is performed using three different algorithms: K-Means, Bisecting K-Means, and Affinity Propagation algorithms. Furthermore, a comparison is performed with the traditional topic modeling approach. The results indicate that the TF-IDF features combined with the K-means clustering algorithm outperformed the adopted clustering techniques.

Probabilistic topic models, such as latent Dirichlet allocation (LDA), are often used to discover hidden semantic structure of a collection of documents. In recent years, various inference algorithms have been developed to cope with learning of topic models, among which Gibbs sampling methods remain a popular choice. In this paper, we aim to improve the inference of topic models based on the Gibbs sampling framework. We extend a state augmentation based Gibbs sampling method by maximizing the replications of latent states, and propose a new generic deterministic inference method, named maximal latent state replication (MAX), for learning of a family of probabilistic topic models. One key benefit of the proposed method lies in the deterministic nature for inference, which may help to improve its running efficiency as well as predictive perplexity. We have conducted extensive experiments on real-life publicly available datasets, and the results have validated that our proposed method MAX significantly outperforms state-of-the-art baselines for inference of existing well-known topic models.

The widespread use of positioning devices has given rise to many trajectories, with each having three explicit attributes: user ID, location ID, and time-stamp and an implicit attribute: activity type (akin to “topic” in text mining). To model these trajectories, existing works learn different attribute representations by either introducing latent activity types based on topic models or transforming the location and time context into a low-dimensional space via embedding techniques. In this paper, we propose a holistic approach named Human Mobility Representation Model (HMRM) to simultaneously produce the vector representations of all four (explicit and implicit) attributes. The merits of HMRM lie in that: (1) it models the latent activity types and learns trajectory attribute embeddings in an integrated manner, and (2) it connects the activity-related distributions and these attributes embeddings by adding a newly designed collaborative learning component, and makes them mutually exchanged to take the best of both worlds. We apply HMRM to both unsupervised and supervised tasks including two activity evaluation tasks and two embedding evaluation tasks, on two real check-in datasets collected from Foursquare. Experimental results show that HMRM could not only improve the performance of capturing latent activity types, but also learn better trajectory embeddings.

Recently, advances in neural network approaches have achieved many successes in both sentiment classification and probabilistic topic modelling. On the one hand, latent topics derived from the global context of documents could be helpful in capturing more accurate word semantics and hence could potentially improve the sentiment classification accuracy. On the other hand, the word-level attention vectors obtained during the learning of sentiment classifiers could carry word-level polarity information and can be used to guide the discovery of topics in topic modelling. This paper proposes a multi-task learning framework which jointly learns a sentiment classifier and a topic model by making the word-level latent topic distributions in the topic model to be similar to the word-level attention vectors in the classifier through mutual learning. Experimental results on the Yelp and IMDB datasets verify the superior performance of the proposed framework over strong baselines on both sentiment classification accuracy and topic modelling evaluation results including perplexity and topic coherence measures. The proposed framework also extracts more interpretable topics compared to other conventional topic models and neural topic models.

We introduce and assess the use of supervised learning in cross-domain topic classification. In this approach, an algorithm learns to classify topics in a labeled source corpus and then extrapolates topics in an unlabeled target corpus from another domain. The ability to use existing training data makes this method significantly more efficient than within-domain supervised learning. It also has three advantages over unsupervised topic models: the method can be more specifically targeted to a research question and the resulting topics are easier to validate and interpret. We demonstrate the method using the case of labeled party platforms (source corpus) and unlabeled parliamentary speeches (target corpus). In addition to the standard within-domain error metrics, we further validate the cross-domain performance by labeling a subset of target-corpus documents. We find that the classifier accurately assigns topics in the parliamentary speeches, although accuracy varies substantially by topic. We also propose tools diagnosing cross-domain classification. To illustrate the usefulness of the method, we present two case studies on how electoral rules and the gender of parliamentarians influence the choice of speech topics., Political Analysis, 31 (1), ISSN:1047-1987, ISSN:1476-4989

With the rapid growth of social network users, the social network has accumulated massive social network topics. However, due to the randomness of content, it becomes sparse and noisy, accompanied by many daily chats and meaningless topics, which brings challenges to bursty topics discovery. To deal with these problems, this paper proposes the spatial-temporal topic model with sparse prior and recurrent neural networks (RNN) prior for bursty topic discovering (ST-SRTM). The semantic relationship of words is learned through RNN to alleviate the sparsity. The spatial-temporal areas information is introduced to focus on bursty topics for further weakening the semantic sparsity of social network context. Besides, we introduced the “Spike and Slab” prior to decouple the sparseness and smoothness. Simultaneously, we realized the automatic discovery of social network bursts by introducing the burstiness of words as the prior and binary switching variables. We constructed multiple sets of comparative experiments to verify the performance of ST-SRTM by leveraging different evaluation indicators on real Sina Weibo data sets. The experimental results confirm the superiority of our ST-SRTM.

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm., arXiv admin note: text overlap with arXiv:1808.02215 by other authors

The parallel Hierarchical Dirichlet Process (pHDP) is an efficient topic model which explores the equivalence of the generation process between Hierarchical Dirichlet Process (HDP) and Gamma-Gamma-Poisson Process (G2PP), in order to achieve parallelism at the topic level. Unfortunately, pHDP loses the non-parametric feature of HDP, i.e., the number of topics in pHDP is predetermined and fixed. Furthermore, under the bootstrap structure of pHDP, the topic-indiscriminate words are of high probabilities to be assigned to different topics, resulting in poor qualities of the extracted topics. To achieve parallelism without sacrificing the non-parametric feature of HDP, in addition to improve the quality of extracted topics, we propose a parallel dynamic topic model by developing an adjustment mechanism of evolving topics and reducing the sampling probabilities of topic-indiscriminate words. Both supervised and unsupervised experiments on benchmark datasets show the competitive performance of our model.

The amount of digital text-based consumer review data has increased dramatically and there exist many machine learning approaches for automated text-based sentiment analysis. Marketing researchers have employed various methods for analyzing text reviews but lack a comprehensive comparison of their performance to guide method selection in future applications. We focus on the fundamental relationship between a consumer’s overall empirical evaluation, and the text-based explanation of their evaluation. We study the empirical tradeoff between predictive and diagnostic abilities, in applying various methods to estimate this fundamental relationship. We incorporate methods previously employed in the marketing literature, and methods that are so far less common in the marketing literature. For generalizability, we analyze 25,119 products in nine product categories, and 260,489 reviews across five review platforms. We find that neural network-based machine learning methods, in particular pre-trained versions, offer the most accurate predictions, while topic models such as Latent Dirichlet Allocation offer deeper diagnostics. However, neural network models are not suited for diagnostic purposes and topic models are ill equipped for making predictions. Consequently, future selection of methods to process text reviews is likely to be based on analysts’ goals of prediction versus diagnostics.

Sentiment analysis systems can collect and automatically structure unstructured information by collecting public views of services, products, policy, brands, etc. This information is of major value in the fields of marketing analysis, public relations, product reviews, net promoter evaluations, customer feedback and client reviews. Literary works, on the other hand, are less susceptible to computational analysis because there are no immediate commercial incentives. However, similar techniques can be used to evaluate literary work, comprehend the underlying social network, and obtain or validate literary work. This project is about analyzing the book's characters and predicting their characteristics and relationships with one another. A lot of human effort is expended during the adaptation of a novel/book in any form, which is inconvenient and undesirable. Furthermore, the human brain has a tendency to overlook a number of minor details about the events/characters in the book. The scenario described above can frequently result in inaccuracies in the adaptation's plot. As a result, the project is an innovation that aims to aid in the easy and accurate adaptation of a book, making the process much simpler and precise. Machine learning (supervised and unattended) and lexical approaches include the current sentiment analysis techniques. The model's goal is to scan the massive amounts of text in the book. Following digitization, the model will display interesting ideas derived from the given book using a combination of natural language processing, feelings and emotions analysis, and social network analysis methodology.

Generating explanations within a local and model-agnostic explanation scenario for text classification is often accompanied by a local approximation task. In order to create a local neighborhood for a document, whose classification shall be explained, sampling techniques are used that most often treat the according features at least semantically independent from each other. Hence, contextual as well as semantic information is lost and therefore cannot be used to update a human’s mental model within the according explanation task. In case of dependent features, such explanation techniques are prone to extrapolation to feature areas with low data density, therefore causing misleading interpretations. Additionally, the ”the whole is greater than the sum of its parts” phenomenon is disregarded when using explanations that treat the according words independently from each other. In this paper, an architecture named CaSE is proposed that either uses Semantic Feature Arrangements or Semantic Interrogations to overcome these drawbacks. Combined with a modified version of Local interpretable model-agnostic explanations (LIME), a state of the art local explanation framework, it is capable of generating meaningful and coherent explanations. The approach utilizes contextual and semantic knowledge from unsupervised topic models in order to enable realistic and semantic sampling and based on that generate understandable explanations for any text classifier. The key concepts of CaSE that are deemed essential for providing humans with high quality explanations are derived from findings of psychology. In a nutshell, CaSE shall enable Semantic Alignment between humans and machines and thus further improve the basis for Interactive Machine Learning. An extensive experimental validation of CaSE is conducted, showing its effectiveness by generating reliable and meaningful explanations whose elements are made of contextually coherent words and therefore are suitable to update human mental models in an appropriate way. In the course of a quantitative analysis, the proposed architecture is evaluated w.r.t. a consistency property and to Local Fidelity of the resulting explanation models. According to that, CaSE generates more realistic explanation models leading to higher Local Fidelity compared to LIME.

Learning hidden topics from data streams has become absolutely necessary but posed challenging problems such as concept drift as well as short and noisy data. Using prior knowledge to enrich a topic model is one of potential solutions to cope with these challenges. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g. Word2vec) is very valuable and useful to help topic models work better. However, in a streaming environment where data arrives continually and infinitely, existing studies are limited to exploiting these resources effectively. Especially, a knowledge graph, that contains meaningful word relations, is ignored. In this paper, to aim at exploiting a knowledge graph effectively, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both a human knowledge graph (Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift. The implementation of GCTM is available at \url{https://github.com/bachtranxuan/GCTM.git}.

Blood pressure (BP) is an important indicator of an individual's health status and is closely related to daily behaviors. Thus, a continuous daily measurement of BP is critical for hypertension control. To assist continuous measurement, BP prediction based on non-physiological data (ubiquitous mobile phone data) was studied in the research. An algorithm was proposed that predicts BP based on patients’ daily routine, which includes activities such as sleep, work, and commuting. The aim of the research is to provide insight into the application of mobile data in telemonitoring and the continuous unobtrusive daily measurement of BP. A half-year data set from October 2017 of 320 individuals, including telecom data and BP measurement data, was analyzed. Two hierarchical Bayesian topic models were used to extract individuals’ location-driven daily routine patterns (topics) and calculate probabilities among these topics from their day-level mobile trajectories. Based on the topic probability distribution and patients’ contextual data, their BP were predicted using different models. The prediction model comparison shows that the long short-term memory (LSTM) method exceeds others when the data has a high dependency. Otherwise, the Random Forest regression model outperforms the LSTM method. Also, the experimental results validate the effectiveness of the topics in BP prediction.

Sparse and short news headlines can be arbitrary, noisy, and ambiguous, making it difficult for classic topic model LDA (latent Dirichlet allocation) designed for accommodating long text to discover knowledge from them. Nonetheless, some of the existing research about text-based crude oil forecasting employs LDA to explore topics from news headlines, resulting in a mismatch between the short text and the topic model and further affecting the forecasting performance. Exploiting advanced and appropriate methods to construct high-quality features from news headlines becomes crucial in crude oil forecasting. This paper introduces two novel indicators of topic and sentiment for the short and sparse text data to tackle this issue. Empirical experiments show that AdaBoost.RT with our proposed text indicators, with a more comprehensive view and characterization of the short and sparse text data, outperforms the other benchmarks. Another significant merit is that our method also yields good forecasting performance when applied to other futures commodities.

It is necessary to reduce the cognitive overhead of interpreting the native topic term list of the Latent Dirichlet Allocation (LDA) style topic model. In this regard, automatic topic labeling has become an effective approach to generate meaningful alternative representations of topics discovered for end-users. In this study, we introduced a novel two-phase neural embedding framework with the redundancy-aware graph-based ranking process. It demonstrated how pre-trained neural embedding could be usefully applied in topic terms, sentence presentations, and automatic topic labeling tasks. Moreover, reranking the topic terms optimized the discovered topics with fewer yet more representative terms while retaining the topic information integrality and fidelity. It further decreased the burden of computation caused by neural embedding and improved the overall effectiveness of the labeling system. Compared with the prevailing state-of-the-art and classical labeling systems, our efficient model boosted the quality of the topic labels generated and discovered more meaningful topic labels.

Learning from past incidents is critical to achieving and maintaining high process safety performance. Accident and incident records provide one way for learning; however, these are usually in the form of unstructured texts, which makes analysis difficult. Recently, text mining methods based on supervised learning have been proposed for analyzing accident reports; however, they require an impractically large number of labeled records as training examples. This paper proposes an automated, semi-supervised, domain-independent approach for analyzing accident reports. Given a set of user-defined classification topics and domain literature such as handbooks, glossaries, and Wikipedia articles, the method can identify domain-specific keywords and group them into topics with minimal expert involvement. These keywords and topics can then be used for various data mining purposes, including classification. The proposed approach is demonstrated using two different case studies across domains: (1) in aviation to identify the stage of flight when an accident occurs, and (2) in the process industry domain to identify the cause of pipeline accidents. The average classification accuracy of the proposed method was 80% which is comparable to that of supervised learning methods. The key benefits of this approach are that it can generate domain-specific predictive models with limited manual intervention.

The increasing importance of online distribution channels is paralleled by a rising interest in gaining insights into the customer journey and browsing behavior. We evaluate several machine learning methods (latent Dirichlet allocation, correlated topic model, structural topic model, replicated softmax model) with respect to their ability to reproduce the browsing behavior of households across websites. In addition, we compare these machine learning methods to a related classical technique, singular value decomposition. In our study, the replicated softmax model outperforms latent Dirichlet allocation, but the correlated topic model attains the overall best performance. Compared to singular value decomposition both the correlated topic model and the replicated softmax model lead to a more efficient compression of web browsing data. On the other hand, singular value decomposition surpasses latent Dirichlet allocation. We interpret results of the correlated topic model and the replicated softmax model by determining combinations of topics or hidden variables that are heterogeneous with respect to visited websites. We show that decision makers should not rely on bivariate measures of site visits, as these do not agree with measures of interdependences between sites that can be inferred from the correlated topic model or the replicated softmax model. We investigate how well topics or hidden variables measured by these methods predict yearly household expenditures. The correlated topic model leads to the best predictive performance, followed by the replicated softmax model. We also discuss how the replicated softmax model can be used to support online marketing decisions of websites.

Dynamic text documents, including news articles, user reviews, and blogs, are now commonly encountered in many fields. Accordingly, the topics underlying text streams also change over time. To grasp the topic changes in the increasing accumulation of text documents, there is a great need to develop automatic text analysis models to find the key changes in topics. To this end, this study proposes a topic change point detection (Topic-CD) model. Different from previous studies, we define the change point of topics from the perspective of hyperparameters associated with topic-word distributions. This allows the model to detect change points underlying the whole topic set. Under this definition, the topic modeling and change point detection are combined in a unified framework and then performed simultaneously using a Markov chain Monte Carlo algorithm. In addition, the Topic-CD model is free from setting the number of change points in advance, which makes it more convenient for practical use. We investigate the performance of the Topic-CD model numerically using synthetic data and three real datasets. The results show that the Topic-CD model can well identify the change points in topics when compared with several state-of-the-art methods.

Topic models have been widely used for learning the latent explainable representation of documents, but most of the existing approaches discover topics in a flat structure. In this study, we propose an effective hierarchical neural topic model with strong interpretability. Unlike the previous neural topic models, we explicitly model the dependency between layers of a network, and then combine latent variables of different layers to reconstruct documents. Utilizing this network structure, our model can extract a tree-shaped topic hierarchy with low redundancy and good explainability by exploiting dependency matrices. Furthermore, we introduce manifold regularization into the proposed method to improve the robustness of topic modeling. Experiments on real-world datasets validate that our model outperforms other topic models in several widely used metrics with much fewer computation costs.

Recently, listwise collaborative filtering (CF) algorithms are attracting increasing interest due to their efficiency and prediction quality. Different from rating-oriented (pointwise) CF, they recommend a preference ranking of items to each user without estimating the absolute value of the ratings. In practice, there are extensive side information about users and items in many corpora that can improve recommendation performance and specially help CF methods to address the cold-start problem. However, a model has not been proposed that incorporates side information to listwise CF. Therefore, in this work, a Bayesian graphical model, called Content-Aware Listwise Collaborative Filtering (CALCF), is developed which incorporates text information, represented as a bag of words, to listwise CF using topic models. We propose a Gibbs sampler with closed-form samples using data augmentation techniques to infer the latent variables. CALCF has been validated by comparison with previous listwise and pointwise algorithms on implicit (click, view, purchase) and explicit (numeric ratings) feedback data in both cold-start and warm-start scenarios. The results demonstrate that in most cases CALCF achieves better Normalized Discounted Cumulative Gain (NDCG) and Recall at top M recommendation compared with the previous models.

Social event classification has always been a research topic of great interest in the field of social event analysis. In existing social event classification methods, the probability of word generation under topics is calculated from its frequency of occurrence in a corpus. In fact, documents of each modality in a corpus of multi-modal social events contain rich semantics. We argue that Semantic Words carry considerable information explicitly or implicitly expressed in multi-modal documents, and therefore the model should pay more attention to these semantically rich words in the event classification task. In this paper, we propose a multi-modal supervised topic model based on semantic weighting (Sem-MMSTM), which uses two kinds of semantics, namely part of speech semantics and category semantics. Our Sem-MMSTM model is capable of mining and making use of the semantics of the text itself and the its category semantics of multi-modal supervised corpus. The experimental results show that,compared with the state-of-the-art model, our proposed Sem-MMSTM yields significant performance improvement both on the metrics of classification accuracy (ACC) and interpretability of topics (PMI) due to the introduction of effective semantic information.

Many B2B firms have widely accepted AI-based chatbots to provide human-like service interaction at different customer touchpoints in recent years. One of the objectives behind introducing this technology is to provide an enhanced, live channel Customer Experience (CX) all round the clock. Researchers have focused on delivering the CX by improvising the chatbot's internal algorithm, giving limited attention to CX theories from management literature, which leaves a gap. With the proposed paper, we have investigated the influencing factors of AI-based chatbots from the lens of CX theories for B2B firms. In this paper, a model for organizing CX has been proposed using the diffusion of innovation theory, trust commitment theory, information systems success model, and Hoffman & Novak's flow model for the computer-mediated environment and verified using the social media data. The methodology used for this study is the social media analytics-based content analysis method (sentiment analysis, hierarchical clustering, topic modeling) for data preparation, followed by lasso and ridge regression for model verification. The results suggest that CX in B2B enterprises using chatbots is influenced by these bots' overall system design, customers' ability to use technology, and customer trust towards brand and system.

Aim: To identify knowledge gaps in heart failure (HF) research among women, especially postmenopausal women. Materials & methods: We retrieved HF articles from PubMed. Natural language processing and text mining techniques were used to screen relevant articles and identify study objective(s) from abstracts. After text preprocessing, we performed topic modeling with non-negative matrix factorization to cluster articles based on the primary topic. Clusters were independently validated and labeled by three investigators familiar with HF research. Results: Our model yielded 15 topic clusters from articles on HF among women. Atrial fibrillation was found to be the most understudied topic. From articles specific to postmenopausal women, five clusters were identified. The smallest cluster was about stress-induced cardiomyopathy. Conclusion: Topic modeling can help identify understudied areas in medical research.

Deciding when and which products to recommend to whom is always an essential issue for retailers. In this study, we propose a mixed framework with two components to capture customer buying behavior and its changes over time and visualize these results to better help retailers choose and target products strategically for marketing. In this framework, a topic model is first used to extract customer’s purchase behavior instead of association rules or K-means as mainly used in market field. To automatically choose the optimal number of topics, we implement an approach proposed by Koltcov et al. on point-of-sale (POS) data in the supermarket. Meanwhile, to grasp the change of topics over time, we divided monthly POS data in half and applied the topic model with Renyi entropy separately. The results suggest that splitting data might be a better way to understand customer behavior. Second, we consider how to develop an effective way to visualize the results of the topic model, which is essential, because in a supermarket context, simply knowing which product categories are included under which topics is not enough to support how a supermarket promotes their products. To address this, we design a three-layer visualization approach to better interpret the topic model results and to help retailers design target promotion strategies. The design of visualization was overlooked by studies related to the use of topic models on supermarket data. Finally, to demonstrate the usefulness of our proposed framework, we conduct a simple scenario-based analysis between our framework and other models, such as Latent Dirichlet Allocation (LDA) and the Dynamic Topic Model (DTM). The results show that for most periods, our proposed framework outperforms LDA and DTM.

Virus Corona atau Covid-19 menjadi perhatian khusus diseluruh dunia. Banyak masyarakat membicarakan virus ini melalui unggahan komentar dan opini di Media Sosial.Twitter merupakan salah satu media sosial yang saat ini masih banyak digunakan masyarakat untuk menyampaikan opini berupa kumpulan kata yang disebut tweets. Tweets yang berkaitan dengan topik covid-19 ini dapat di klasifikasikan menggunakan metode Topic Modeling untuk menghasilkan sebuah data topic yang sering dibicarakan pengguna twitter. Salah satu algoritma yang digunakan untuk melakukan Topic Modeling adalah menggunakan Latent Dirichlet Alocation (LDA). Pada penelitian ini LDA digunakan untuk mengetahui kata-kata apa saja yang banyak muncul pada data tweets tentang Covid-19 yang telah di unggah masyarakat melalui twitter. Sebelum data tweet dimodelkan dengan LDA, dilakukan terlebih dahulu analisis sentiment dengan Naïve Bayes Classifier untuk menghasilkan sentiment Positif, Negati dan Netral. Terdapat 5000 tweets dijadikan dataset untuk diklasifikasikan menggunakan Topic Modeling. Semua tweets yang yang di tambang masih perlu dilakukan preprocessing text yang bertujuan untuk menghapus kata-kata yang tidak baku, menghapus tanda baca, dan menghapus kata penyambung. Tweets yang sudah dilakukan preprocessing text lalu diberikan nilai bobot sehingga diketahui kata apa saja yang banyak muncul dalam tweets yang berkaitan dengan Covid-19. Kata-kata yang banyak muncul dan sudah diberikan bobot akan divisualisasikan menggunakan World Cloud sehingga dapat dilihat pemetaan kata apa saja yang banyak muncul dalam bentuk gambar.

User relations and information propagation on social networks can reflect events in real society. Online detection of bursty events is of great significance to studying the evolution of social networks and cyberspace security. Current research works focus on building an event recognition model based on text information and then utilizing clustering or topic model methods to extract features from the data stream and then detect events that have not existed before. The text-based method is designed for specific content, and it does not consider the features of network dynamic evolution. It is restricted by the type and quality of text in social networks, limiting its practical application scenarios. However, there exist remarkable correlations between the occurrence of events and the evolution of the network. In this paper, we consider mining the network structure changes to identify bursty events, the superiority which is that it is sensitive and widely used. We integrate snapshot network topology indexes to quantify its structural features. Then we can judge whether there is a burst event by investigating the change degree of the network structure features of the adjacent snapshots. The effectiveness and efficiency of our approach are further confirmed by experimental studies on four real social network data sets. In addition, we also discuss the salient features of the bursty events and compare the impact of the bursty events on the network structure with that of the scheduled events.

The advent of single-cell RNA sequencing (scRNA-seq) technologies has revolutionized transcriptomic studies. However, large-scale integrative analysis of scRNA-seq data remains a challenge largely due to unwanted batch effects and the limited transferabilty, interpretability, and scalability of the existing computational methods. We present single-cell Embedded Topic Model (scETM). Our key contribution is the utilization of a transferable neural-network-based encoder while having an interpretable linear decoder via a matrix tri-factorization. In particular, scETM simultaneously learns an encoder network to infer cell type mixture and a set of highly interpretable gene embeddings, topic embeddings, and batch-effect linear intercepts from multiple scRNA-seq datasets. scETM is scalable to over 106 cells and confers remarkable cross-tissue and cross-species zero-shot transfer-learning performance. Using gene set enrichment analysis, we find that scETM-learned topics are enriched in biologically meaningful and disease-related pathways. Lastly, scETM enables the incorporation of known gene sets into the gene embeddings, thereby directly learning the associations between pathways and topics via the topic embeddings., Computational single-cell RNA-seq analyses often face challenges in scalability, model interpretability, and confounders. Here, we show a new model to address these challenges by learning meaningful embeddings from the data that simultaneously refine gene signatures and cell functions in diverse conditions.

PurposePrevious research concerning automatic extraction of research topics mostly used rule-based or topic modeling methods, which were challenged due to the limited rules, the interpretability issue and the heavy dependence on human judgment. This study aims to address these issues with the proposal of a new method that integrates machine learning models with linguistic features for the identification of research topics.Design/methodology/approachFirst, dependency relations were used to extract noun phrases from research article texts. Second, the extracted noun phrases were classified into topics and non-topics via machine learning models and linguistic and bibliometric features. Lastly, a trend analysis was performed to identify hot research topics, i.e. topics with increasing popularity.FindingsThe new method was experimented on a large dataset of COVID-19 research articles and achieved satisfactory results in terms of f-measures, accuracy and AUC values. Hot topics of COVID-19 research were also detected based on the classification results.Originality/valueThis study demonstrates that information retrieval methods can help researchers gain a better understanding of the latest trends in both COVID-19 and other research areas. The findings are significant to both researchers and policymakers.

According to the survey, India has the world's second-largest newspaper market, with more than 100 K newspaper outlets, approx 240 million circulation, and 1300 million subscribers or readers. The topic modeling work is increasing day by day, and researchers have published multiple topic modeling papers and have implemented them in different areas like software engineering, political science and medical, etc. LDA topic modeling is used in this research because it has been introduced successfully for topic modeling and classification and it measures the probability of a text-dependent on the bag-of-words scheme without considering the word series. LDA is a common topic modeling algorithm with excellent implementation in the Gensim Python package. However, the challenge is how to extract good quality topics that are simple, separated, and meaningful. The purpose of this research deals with finding the main topics of the same category news articles which are in two different languages (Hindi and English) and then classifying these different language news topics with similarity measurement. In this research, the corpus is constructed with bigram. To achieve the research goal, we have to first build a headline and link extractor that scrap the top news from Google News feeds for both English and Hindi languages (Google News collects news stories that have appeared on different news website which is already accessible in 35 languages over the last 30 days) and then analyses which two news headlines are similar.

Topic models are widely used in building clusters of documents for more than a decade, yet problems occurring in choosing the optimal number of topics. The main problem is the lack of a stable metric of the quality of topics obtained during the construction of topic models. The authors analyzed from previous works, most of the models used in determining the number of topics are non-parametric and the quality of topics determined by using perplexity and coherence measures and concluded that they are not applicable in solving this problem. In this paper, we used the parametric method, which is an extension of the traditional topic model with visual access tendency for visualization of the number of topics (clusters) to complement clustering and to choose the optimal number of topics based on results of cluster validity indices. Developed hybrid topic models are demonstrated with different Twitter datasets on various topics in obtaining the optimal number of topics and in measuring the quality of clusters. The experimental results showed that the Visual Non-negative Matrix Factorization (VNMF) topic model performs well in determining the optimal number of topics with interactive visualization and in performance measure of the quality of clusters with validity indices.

Understanding the real-world short texts become an essential task in the recent research area. The document deduction analysis and latent coherent topic named as the important aspect of this process. Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic Analysis (PLSA) are suggested to model huge information and documents. This type of contexts’ main problem is the information limitation, words relationship, sparsity, and knowledge extraction. The knowledge discovery and machine learning techniques integrated with topic modeling were proposed to overcome this issue. The knowledge discovery was applied based on the hidden information extraction to increase the suitable dataset for further analysis. The integration of machine learning techniques, Artificial Neural Network (ANN) and Long Short-Term (LSTM) are applied to anticipate topic movements. LSTM layers are fed with latent topic distribution learned from the pre-trained Latent Dirichlet Allocation (LDA) model. We demonstrate general information from different techniques applied in short text topic modeling. We proposed three categories based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation using representative design and analysis of all categories’ performance in different tasks. Finally, the proposed system evaluates with state-of-art methods on real-world datasets, comprises them with long document topic modeling algorithms, and creates a classification framework that considers further knowledge and represents it in the machine learning pipeline.

We propose a dynamic joint sentiment-topic model (dJST) which allows the detection and tracking of views of current and recurrent interests and shifts in topic and sentiment. Both topic and sentiment dynamics are captured by assuming that the current sentiment-topic specific word distributions are generated according to the word distributions at previous epochs. We derive efficient online inference procedures to sequentially update the model with newly arrived data and show the effectiveness of our proposed model on the Mozilla add-on reviews crawled between 2007 and 2011.

The problem of detecting anomalous documents in text collections is considered. The existing methods for detecting anomalies are not universal and do not show a stable result on different data sets. The accuracy of the results depends on the choice of parameters at each step of the problem solving algorithm process, and for different collections different sets of parameters are optimal. Not all of the existing algorithms for detecting anomalies work effectively with text data, which vector representation is characterized by high dimensionality with strong sparsity.The problem of finding anomalies is considered in the following statement: it is necessary to checking a new document uploaded to an applied intelligent information system for congruence with a homogeneous collection of documents stored in it. In such systems that process legal documents the following limitations are imposed on the anomaly detection methods: high accuracy, computational efficiency, reproducibility of results and explicability of the solution. Methods satisfying these conditions are investigated.The paper examines the possibility of evaluating text documents on the scale of anomaly by deliberately introducing a foreign document into the collection. A strategy for detecting novelty of the document in relation to the collection is proposed, which assumes a reasonable selection of methods and parameters. It is shown how the accuracy of the solution is affected by the choice of vectorization options, tokenization principles, dimensionality reduction methods and parameters of novelty detection algorithms.The experiment was conducted on two homogeneous collections of documents containing technical norms: standards in the field of information technology and railways. The following approaches were used: calculation of the anomaly index as the Hellinger distance between the distributions of the remoteness of documents to the center of the collection and to the foreign document; optimization of the novelty detection algorithms depending on the methods of vectorization and dimensionality reduction. The vector space was constructed using the TF-IDF transformation and ARTM topic modeling. The following algorithms have been tested: Isolation Forest, Local Outlier Factor and One-Class SVM (based on Support Vector Machine).The experiment confirmed the effectiveness of the proposed optimization strategy for determining the appropriate method for detecting anomalies for a given text collection. When searching for an anomaly in the context of topic clustering of legal documents, the Isolating Forest method is proved to be effective. When vectorizing documents using TF-IDF, it is advisable to choose the optimal dictionary parameters and use the One-Class SVM method with the corresponding feature space transformation function.

This paper describes the implementation and evaluation of a proof-of-concept Question Answering (QA) system for accessing chemical data from knowledge graphs (KGs) which offer data from chemical kinetics to the chemical and physical properties of species. We trained the question classification and named the entity recognition models that specialize in interpreting chemistry questions. The system has a novel design which applies a topic model to identify the question-to-ontology affiliation to handle ontologies with different structures. The topic model also helps the system to provide answers with a higher quality. Moreover, a new method that automatically generates training questions from ontologies is also implemented. The question set generated for training contains 432,989 questions under 11 types. Such a training set has been proven to be effective for training both the question classification model and the named entity recognition model. We evaluated the system using other KGQA systems as baselines. The system outperforms the chosen KGQA system answering chemistry-related questions. The QA system is also compared to the Google search engine and the WolframAlpha engine. It shows that the QA system can answer certain types of questions better than the search engines.

The Lanzhou-Xinjiang (Lan-Xin) high-speed railway is one of the principal sections of the railway network in western China, and signal equipment is of great importance in ensuring the safe and efficient operation of the high-speed railway. Over a long period, in the railway operation and maintenance process, the railway signaling and communications department has recorded a large amount of unstructured text information about equipment faults in the form of natural language. However, due to irregularities in the recording methods of these data, it is difficult to use directly. In this paper, a method based on natural language processing (NLP) was adopted to analyze and classify this information. First, the Latent Dirichlet Allocation (LDA) topic model was used to extract the semantic features of the text, which were then expressed in the corresponding topic feature space. Next, the Support Vector Machine (SVM) algorithm was used to construct a signal equipment fault diagnostic model that reduced the impact of sample data imbalance on the classification accuracy. This was compared and analyzed with the traditional Naive Bayes (NB), Logistic Regression (LR), Random Forest (RF), and K-Nearest Neighbor (KNN) algorithms. This study used signal equipment failure text data from the Lan-Xin high-speed railway to conduct experimental analysis and verify the effectiveness of the proposed method. Experiments showed that the accuracy of the SVM classification algorithm could reach 0.84 after being combined with the LDA topic model, which verifies that the natural language processing method can effectively realize the fault diagnosis of signal equipment and has certain guiding significance for the maintenance of field signal equipment.

Latent Dirichlet Allocation (LDA) has been widely used for topic modeling, with applications spanning various areas such as natural language processing and information retrieval. While LDA on small and static datasets has been extensively studied, several real-world challenges are posed in practical scenarios where datasets are often huge and are gathered in a streaming fashion. As the state-of-the-art LDA algorithm on streams, Streaming Variational Bayes (SVB) introduced Bayesian updating to provide a streaming procedure. However, the utility of SVB is limited in applications since it ignored three challenges of processing real-world streams: topic evolution , data turbulence , and real-time inference . In this article, we propose a novel distributed LDA algorithm—referred to as StreamFed-LDA— to deal with challenges on streams. For topic modeling of streaming data, the ability to capture evolving topics is essential for practical online inference. To achieve this goal, StreamFed-LDA is based on a specialized framework that supports lifelong (continual) learning of evolving topics. On the other hand, data turbulence is commonly present in streams due to real-life events. In that case, the design of StreamFed-LDA allows the model to learn new characteristics from the most recent data while maintaining the historical information. On massive streaming data, it is difficult and crucial to provide real-time inference results. To increase the throughput and reduce the latency, StreamFed-LDA introduces additional techniques that substantially reduce both computation and communication costs in distributed systems. Experiments on four real-world datasets show that the proposed framework achieves significantly better performance of online inference compared with the baselines. At the same time, StreamFed-LDA also reduces the latency by orders of magnitudes in real-world datasets.

This article uses a machine learning algorithm to demonstrate a proof-of-concept case for moderating and managing online comments as a form of content moderation, which is an emerging area of interest for technical and professional communication (TPC) researchers. The algorithm sorts comments by topical similarity to a reference comment/article rather than display comments by linear time and popularity. This approach has the practical benefit of enabling TPC researchers to reconceptualize content display systems in dynamic ways.

An innovative model-based approach to coupling text clustering and topic modeling is introduced, in which the two tasks take advantage of each other. Specifically, the integration is enabled by a new generative model of text corpora. This explains topics, clusters and document content via a Bayesian generative process. In this process, documents include word vectors, to capture the (syntactic and semantic) regularities among words. Topics are multivariate Gaussian distributions on word vectors. Clusters are assigned corresponding topic distributions as their semantics. Content generation is ruled by text clusters and topics, which act as interacting latent factors. Documents are at first placed into respective clusters, then the semantics of these clusters is then repeatedly sampled to draw document topics, which are in turn sampled for word-vector generation. Under the proposed model, collapsed Gibbs sampling is derived mathematically and implemented algorithmically with parameter estimation for the simultaneous inference of text clusters and topics. A comparative assessment on real-world benchmark corpora demonstrates the effectiveness of this approach in clustering texts and uncovering their semantics. Intrinsic and extrinsic criteria are adopted to investigate its topic modeling performance, whose results are shown through a case study. Time efficiency and scalability are also studied.

We provide a first comprehensive structuring of the literature applying machine learning to finance. We use a probabilistic topic modeling approach to make sense of this diverse body of research spanning across the disciplines of finance, economics, computer sciences, and decision sciences. Through the topic modelling approach, a Latent Dirichlet Allocation technique, we are able to extract the 14 coherent research topics that are the focus of the 5,204 academic articles we analyze from the years 1990 to 2018. We first describe and structure these topics, and then further show how the topic focus has evolved over the last two decades. Our study thus provides a structured topography for finance researchers seeking to integrate machine learning research approaches in their exploration of finance phenomena. We also showcase the benefits to finance researchers of the method of probabilistic modeling of topics for deep comprehension of a body of literature, especially when that literature has diverse multi-disciplinary actors.

Existing software intelligent defect classification approaches don’t consider radar characters and prior statistics information. Thus when applying these appaoraches into radar software testing and validation, the precision rate and recall rate of defect classification are poor and have effect on the reuse effectiveness of software defects. To solve this problem, a new intelligent defect classification approach based on the latent Dirichlet allocation (LDA) topic model is proposed for radar software in this paper. The proposed approach includes the defect text segmentation algorithm based on the dictionary of radar domain, the modified LDA model combining radar software requirement, the top acquisition and classification approach of radar software defect based on the modified LDA model. The proposed approach is applied on the typical radar software defects to validate the effectiveness and applicability. The application results illustrate that the prediction precison rate and recall rate of the poposed approach are improved up to 15%~20% compared with the other defect classification approaches. Thus, the proposed approach can be applied in the segmentation and classification of radar software defecs effectively to improve the identifying adequacy of the defects in radar software.

Automatic generation of hierarchies from social tags is a challenging task. We identified three rules, set inclusion, graph centrality and information-theoretic condition from the literature and proposed two new rules, fuzzy set inclusion and probabilistic association to induce hierarchical relations. We proposed an hierarchy generation algorithm, which can incorporate each rule with different data representations, i.e., resource and Probabilistic Topic Model based representations. The learned hierarchies were compared to some of the widely used reference concept hierarchies. We found that probabilistic association and set inclusion based rules helped produce better quality hierarchies according to the evaluation metrics.