Search

Cycling is an eco-friendly and sustainable mode of transportation. Despite its benefits, the cyclists’ risk of collision is still high when interacting with other road users. This study analyzed self-reported near-miss and collision event descriptions for the United States provided by the crowdsourcing platform, BikeMaps.org. Innovative and efficient analytic methods are needed to generate useful information from unstructured textual data sources in the transportation domain. In this study, explorative text mining, topic modeling, and machine learning are utilized to gain insights from the unstructured textual descriptions of crowdsourced near-miss and collision events. The approaches are used to unveil prevalent words and word associations for near-miss and collision events. Structural Topic Modeling (STM) is deployed to autogenerate latent themes or topics from the event descriptions. The generated topic proportions are used as input in Artificial Neural Networks (ANN) to estimate the cyclist’s propensity to a collision. It was found that cyclists had a higher propensity to a collision in topics that articulated vehicle encroachment to the bike lane, on-street parking close or into the bike lane resulting in dooring, and drivers’ violations at the crosswalk. The results and methodology used in this study can assist engineers, policymakers, and law enforcement officers to proactively reduce potential cyclist collisions, prioritizing areas where cyclist safety improvements are needed, and ultimately promoting bicycle ridership in our communities.

Ranked keyword search has gained increasing attention due to its attractive advantages such as flexibility and accessibility. However, most existing ranked keyword search schemes ignore the semantic associations between the documents and queries. To solve this challenging issue in cloud-assisted edge computing, we first design the Semantic-aware Ranked Multi-keyword \textbf{S}earch (SRMS) scheme by adopting the Latent Dirichlet Allocation (LDA) topic model and the Chinese Remainder Theorem (CRT)-based secret sharing mechanism. Considering that the cloud server may be malicious, we implement a basic verification mechanism in SRMS to verify the correctness and completeness of search results and extend this verification mechanism in cloud-assisted edge computing scenarios. Formal security analysis proves that SRMS and extended result verification mechanisms are secure in both the known ciphertext model and the known background model. Extensive experiments using the real-world dataset demonstrate that SRMS is efficient and feasible in practical applications.

CyberPhysicalSocial System (CPSS) technology closely integrates and coordinates computing resources, physical resources, and social information to provide humans with efficient and convenient services. Service recommendation technology satisfies the individual needs of users by collecting, processing, and analyzing user characteristic data. In recent years, service recommendation algorithms based on social networks and collaborative filtering have been widely used in CPSS. However, this type of service recommendation algorithm cannot effectively use the aliasing and auxiliary information in the service, nor can it generate different service recommendation schemes according to different scenarios. We propose a collaborative filtering service recommendation algorithm that combines heterogeneous information networks and topic models for CPSS. The proposed algorithm designs a more effective functional similarity measurement method through the word vectors learned by Word2vec. Then, we use the topic model to cluster the scenario-based service (SBS), thereby greatly reducing the time required for service recommendation. Experimental analysis results show that we finally determine the best combination of Mashup similarity between the two scenarios with better recommendation efficiency and accuracy. Thus, the proposed recommendation algorithm can effectively improve the quality of service of CPSS, and provide a basis for the personalized service of CPSS.

For multimodal representation learning, traditional black-box approaches often fall short of extracting interpretable multilayer hidden structures, which contribute to visualize the connections between different modalities at multiple semantic levels. To extract interpretable multimodal latent representations and visualize the hierarchial semantic relationships between different modalities, based on deep topic models, we develop a novel multimodal Poisson gamma belief network (mPGBN) that tightly couples the observations of different modalities via imposing sparse connections between their modality-specific hidden layers. To alleviate the time-consuming Gibbs sampler adopted by traditional topic models in the testing stage, we construct a Weibull-based variational inference network (encoder) to directly map the observations to their latent representations, and further combine it with the mPGBN (decoder), resulting in a novel multimodal Weibull variational autoencoder (MWVAE), which is fast in out-of-sample prediction and can handle large-scale multimodal datasets. Qualitative evaluations on bimodal data consisting of image-text pairs show that the developed MWVAE can successfully extract expressive multimodal latent representations for downstream tasks like missing modality imputation and multimodal retrieval. Further extensive quantitative results demonstrate that both MWVAE and its supervised extension sMWVAE achieve state-of-the-art performance on various multimodal benchmarks.

One of the most significant differences of M5 over previous forecasting competitions is that it was held on Kaggle, an online platform for data scientists and machine learning practitioners. Kaggle provides a gathering place, or virtual community, for web users who are interested in the M5 competition. Users can share code, models, features, and loss functions through online notebooks and discussion forums. Here, we study the social influence of this virtual community on user behavior in the M5 competition. We first research the content of the M5 virtual community by topic modeling and trend analysis. Further, we perform social media analysis to identify the potential relationship network of the virtual community. We study the roles and characteristics of some key participants who promoted the diffusion of information within the M5 virtual community. Overall, this study provides in-depth insights into the mechanism of the virtual community’s influence on the participants and has potential implications for future online competitions.

Traditional searchable encryption schemes based on the Term Frequency-Inverse Document Frequency (TF-IDF) model adopt the presence of keywords to measure the relevance of documents to queries, which ignores the latent semantic meanings that are concealed in the context. Latent Dirichlet Allocation (LDA) topic model can be utilized for modeling the semantics among texts to achieve semantic-aware multi-keyword search. However, the LDA topic model treats queries and documents from the perspective of topics, and the keywords information is ignored. In this paper, we propose a privacy-preserving searchable encryption scheme based on the LDA topic model and the query likelihood model. We extract the feature keywords from the document using the LDA-based Information Gain (IG) and Topic Frequency-Inverse Topic Frequency (TF-ITF) model. With feature keyword extraction and the query likelihood model, our scheme can achieve a more accurate semantic-aware keyword search. A special index tree is used to enhance search efficiency. The secure inner product operation is utilized to implement the privacy-preserving ranked search. The experiments on real-world datasets demonstrate the effectiveness of our scheme.

The demand for Natural Language Processing has been thriving rapidly due to the various emerging Internet services such as social networks, e-commerce, intelligent assistant, etc. The large volume of natural language data and the new market requirements have brought significant challenges and opportunities for academia and industry. The challenges include (1) efficiently processing a massive volume of data and devising a highly scalable architecture, (2) extending the horizon of the current natural language processing to tackle problems that cannot be handled well. This thesis demonstrates several advanced techniques to tackle the challenges by boosting scalability and efficiency or extending the horizon of existing core NLP techniques on different granularities. This thesis introduces (i) a neural-based lexical simplification algorithm that utilizes contextualized word embedding to capture the meaning more accurately, (ii) a sample-efficient algorithm that solves extreme large-output classification for contrastive representation learning. (iii) a scalable and efficient algorithm for the probabilistic topic model LDA.First, we propose amplified negative sampling and rigorous mathematical analysis of the famous negative sampling method for the word level distributed representations. Specifically, we distinguish the impact of various loss functions for the contrastive representation learning process. Our theoretical analysis provides a better understanding of a few empirical observations that are well known among practitioners when negative sampling is employed. We also propose the amplified negative sampling based on our analysis. It's a simple yet efficient method that can effectively improve the negative sampling efficiency and boost the downstream tasks' performance.Second, we propose the neural simplicity ranking (NSR) model that utilizes the contextualized word representations in an unconventional way for lexical simplification for the sentence level tasks. This model creatively uses the contextualized word embeddings beyond its original masked language model (MLM) task to identify the relatedness among candidate words and the original words. By redesigning the feature set for the simplicity measure and the ranking scheme, the NSR model achieves new state-of-the-art performance.Finally, we propose sd-LDA for the paragraph-level tasks, a scalable disk-based LDA model with sparse-prior sampling. The sd-LDA brings a highly optimized disk-based design that reduces the memory consumption by two orders of magnitude, making it possible to do large topic analysis on mobile devices. A sparse prior sampling technique is introduced in this work to accelerate the processing time. The sd-LDA algorithm can effectively reduce the memory consumption and processing time at the same time without loss of performance.

Personalized venues recommendation is essential to help people find attractive venue to visit as growth of location-based social networks. Existing approaches never distinguish user individual interests from her social preferences, which leads to a bottleneck of modeling user check-in behaviors accurately. In this paper, we find the differences between user interests and her social preferences clearly and investigate the time law of user check-in behaviors in depth. Consequently, we propose a social-period-aware topic model (SPATM) to learn the influence weights of both user interests and her social preferences on making-decision for each check-in time automatically. Especially, we model latent topic by leveraging smaller size of dynamic activities instead of static categories, which can alleviate the data sparsity problem by using more co-occurrent activities information. Moreover, our approach can automatically judge whether a user's social preference is periodic or aperiodic and learn the periodicity of periodic one. Furthermore, the Alias Sampling based training approach is introduced to improve sampling efficiency. The results demonstrate our proposed model is effective and outperforms the state-of-the-art approaches in terms of effectiveness and efficiency. Besides, SPATM can learn semantically coherent latent topics and geographically dispersed latent social topics which are useful to explain recommendation.

Topic modeling is a significant branch of natural language processing and machine learning focused on inferring the generative process of text. Traditionally, algorithms for estimating topic models have relied on Bayesian inference and Gibbs sampling. This paper proposes a novel “acceptable set” framework for formulating topic modeling problems inspired by ideas from discrete component analysis and data-driven robust optimization. Our approach not only simplifies the design and inference of topic models, but also allows for extensions and generalizations that are challenging to integrate into traditional approaches. Different restrictions (e.g., sparsity) and assumptions (e.g., alternative generative processes) can be easily incorporated into our formulations through additional or modified constraints. Our formulations also naturally control a widely used metric of model quality, perplexity. We adapt state-of-the-art stochastic gradient methods to find good local optima for the optimization formulations. The algorithms are efficient, scaling to realistic problem sizes with runtimes comparable to existing methods. Through extensive computational experiments, we show that our methods have improved solution quality compared to baseline methods and reconstruct more reliably the underlying generative models. Our framework overcomes known vulnerabilities of traditional topic modeling algorithms: our methods are effective in low-data settings, register good out-of-sample performance, and perform well for a variety of initial assumptions on input parameter values.

In the era of social networks, the rapid growth of data mining in information retrieval and natural language processing makes automatic text summarization necessary. Currently, pretrained word embedding and sequence to sequence models can be effectively adapted in social network summarization to extract significant information with strong encoding capability. However, how to tackle the long text dependence and utilize the latent topic mapping has become an increasingly crucial challenge for these models. In this article, we propose a topic-aware extractive and abstractive summarization model named T-BERTSum, based on Bidirectional Encoder Representations from Transformers (BERTs). This is an improvement over previous models, in which the proposed approach can simultaneously infer topics and generate summarization from social texts. First, the encoded latent topic representation, through the neural topic model (NTM), is matched with the embedded representation of BERT, to guide the generation with the topic. Second, the long-term dependencies are learned through the transformer network to jointly explore topic inference and text summarization in an end-to-end manner. Third, the long short-term memory (LSTM) network layers are stacked on the extractive model to capture sequence timing information, and the effective information is further filtered on the abstractive model through a gated network. In addition, a two-stage extractive-abstractive model is constructed to share the information. Compared with the previous work, the proposed model T-BERTSum focuses on pretrained external knowledge and topic mining to capture more accurate contextual representations. Experimental results on the CNN/Daily mail and XSum datasets demonstrate that our proposed model achieves new state-of-the-art results while generating consistent topics compared with the most advanced method.

Currently, the world is in the midst of a severe global pandemic, which has affected all aspects of people's lives. As a result, there is a deluge of COVID-related digital media articles published in the United States, due to the disparate effects of the pandemic. This large volume of information is difficult to consume by the audience in a reasonable amount of time. In this paper, we develop a Natural Language Processing (NLP) pipeline that is capable of automatically distilling various digital articles into manageable pieces of information, while also modelling the progression topics discussed over time in order to aid readers in rapidly gaining holistic perspectives on pressing issues (i.e., the COVID-19 pandemic) from a diverse array of sources. We achieve these goals by first collecting a large corpus of COVID-related articles during the onset of the pandemic. After, we apply unsupervised and semi-supervised learning procedures to summarize articles, then cluster them based on their similarities using the community detection methods. Next, we identify the topic of each cluster of articles using the BART algorithm. Finally, we provide a detailed digital media analysis based on the NLP-pipeline outputs and show how the conversation surrounding COVID-19 evolved over time.

Classroom observation is an essential component to improve the quality of teaching and develop educational research. Nonetheless, traditional observation procedures involve previously trained observers, rendering them expensive and time-consuming. Thus, there is a need for a tool which enables to analyze multiple lessons in short time, represent those lessons consistently, and describe them according to the teaching strategies used over time to give teachers timely and continuous feedback. In this work, we propose two unsupervised approaches to represent the evolution of lessons based on topic modeling: textbook- and BERT-based. This way, we can improve the generalization capability of these tools and avoid the need of previously annotated data. Subsequently, we collapsed the derived topics into two super-topics to ease interpretability, and clustered the time series they define to characterize them according to the teaching strategies used. We applied our methodology to 195 science lessons revealing three patterns related to different strategies. Lastly, we compare the two approaches using different metrics for clustering assessment and discuss their benefits and detriments. We expect these models can become useful tools for teachers to obtain an automated and objective representation of their lessons, but also to other stakeholders who want to analyze teaching practices at a large scale.

The aim of the paper is to exploit Twitter to enhance the comprehension of COVID-19 diffusion and people reactions. To this purpose, the objectives are to identify the key terms and features used in the tweets, and the interest in the COVID-19 topics. To address those goals the paper proposes an approach that combines peak detection and clustering techniques. Space-time features are extracted from the tweets and modeled as time series. After that, peaks are detected and the textual features are clustered based on the co-occurrence in the tweets. Each cluster obtained is then associated to a topic. Results, performed over a real-world dataset of tweets related to COVID-19 in US, show that the approach is able to detect several relevant topics, of varying importance and character. A case study about the correlation of Twitter data with COVID-19 confirmed cases has been presented, also evaluating the feasibility of exploiting Twitter for the outbreak diffusion prediction. Results highlight a high correlation between tweets and real COVID-19 data, proving that Twitter can be considered a reliable indicator of the epidemic spreading and that data generated by user activity on social media is becoming an invaluable source for capturing and understanding epidemics outbreaks. IEEE

Deep learning models have achieved high performance across different domains, such as medical decision-making, autonomous vehicles, decision support systems, among many others. However, despite this success, the inner mechanisms of these models are opaque because their internal representations are too complex for a human to understand. This opacity makes it hard to understand the how or the why of the predictions of deep learning models. There has been a growing interest in model-agnostic methods that make deep learning models more transparent and explainable to humans. Some researchers recently argued that for a machine to achieve human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals. This paper presents an in-depth systematic review of the diverse existing literature on counterfactuals and causability for explainable artificial intelligence (AI). We performed a Latent Dirichlet topic modeling analysis (LDA) under a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework to find the most relevant literature articles. This analysis yielded a novel taxonomy that considers the grounding theories of the surveyed algorithms, together with their underlying properties and applications to real-world data. Our research suggests that current model-agnostic counterfactual algorithms for explainable AI are not grounded on a causal theoretical formalism and, consequently, cannot promote causability to a human decision-maker. Furthermore, our findings suggest that the explanations derived from popular algorithms in the literature provide spurious correlations rather than cause/effects relationships, leading to sub-optimal, erroneous, or even biased explanations. Thus, this paper also advances the literature with new directions and challenges on promoting causability in model-agnostic approaches for explainable AI.

Document clustering allows a user to group semantically similar documents. It has been an interesting research area for the past many years and various methods and techniques have been developed. However, the research has primarily been limited to English and other high resource languages. For low-resource languages, such as Urdu, the area of document clustering is open to contributions. This work presents an experimental evaluation of clustering techniques on Urdu tweets. It is a challenging task to semantically cluster tweets due to their very short length. In this paper, various features, including sentence and phrase-level embeddings, TF-IDF features and document embeddings are extracted from tweets and clustering is performed using three different algorithms: K-Means, Bisecting K-Means, and Affinity Propagation algorithms. Furthermore, a comparison is performed with the traditional topic modeling approach. The results indicate that the TF-IDF features combined with the K-means clustering algorithm outperformed the adopted clustering techniques.

Probabilistic topic models, such as latent Dirichlet allocation (LDA), are often used to discover hidden semantic structure of a collection of documents. In recent years, various inference algorithms have been developed to cope with learning of topic models, among which Gibbs sampling methods remain a popular choice. In this paper, we aim to improve the inference of topic models based on the Gibbs sampling framework. We extend a state augmentation based Gibbs sampling method by maximizing the replications of latent states, and propose a new generic deterministic inference method, named maximal latent state replication (MAX), for learning of a family of probabilistic topic models. One key benefit of the proposed method lies in the deterministic nature for inference, which may help to improve its running efficiency as well as predictive perplexity. We have conducted extensive experiments on real-life publicly available datasets, and the results have validated that our proposed method MAX significantly outperforms state-of-the-art baselines for inference of existing well-known topic models.

The widespread use of positioning devices has given rise to many trajectories, with each having three explicit attributes: user ID, location ID, and time-stamp and an implicit attribute: activity type (akin to “topic” in text mining). To model these trajectories, existing works learn different attribute representations by either introducing latent activity types based on topic models or transforming the location and time context into a low-dimensional space via embedding techniques. In this paper, we propose a holistic approach named Human Mobility Representation Model (HMRM) to simultaneously produce the vector representations of all four (explicit and implicit) attributes. The merits of HMRM lie in that: (1) it models the latent activity types and learns trajectory attribute embeddings in an integrated manner, and (2) it connects the activity-related distributions and these attributes embeddings by adding a newly designed collaborative learning component, and makes them mutually exchanged to take the best of both worlds. We apply HMRM to both unsupervised and supervised tasks including two activity evaluation tasks and two embedding evaluation tasks, on two real check-in datasets collected from Foursquare. Experimental results show that HMRM could not only improve the performance of capturing latent activity types, but also learn better trajectory embeddings.

Firms apply strategic foresight in technology and innovation management to detect discontinuous changes early, to assess their expected consequences, and to develop a future course of action enabling superior company performance. For this purpose, an ever-increasing amount of data has to be collected, analyzed, and interpreted. Still, a major part of these activities is performed manually, which requires high investments in various resources. To support these processes more efficiently, this article presents an artificial-intelligence-based data mining model that helps firms spot emerging topics and trends at a higher level of automation than before. Its modular structure consists of components for query generation, data collection, data preprocessing, topic modeling, topic analysis, and visualization, combined in such a way that only a minimum amount of manual effort is required during its initial set up. The approach also incorporates self-adaptive capabilities, allowing the model to automatically update itself once new data has become available. The model parameterization is based on latest research in this area, and its threshold parameter is learnt during supervised training using a training data set. We have applied our model to an independent test data set to verify its effectiveness as an early warning system. By means of a retrospective analysis, we show in three case studies that our model is able to identify emerging technologies prior to their first publication in the Gartner Hype Cycle for Emerging Technologies. Based on our findings, we derive both theoretical and practical implications for the technology and innovation management of firms, and we suggest future research opportunities to further advance this field.

With the prevalence of smartphones, people have left abundant behavior records in cyberspace. Discovering and understanding individuals' cyber activities can provide useful implications for policymakers, service providers, and app developers. In this paper, we propose a framework to discover daily cyber activity patterns across people's mobile app usage. We first segment app usage traces into small time windows and then design a probabilistic topic model to infer users' cyber activities of each window. By exploring the coherence of users' activity sequences, the daily patterns of individuals are identified. Next, we recognize the common patterns across diverse groups of individuals using a hierarchical clustering algorithm. We then apply our framework on a large-scale and real-world dataset, consisting of 653,092 users with 971,818,946 usage records of 2,000 popular mobile apps. Our analysis shows that people usually obey yesterday's activity patterns, but the patterns tend to deviate as the time-lapse increases. We also discover five common daily cyber activity patterns, including afternoon reading, nightly entertainment, pervasive socializing, commuting, and nightly socializing. Our findings have profound implications on identifying the demographics of users and their lifestyles, habits, service requirements, and further detecting other disrupting trends such as working overtime and addiction to the game and social media.

Recently, advances in neural network approaches have achieved many successes in both sentiment classification and probabilistic topic modelling. On the one hand, latent topics derived from the global context of documents could be helpful in capturing more accurate word semantics and hence could potentially improve the sentiment classification accuracy. On the other hand, the word-level attention vectors obtained during the learning of sentiment classifiers could carry word-level polarity information and can be used to guide the discovery of topics in topic modelling. This paper proposes a multi-task learning framework which jointly learns a sentiment classifier and a topic model by making the word-level latent topic distributions in the topic model to be similar to the word-level attention vectors in the classifier through mutual learning. Experimental results on the Yelp and IMDB datasets verify the superior performance of the proposed framework over strong baselines on both sentiment classification accuracy and topic modelling evaluation results including perplexity and topic coherence measures. The proposed framework also extracts more interpretable topics compared to other conventional topic models and neural topic models.

We introduce and assess the use of supervised learning in cross-domain topic classification. In this approach, an algorithm learns to classify topics in a labeled source corpus and then extrapolates topics in an unlabeled target corpus from another domain. The ability to use existing training data makes this method significantly more efficient than within-domain supervised learning. It also has three advantages over unsupervised topic models: the method can be more specifically targeted to a research question and the resulting topics are easier to validate and interpret. We demonstrate the method using the case of labeled party platforms (source corpus) and unlabeled parliamentary speeches (target corpus). In addition to the standard within-domain error metrics, we further validate the cross-domain performance by labeling a subset of target-corpus documents. We find that the classifier accurately assigns topics in the parliamentary speeches, although accuracy varies substantially by topic. We also propose tools diagnosing cross-domain classification. To illustrate the usefulness of the method, we present two case studies on how electoral rules and the gender of parliamentarians influence the choice of speech topics., Political Analysis, 31 (1), ISSN:1047-1987, ISSN:1476-4989

With the rapid growth of social network users, the social network has accumulated massive social network topics. However, due to the randomness of content, it becomes sparse and noisy, accompanied by many daily chats and meaningless topics, which brings challenges to bursty topics discovery. To deal with these problems, this paper proposes the spatial-temporal topic model with sparse prior and recurrent neural networks (RNN) prior for bursty topic discovering (ST-SRTM). The semantic relationship of words is learned through RNN to alleviate the sparsity. The spatial-temporal areas information is introduced to focus on bursty topics for further weakening the semantic sparsity of social network context. Besides, we introduced the “Spike and Slab” prior to decouple the sparseness and smoothness. Simultaneously, we realized the automatic discovery of social network bursts by introducing the burstiness of words as the prior and binary switching variables. We constructed multiple sets of comparative experiments to verify the performance of ST-SRTM by leveraging different evaluation indicators on real Sina Weibo data sets. The experimental results confirm the superiority of our ST-SRTM.

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm., arXiv admin note: text overlap with arXiv:1808.02215 by other authors

The parallel Hierarchical Dirichlet Process (pHDP) is an efficient topic model which explores the equivalence of the generation process between Hierarchical Dirichlet Process (HDP) and Gamma-Gamma-Poisson Process (G2PP), in order to achieve parallelism at the topic level. Unfortunately, pHDP loses the non-parametric feature of HDP, i.e., the number of topics in pHDP is predetermined and fixed. Furthermore, under the bootstrap structure of pHDP, the topic-indiscriminate words are of high probabilities to be assigned to different topics, resulting in poor qualities of the extracted topics. To achieve parallelism without sacrificing the non-parametric feature of HDP, in addition to improve the quality of extracted topics, we propose a parallel dynamic topic model by developing an adjustment mechanism of evolving topics and reducing the sampling probabilities of topic-indiscriminate words. Both supervised and unsupervised experiments on benchmark datasets show the competitive performance of our model.

The amount of digital text-based consumer review data has increased dramatically and there exist many machine learning approaches for automated text-based sentiment analysis. Marketing researchers have employed various methods for analyzing text reviews but lack a comprehensive comparison of their performance to guide method selection in future applications. We focus on the fundamental relationship between a consumer’s overall empirical evaluation, and the text-based explanation of their evaluation. We study the empirical tradeoff between predictive and diagnostic abilities, in applying various methods to estimate this fundamental relationship. We incorporate methods previously employed in the marketing literature, and methods that are so far less common in the marketing literature. For generalizability, we analyze 25,119 products in nine product categories, and 260,489 reviews across five review platforms. We find that neural network-based machine learning methods, in particular pre-trained versions, offer the most accurate predictions, while topic models such as Latent Dirichlet Allocation offer deeper diagnostics. However, neural network models are not suited for diagnostic purposes and topic models are ill equipped for making predictions. Consequently, future selection of methods to process text reviews is likely to be based on analysts’ goals of prediction versus diagnostics.

Sentiment analysis systems can collect and automatically structure unstructured information by collecting public views of services, products, policy, brands, etc. This information is of major value in the fields of marketing analysis, public relations, product reviews, net promoter evaluations, customer feedback and client reviews. Literary works, on the other hand, are less susceptible to computational analysis because there are no immediate commercial incentives. However, similar techniques can be used to evaluate literary work, comprehend the underlying social network, and obtain or validate literary work. This project is about analyzing the book's characters and predicting their characteristics and relationships with one another. A lot of human effort is expended during the adaptation of a novel/book in any form, which is inconvenient and undesirable. Furthermore, the human brain has a tendency to overlook a number of minor details about the events/characters in the book. The scenario described above can frequently result in inaccuracies in the adaptation's plot. As a result, the project is an innovation that aims to aid in the easy and accurate adaptation of a book, making the process much simpler and precise. Machine learning (supervised and unattended) and lexical approaches include the current sentiment analysis techniques. The model's goal is to scan the massive amounts of text in the book. Following digitization, the model will display interesting ideas derived from the given book using a combination of natural language processing, feelings and emotions analysis, and social network analysis methodology.

Generating explanations within a local and model-agnostic explanation scenario for text classification is often accompanied by a local approximation task. In order to create a local neighborhood for a document, whose classification shall be explained, sampling techniques are used that most often treat the according features at least semantically independent from each other. Hence, contextual as well as semantic information is lost and therefore cannot be used to update a human’s mental model within the according explanation task. In case of dependent features, such explanation techniques are prone to extrapolation to feature areas with low data density, therefore causing misleading interpretations. Additionally, the ”the whole is greater than the sum of its parts” phenomenon is disregarded when using explanations that treat the according words independently from each other. In this paper, an architecture named CaSE is proposed that either uses Semantic Feature Arrangements or Semantic Interrogations to overcome these drawbacks. Combined with a modified version of Local interpretable model-agnostic explanations (LIME), a state of the art local explanation framework, it is capable of generating meaningful and coherent explanations. The approach utilizes contextual and semantic knowledge from unsupervised topic models in order to enable realistic and semantic sampling and based on that generate understandable explanations for any text classifier. The key concepts of CaSE that are deemed essential for providing humans with high quality explanations are derived from findings of psychology. In a nutshell, CaSE shall enable Semantic Alignment between humans and machines and thus further improve the basis for Interactive Machine Learning. An extensive experimental validation of CaSE is conducted, showing its effectiveness by generating reliable and meaningful explanations whose elements are made of contextually coherent words and therefore are suitable to update human mental models in an appropriate way. In the course of a quantitative analysis, the proposed architecture is evaluated w.r.t. a consistency property and to Local Fidelity of the resulting explanation models. According to that, CaSE generates more realistic explanation models leading to higher Local Fidelity compared to LIME.

The agricultural futures prices are generally considered difficult to forecast because the causes of fluctuations are incredibly complicated. We propose a text-based forecasting framework, which can effectively identify and quantify factors affecting agricultural futures based on massive online news headlines. A comprehensive list of influential factors can be formed using a text mining method called topic modeling. A new sentiment-analysis-based way is designed to quantify the factors such as the weather and policies that are important yet difficult to quantify. The proposed framework is empirically tested at forecasting soybean futures prices in the Chinese market. Testing was based on 9715 online news headlines from July 19, 2012 to July 9, 2018. The results show that the identified influential factors and sentiment-based variables are effective, and the proposed framework performs significantly better in medium-term and long-term forecasting than the benchmark model.

Learning hidden topics from data streams has become absolutely necessary but posed challenging problems such as concept drift as well as short and noisy data. Using prior knowledge to enrich a topic model is one of potential solutions to cope with these challenges. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g. Word2vec) is very valuable and useful to help topic models work better. However, in a streaming environment where data arrives continually and infinitely, existing studies are limited to exploiting these resources effectively. Especially, a knowledge graph, that contains meaningful word relations, is ignored. In this paper, to aim at exploiting a knowledge graph effectively, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both a human knowledge graph (Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift. The implementation of GCTM is available at \url{https://github.com/bachtranxuan/GCTM.git}.

Hyperspectral unmixing (HU) has been one of the hot spots in hyperspectral remote sensing research and has great potential in many applications. In recent years, the employment of the probabilistic topic model to mine latent topics in hyperspectral images has been an effective way for spectral unmixing. However, these methods fail to fully exploit the potential of topic models in uncovering image semantics and need extra sparsity constraints, which greatly increases the complexity of the model. In addition, the spatial information which can provide the correlation of features in adjacent pixels is usually ignored in topic model-based HU. To solve these problems, a spectral-spatial unmixing framework of hyperspectral imagery based on a sparse topic relaxation-clustering model (S³TRM) is proposed. In S³TRM, the topic model combined with implied sparse prior constraints are introduced, and the sparse characteristics of S³TRM are used to capture the semantic representation of the spectrum. With the proposed relaxation-clustering strategy, multiple possible spectral representations of features can be obtained, which further alleviates the influence caused by endmember variability. Group clustering is used to locate the endmember quickly and accurately. Moreover, superpixel segmentation is considered to supplement the spatial distribution information of features, thereby improving the fractional abundance. Experiments on a synthetic dataset and three well-known real hyperspectral datasets confirm the excellent performance of the proposed framework in both qualitative assessment and quantitative evaluation, compared with the other state-of-the-art methods.

Blood pressure (BP) is an important indicator of an individual's health status and is closely related to daily behaviors. Thus, a continuous daily measurement of BP is critical for hypertension control. To assist continuous measurement, BP prediction based on non-physiological data (ubiquitous mobile phone data) was studied in the research. An algorithm was proposed that predicts BP based on patients’ daily routine, which includes activities such as sleep, work, and commuting. The aim of the research is to provide insight into the application of mobile data in telemonitoring and the continuous unobtrusive daily measurement of BP. A half-year data set from October 2017 of 320 individuals, including telecom data and BP measurement data, was analyzed. Two hierarchical Bayesian topic models were used to extract individuals’ location-driven daily routine patterns (topics) and calculate probabilities among these topics from their day-level mobile trajectories. Based on the topic probability distribution and patients’ contextual data, their BP were predicted using different models. The prediction model comparison shows that the long short-term memory (LSTM) method exceeds others when the data has a high dependency. Otherwise, the Random Forest regression model outperforms the LSTM method. Also, the experimental results validate the effectiveness of the topics in BP prediction.

Sparse and short news headlines can be arbitrary, noisy, and ambiguous, making it difficult for classic topic model LDA (latent Dirichlet allocation) designed for accommodating long text to discover knowledge from them. Nonetheless, some of the existing research about text-based crude oil forecasting employs LDA to explore topics from news headlines, resulting in a mismatch between the short text and the topic model and further affecting the forecasting performance. Exploiting advanced and appropriate methods to construct high-quality features from news headlines becomes crucial in crude oil forecasting. This paper introduces two novel indicators of topic and sentiment for the short and sparse text data to tackle this issue. Empirical experiments show that AdaBoost.RT with our proposed text indicators, with a more comprehensive view and characterization of the short and sparse text data, outperforms the other benchmarks. Another significant merit is that our method also yields good forecasting performance when applied to other futures commodities.

This study attempts to explain the interplay between digital technologies and venture creation using structural topic modeling (STM), an automatic text analysis method, to detect latent topics related to digital technologies from large texts. The profile data of 133,344 U.S. companies from CrunchBase are analyzed. The data provide detailed descriptions in text about each company: what each company does. STM enables identification of topics related to digital technologies from the large company profile data. The findings identify those digital technologies that appear to have played the biggest role in the landscape of venture activities in the United States. The study also presents findings showing the temporal changes in such ``digital'' topics and the association of those topics with different industry sectors. The proposed analytical framework and the findings make contributions to the growing discussion about the intersection between digital technologies and new business creation.

International audience; We introduce a deep latent recommender system named deepLTRS in order to provide users with high quality recommendations based on observed user ratings \textit{and} texts of product reviews. The underlying motivation is that, when a user scores only a few products, the texts used in the reviews represent a significant source of information, thereby enhancing the predictive ability of the model. Our approach adopts a variational auto-encoder (VAE) architecture as a deep generative latent model for an ordinal matrix encoding ratings and a document-term matrix encoding the reviews. Taking into account both matrices as model inputs, deepLTRS uses a neural network to capture the relationship between latent factors and latent topics. Moreover, a user-majoring encoder and a product-majoring encoder are constructed to jointly capture user and product preferences. Due to the specificity of the model structure, an original row-column alternated mini-batch optimization algorithm is proposed to deal with user-product dependencies and computational burden. Numerical experiments on simulated and real-world data sets demonstrate that deepLTRS outperforms the state-of-the-art, in particular in context of extreme data sparsity.

Researchers and practitioners have long realized the importance of acquiring innovative knowledge from online innovation communities. However, it has still been a challenge to identify valuable information of interest from unstructured and noisy user-generated text. The extant research trying to solve this problem is limited by the generalizability issue and the lack of topic-level investigation. In this article, we proposed an automatic hybrid model to simultaneously extract knowledge topics and knowledge structures from free text in online innovation communities. By employing a topic model, one important contribution of this article is the elimination of manual intervention and addressing of the generalizability issue faced by existing supervised approaches. Another major contribution of this article is the design of a novel solution for knowledge structure construction by combining a topic model with association analysis; this solution addresses the weakness of the traditional topic model. We evaluate the validity of the proposed hybrid model in experiments with a real-world dataset. The results show that by providing a new angle for innovative knowledge acquisition and management, this proposed method is effective in extracting meaningful innovative knowledge and in uncovering knowledge structures.

PurposeDigital voice-of-customer (digital VoC) analysis is gaining much attention in the field of quality management. Digital VoC can be a great source of knowledge about customer needs, habits and expectations. To this end, the most popular approach is based on the application of text mining algorithms named topic modelling. These algorithms can identify latent topics discussed within digital VoC and categorise each source (e.g. each review) based on its content. This paper aims to propose a structured procedure for validating the results produced by topic modelling algorithms.Design/methodology/approachThe proposed procedure compares, on random samples, the results produced by topic modelling algorithms with those generated by human evaluators. The use of specific metrics allows to make a comparison between the two approaches and to provide a preliminary empirical validation.FindingsThe proposed procedure can address users of topic modelling algorithms in validating the obtained results. An application case study related to some car-sharing services supports the description.Originality/valueDespite the vast success of topic modelling-based approaches, metrics and procedures to validate the obtained results are still lacking. This paper provides a first practical and structured validation procedure specifically employed for quality-related applications.

Introducción Amelia SANZ CABRERIZO

PurposeThis study aims to explore knowledge structure and research trends in the domain of digital humanities (DH) in the recent decade. The study identified prevailing topics and then, analyzed trends of such topics over time in the DH field.Design/methodology/approachResearch bibliographic data in the area of DH were collected from scholarly databases. Multiple text mining techniques were used to identify prevailing research topics and trends, such as keyword co-occurrences, bigram analysis, structural topic models and bi-term topic models.FindingsTerm-level analysis revealed that cultural heritage, geographic information, semantic web, linked data and digital media were among the most popular topics in the recent decade. Structural topic models identified that linked open data, text mining, semantic web and ontology, text digitization and social network analysis received increased attention in the DH field.Originality/valueThis study applied existent text mining techniques to understand the research domain in DH. The study collected a large set of bibliographic text, representing the area of DH from multiple academic databases and explored research trends based on structural topic models.

We analyze text data in all the articles published in the top five (T5) economics journals between 2002 and 2019 in order to find gender differences in their research approach. We implement an unsupervised machine learning algorithm: the structural topic model (STM), so as to incorporate gender document-level meta-data into a probabilistic text model. This algorithm characterizes jointly the set of latent topics that best fits our data (the set of abstracts) and how the documents/abstracts are allocated to each topic. Latent topics are mixtures over words where each word has a probability of belonging to a topic after controlling by journal name and publication year (the meta-data). Thus, the topics may capture research fields but also other more subtle characteristics related to the way in which the articles are written. We find that females are unevenly distributed over the estimated latent topics. This and other findings rely on “automatically” generated built-in data given the contents in the abstracts of the articles in the T5 journals, without any arbitrary allocation of texts to particular categories (as JEL codes, or research areas). José Ignacio Conde-Ruiz acknowledges the Spanish Ministry of Science and Innovation for financial support through the project PID2019-105499GB-I00. Manu García and Luis Puch acknowledge the support through the project PID2019-107161GB-C32. Juan-José Ganuza gratefully acknowledges the financial support from the Spanish Agencia Estatal de Investigación, through the Severo Ochoa Programme for Centres of Excellence in R&D (CEX2019-000915-S) and the Spanish Ministry of Science and Innovation through Project PID2020-115044GB-I00.

Recently, topic modeling has been upgraded by neural variational inference, which simultaneously allows the model structures deeper and proposes efficient update rules with the reparameterization trick. We formally call this recent new art as neural topic model. In this paper, we investigate a problem of neural topic models, where they formulate topic embeddings and measure the word weights within topics by linear transformation between topic and word embeddings, resulting in redundant and inaccurate topic representations. To solve this problem, we propose a novel neural topic model, namely G enerative M odel with N onlinear N eural T opics (GMnnt). The insight of GMnnt is to replace the topic embeddings with neural networks of topics, named neural topic, so as to capture nonlinear relationships between words in the embedding space, enabling to induce more accurate topic representations. We derive the inference process of GMnnt under the framework of neural variational inference. Extensive empirical studies have been conducted on several widely used collections of documents, including datasets of both short texts and normal long texts. The experimental results validate that GMnnt can output more semantically coherent topics compared with traditional topic models and neural topic models.

Introduction. Fertilizer is one of the most important production factors in the world of agriculture. It is crucial to increase the capacity of technology related to fertilizers. Analysis of patent documents can be one way to analyze technological developments, especially fertilizers. Data Collection Methods. The data used in this research are metadata, especially the title and abstract of a patent document in Indonesia. With the keyword "fertilizer," Patent metadata was processed in the 1945-2017 period. Data Analysis. The LDA model can provide a reasonable interpretation regarding topic modeling based on text data. Results and Discussion. The results find that degree of the patent title is better than the abstract of the patent. The LDA approach can adequately separate the topics of fertilizer patent technology so that it does not have multiple interpretations. Conclusion. Based on the findings, there are nine essential topics in the development of fertilizer technology. There is a phenomenon of the lack of technology collaboration between IPC technology sections.

Due to the overload of published scientific articles, citation recommendation has long been a critical research problem for automatically recommending the most relevant citations of given articles. Relational topic models (RTMs) have shown promise on citation prediction via joint modeling of document contents and citations. However, existing RTMs can only capture pairwise or direct (first-order) citation relationships among documents. The indirect (high-order) citation links have been explored in graph neural network–based methods, but these methods suffer from the well-known explainability problem. In this article, we propose a model called Graph Neural Collaborative Topic Model that takes advantage of both relational topic models and graph neural networks to capture high-order citation relationships and to have higher explainability due to the latent topic semantic structure. Experiments on three real-world citation datasets show that our model outperforms several competitive baseline methods on citation recommendation. In addition, we show that our approach can learn better topics than the existing approaches. The recommendation results can be well explained by the underlying topics.

We propose a new approach for tracing value change. Value change may lead to a mismatch between current value priorities in society and the values for which technologies were designed in the past, such as energy technologies based on fossil fuels, which were developed when sustainability was not considered a very important value. Better anticipating value change is essential to avoid a lack of social acceptance and moral acceptability of technologies. While value change can be studied historically and qualitatively, we propose a more quantitative approach that uses large text corpora. It uses probabilistic topic models, which allow us to trace (new) values that are (still) latent. We demonstrate the approach for five types of value change in technology. Our approach is useful for testing hypotheses about value change, such as verifying whether value change has occurred and identifying patterns of value change. The approach can be used to trace value change for various technologies and text corpora, including scientific articles, newspaper articles, and policy documents.

Purpose This paper aims to propose an automated mobile app labeling framework based on a novel app classification scheme that is aligned with users’ primary motivations for using smartphones. The study addresses the gaps in incorporating the needs of users and other context information in app classification as well as recommendation systems. Design/methodology/approach Based on a corpus of mobile app descriptions collected from Google Play store, this study applies extensive text analytics and topic modeling procedures to profile mobile apps within the categories of the classification scheme. Sufficient number of representative and labeled app descriptions are then used to train a classifier using machine learning algorithms, such as rule-based, decision tree and artificial neural network. Findings Experimental results of the classifiers show high accuracy in automatically labeling new apps based on their descriptions. The accuracy of the classification results suggests a feasible direction in facilitating app searching and retrieval in different Web-based usage environments. Research limitations/implications As a common challenge in textual data projects, the problem of data size and data quality issues exists throughout the multiple phases of experiments. Future research will extend the data collection scope in many aspects to address the issues that constrained the current experiments. Practical implications These empirical experiments demonstrate the feasibility of textual data analysis in profiling apps and user context information. This study also benefits app developers by improving app descriptions through a better understanding of user needs and context information. Finally, the classification framework can also guide practitioners in customizing products and services beyond mobile apps where context information and user needs play an important role. Social implications Given the widespread usage and applications of smartphones today, the proposed app classification framework will have broader implications to different Web-based application environments. Originality/value While there have been other classification approaches in the literature, to the best of the authors’ knowledge, this framework is the first study on building an automated app labeling framework based on primary motivations of smartphone usage.