Search

Cycling is an eco-friendly and sustainable mode of transportation. Despite its benefits, the cyclists’ risk of collision is still high when interacting with other road users. This study analyzed self-reported near-miss and collision event descriptions for the United States provided by the crowdsourcing platform, BikeMaps.org. Innovative and efficient analytic methods are needed to generate useful information from unstructured textual data sources in the transportation domain. In this study, explorative text mining, topic modeling, and machine learning are utilized to gain insights from the unstructured textual descriptions of crowdsourced near-miss and collision events. The approaches are used to unveil prevalent words and word associations for near-miss and collision events. Structural Topic Modeling (STM) is deployed to autogenerate latent themes or topics from the event descriptions. The generated topic proportions are used as input in Artificial Neural Networks (ANN) to estimate the cyclist’s propensity to a collision. It was found that cyclists had a higher propensity to a collision in topics that articulated vehicle encroachment to the bike lane, on-street parking close or into the bike lane resulting in dooring, and drivers’ violations at the crosswalk. The results and methodology used in this study can assist engineers, policymakers, and law enforcement officers to proactively reduce potential cyclist collisions, prioritizing areas where cyclist safety improvements are needed, and ultimately promoting bicycle ridership in our communities.

For multimodal representation learning, traditional black-box approaches often fall short of extracting interpretable multilayer hidden structures, which contribute to visualize the connections between different modalities at multiple semantic levels. To extract interpretable multimodal latent representations and visualize the hierarchial semantic relationships between different modalities, based on deep topic models, we develop a novel multimodal Poisson gamma belief network (mPGBN) that tightly couples the observations of different modalities via imposing sparse connections between their modality-specific hidden layers. To alleviate the time-consuming Gibbs sampler adopted by traditional topic models in the testing stage, we construct a Weibull-based variational inference network (encoder) to directly map the observations to their latent representations, and further combine it with the mPGBN (decoder), resulting in a novel multimodal Weibull variational autoencoder (MWVAE), which is fast in out-of-sample prediction and can handle large-scale multimodal datasets. Qualitative evaluations on bimodal data consisting of image-text pairs show that the developed MWVAE can successfully extract expressive multimodal latent representations for downstream tasks like missing modality imputation and multimodal retrieval. Further extensive quantitative results demonstrate that both MWVAE and its supervised extension sMWVAE achieve state-of-the-art performance on various multimodal benchmarks.

One of the most significant differences of M5 over previous forecasting competitions is that it was held on Kaggle, an online platform for data scientists and machine learning practitioners. Kaggle provides a gathering place, or virtual community, for web users who are interested in the M5 competition. Users can share code, models, features, and loss functions through online notebooks and discussion forums. Here, we study the social influence of this virtual community on user behavior in the M5 competition. We first research the content of the M5 virtual community by topic modeling and trend analysis. Further, we perform social media analysis to identify the potential relationship network of the virtual community. We study the roles and characteristics of some key participants who promoted the diffusion of information within the M5 virtual community. Overall, this study provides in-depth insights into the mechanism of the virtual community’s influence on the participants and has potential implications for future online competitions.

The demand for Natural Language Processing has been thriving rapidly due to the various emerging Internet services such as social networks, e-commerce, intelligent assistant, etc. The large volume of natural language data and the new market requirements have brought significant challenges and opportunities for academia and industry. The challenges include (1) efficiently processing a massive volume of data and devising a highly scalable architecture, (2) extending the horizon of the current natural language processing to tackle problems that cannot be handled well. This thesis demonstrates several advanced techniques to tackle the challenges by boosting scalability and efficiency or extending the horizon of existing core NLP techniques on different granularities. This thesis introduces (i) a neural-based lexical simplification algorithm that utilizes contextualized word embedding to capture the meaning more accurately, (ii) a sample-efficient algorithm that solves extreme large-output classification for contrastive representation learning. (iii) a scalable and efficient algorithm for the probabilistic topic model LDA.First, we propose amplified negative sampling and rigorous mathematical analysis of the famous negative sampling method for the word level distributed representations. Specifically, we distinguish the impact of various loss functions for the contrastive representation learning process. Our theoretical analysis provides a better understanding of a few empirical observations that are well known among practitioners when negative sampling is employed. We also propose the amplified negative sampling based on our analysis. It's a simple yet efficient method that can effectively improve the negative sampling efficiency and boost the downstream tasks' performance.Second, we propose the neural simplicity ranking (NSR) model that utilizes the contextualized word representations in an unconventional way for lexical simplification for the sentence level tasks. This model creatively uses the contextualized word embeddings beyond its original masked language model (MLM) task to identify the relatedness among candidate words and the original words. By redesigning the feature set for the simplicity measure and the ranking scheme, the NSR model achieves new state-of-the-art performance.Finally, we propose sd-LDA for the paragraph-level tasks, a scalable disk-based LDA model with sparse-prior sampling. The sd-LDA brings a highly optimized disk-based design that reduces the memory consumption by two orders of magnitude, making it possible to do large topic analysis on mobile devices. A sparse prior sampling technique is introduced in this work to accelerate the processing time. The sd-LDA algorithm can effectively reduce the memory consumption and processing time at the same time without loss of performance.

Personalized venues recommendation is essential to help people find attractive venue to visit as growth of location-based social networks. Existing approaches never distinguish user individual interests from her social preferences, which leads to a bottleneck of modeling user check-in behaviors accurately. In this paper, we find the differences between user interests and her social preferences clearly and investigate the time law of user check-in behaviors in depth. Consequently, we propose a social-period-aware topic model (SPATM) to learn the influence weights of both user interests and her social preferences on making-decision for each check-in time automatically. Especially, we model latent topic by leveraging smaller size of dynamic activities instead of static categories, which can alleviate the data sparsity problem by using more co-occurrent activities information. Moreover, our approach can automatically judge whether a user's social preference is periodic or aperiodic and learn the periodicity of periodic one. Furthermore, the Alias Sampling based training approach is introduced to improve sampling efficiency. The results demonstrate our proposed model is effective and outperforms the state-of-the-art approaches in terms of effectiveness and efficiency. Besides, SPATM can learn semantically coherent latent topics and geographically dispersed latent social topics which are useful to explain recommendation.

Topic modeling is a significant branch of natural language processing and machine learning focused on inferring the generative process of text. Traditionally, algorithms for estimating topic models have relied on Bayesian inference and Gibbs sampling. This paper proposes a novel “acceptable set” framework for formulating topic modeling problems inspired by ideas from discrete component analysis and data-driven robust optimization. Our approach not only simplifies the design and inference of topic models, but also allows for extensions and generalizations that are challenging to integrate into traditional approaches. Different restrictions (e.g., sparsity) and assumptions (e.g., alternative generative processes) can be easily incorporated into our formulations through additional or modified constraints. Our formulations also naturally control a widely used metric of model quality, perplexity. We adapt state-of-the-art stochastic gradient methods to find good local optima for the optimization formulations. The algorithms are efficient, scaling to realistic problem sizes with runtimes comparable to existing methods. Through extensive computational experiments, we show that our methods have improved solution quality compared to baseline methods and reconstruct more reliably the underlying generative models. Our framework overcomes known vulnerabilities of traditional topic modeling algorithms: our methods are effective in low-data settings, register good out-of-sample performance, and perform well for a variety of initial assumptions on input parameter values.

Currently, the world is in the midst of a severe global pandemic, which has affected all aspects of people's lives. As a result, there is a deluge of COVID-related digital media articles published in the United States, due to the disparate effects of the pandemic. This large volume of information is difficult to consume by the audience in a reasonable amount of time. In this paper, we develop a Natural Language Processing (NLP) pipeline that is capable of automatically distilling various digital articles into manageable pieces of information, while also modelling the progression topics discussed over time in order to aid readers in rapidly gaining holistic perspectives on pressing issues (i.e., the COVID-19 pandemic) from a diverse array of sources. We achieve these goals by first collecting a large corpus of COVID-related articles during the onset of the pandemic. After, we apply unsupervised and semi-supervised learning procedures to summarize articles, then cluster them based on their similarities using the community detection methods. Next, we identify the topic of each cluster of articles using the BART algorithm. Finally, we provide a detailed digital media analysis based on the NLP-pipeline outputs and show how the conversation surrounding COVID-19 evolved over time.

Deep learning models have achieved high performance across different domains, such as medical decision-making, autonomous vehicles, decision support systems, among many others. However, despite this success, the inner mechanisms of these models are opaque because their internal representations are too complex for a human to understand. This opacity makes it hard to understand the how or the why of the predictions of deep learning models. There has been a growing interest in model-agnostic methods that make deep learning models more transparent and explainable to humans. Some researchers recently argued that for a machine to achieve human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals. This paper presents an in-depth systematic review of the diverse existing literature on counterfactuals and causability for explainable artificial intelligence (AI). We performed a Latent Dirichlet topic modeling analysis (LDA) under a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework to find the most relevant literature articles. This analysis yielded a novel taxonomy that considers the grounding theories of the surveyed algorithms, together with their underlying properties and applications to real-world data. Our research suggests that current model-agnostic counterfactual algorithms for explainable AI are not grounded on a causal theoretical formalism and, consequently, cannot promote causability to a human decision-maker. Furthermore, our findings suggest that the explanations derived from popular algorithms in the literature provide spurious correlations rather than cause/effects relationships, leading to sub-optimal, erroneous, or even biased explanations. Thus, this paper also advances the literature with new directions and challenges on promoting causability in model-agnostic approaches for explainable AI.

Document clustering allows a user to group semantically similar documents. It has been an interesting research area for the past many years and various methods and techniques have been developed. However, the research has primarily been limited to English and other high resource languages. For low-resource languages, such as Urdu, the area of document clustering is open to contributions. This work presents an experimental evaluation of clustering techniques on Urdu tweets. It is a challenging task to semantically cluster tweets due to their very short length. In this paper, various features, including sentence and phrase-level embeddings, TF-IDF features and document embeddings are extracted from tweets and clustering is performed using three different algorithms: K-Means, Bisecting K-Means, and Affinity Propagation algorithms. Furthermore, a comparison is performed with the traditional topic modeling approach. The results indicate that the TF-IDF features combined with the K-means clustering algorithm outperformed the adopted clustering techniques.

Probabilistic topic models, such as latent Dirichlet allocation (LDA), are often used to discover hidden semantic structure of a collection of documents. In recent years, various inference algorithms have been developed to cope with learning of topic models, among which Gibbs sampling methods remain a popular choice. In this paper, we aim to improve the inference of topic models based on the Gibbs sampling framework. We extend a state augmentation based Gibbs sampling method by maximizing the replications of latent states, and propose a new generic deterministic inference method, named maximal latent state replication (MAX), for learning of a family of probabilistic topic models. One key benefit of the proposed method lies in the deterministic nature for inference, which may help to improve its running efficiency as well as predictive perplexity. We have conducted extensive experiments on real-life publicly available datasets, and the results have validated that our proposed method MAX significantly outperforms state-of-the-art baselines for inference of existing well-known topic models.

The widespread use of positioning devices has given rise to many trajectories, with each having three explicit attributes: user ID, location ID, and time-stamp and an implicit attribute: activity type (akin to “topic” in text mining). To model these trajectories, existing works learn different attribute representations by either introducing latent activity types based on topic models or transforming the location and time context into a low-dimensional space via embedding techniques. In this paper, we propose a holistic approach named Human Mobility Representation Model (HMRM) to simultaneously produce the vector representations of all four (explicit and implicit) attributes. The merits of HMRM lie in that: (1) it models the latent activity types and learns trajectory attribute embeddings in an integrated manner, and (2) it connects the activity-related distributions and these attributes embeddings by adding a newly designed collaborative learning component, and makes them mutually exchanged to take the best of both worlds. We apply HMRM to both unsupervised and supervised tasks including two activity evaluation tasks and two embedding evaluation tasks, on two real check-in datasets collected from Foursquare. Experimental results show that HMRM could not only improve the performance of capturing latent activity types, but also learn better trajectory embeddings.

Firms apply strategic foresight in technology and innovation management to detect discontinuous changes early, to assess their expected consequences, and to develop a future course of action enabling superior company performance. For this purpose, an ever-increasing amount of data has to be collected, analyzed, and interpreted. Still, a major part of these activities is performed manually, which requires high investments in various resources. To support these processes more efficiently, this article presents an artificial-intelligence-based data mining model that helps firms spot emerging topics and trends at a higher level of automation than before. Its modular structure consists of components for query generation, data collection, data preprocessing, topic modeling, topic analysis, and visualization, combined in such a way that only a minimum amount of manual effort is required during its initial set up. The approach also incorporates self-adaptive capabilities, allowing the model to automatically update itself once new data has become available. The model parameterization is based on latest research in this area, and its threshold parameter is learnt during supervised training using a training data set. We have applied our model to an independent test data set to verify its effectiveness as an early warning system. By means of a retrospective analysis, we show in three case studies that our model is able to identify emerging technologies prior to their first publication in the Gartner Hype Cycle for Emerging Technologies. Based on our findings, we derive both theoretical and practical implications for the technology and innovation management of firms, and we suggest future research opportunities to further advance this field.

Recently, advances in neural network approaches have achieved many successes in both sentiment classification and probabilistic topic modelling. On the one hand, latent topics derived from the global context of documents could be helpful in capturing more accurate word semantics and hence could potentially improve the sentiment classification accuracy. On the other hand, the word-level attention vectors obtained during the learning of sentiment classifiers could carry word-level polarity information and can be used to guide the discovery of topics in topic modelling. This paper proposes a multi-task learning framework which jointly learns a sentiment classifier and a topic model by making the word-level latent topic distributions in the topic model to be similar to the word-level attention vectors in the classifier through mutual learning. Experimental results on the Yelp and IMDB datasets verify the superior performance of the proposed framework over strong baselines on both sentiment classification accuracy and topic modelling evaluation results including perplexity and topic coherence measures. The proposed framework also extracts more interpretable topics compared to other conventional topic models and neural topic models.

We introduce and assess the use of supervised learning in cross-domain topic classification. In this approach, an algorithm learns to classify topics in a labeled source corpus and then extrapolates topics in an unlabeled target corpus from another domain. The ability to use existing training data makes this method significantly more efficient than within-domain supervised learning. It also has three advantages over unsupervised topic models: the method can be more specifically targeted to a research question and the resulting topics are easier to validate and interpret. We demonstrate the method using the case of labeled party platforms (source corpus) and unlabeled parliamentary speeches (target corpus). In addition to the standard within-domain error metrics, we further validate the cross-domain performance by labeling a subset of target-corpus documents. We find that the classifier accurately assigns topics in the parliamentary speeches, although accuracy varies substantially by topic. We also propose tools diagnosing cross-domain classification. To illustrate the usefulness of the method, we present two case studies on how electoral rules and the gender of parliamentarians influence the choice of speech topics., Political Analysis, 31 (1), ISSN:1047-1987, ISSN:1476-4989

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm., arXiv admin note: text overlap with arXiv:1808.02215 by other authors

The parallel Hierarchical Dirichlet Process (pHDP) is an efficient topic model which explores the equivalence of the generation process between Hierarchical Dirichlet Process (HDP) and Gamma-Gamma-Poisson Process (G2PP), in order to achieve parallelism at the topic level. Unfortunately, pHDP loses the non-parametric feature of HDP, i.e., the number of topics in pHDP is predetermined and fixed. Furthermore, under the bootstrap structure of pHDP, the topic-indiscriminate words are of high probabilities to be assigned to different topics, resulting in poor qualities of the extracted topics. To achieve parallelism without sacrificing the non-parametric feature of HDP, in addition to improve the quality of extracted topics, we propose a parallel dynamic topic model by developing an adjustment mechanism of evolving topics and reducing the sampling probabilities of topic-indiscriminate words. Both supervised and unsupervised experiments on benchmark datasets show the competitive performance of our model.

The amount of digital text-based consumer review data has increased dramatically and there exist many machine learning approaches for automated text-based sentiment analysis. Marketing researchers have employed various methods for analyzing text reviews but lack a comprehensive comparison of their performance to guide method selection in future applications. We focus on the fundamental relationship between a consumer’s overall empirical evaluation, and the text-based explanation of their evaluation. We study the empirical tradeoff between predictive and diagnostic abilities, in applying various methods to estimate this fundamental relationship. We incorporate methods previously employed in the marketing literature, and methods that are so far less common in the marketing literature. For generalizability, we analyze 25,119 products in nine product categories, and 260,489 reviews across five review platforms. We find that neural network-based machine learning methods, in particular pre-trained versions, offer the most accurate predictions, while topic models such as Latent Dirichlet Allocation offer deeper diagnostics. However, neural network models are not suited for diagnostic purposes and topic models are ill equipped for making predictions. Consequently, future selection of methods to process text reviews is likely to be based on analysts’ goals of prediction versus diagnostics.

Generating explanations within a local and model-agnostic explanation scenario for text classification is often accompanied by a local approximation task. In order to create a local neighborhood for a document, whose classification shall be explained, sampling techniques are used that most often treat the according features at least semantically independent from each other. Hence, contextual as well as semantic information is lost and therefore cannot be used to update a human’s mental model within the according explanation task. In case of dependent features, such explanation techniques are prone to extrapolation to feature areas with low data density, therefore causing misleading interpretations. Additionally, the ”the whole is greater than the sum of its parts” phenomenon is disregarded when using explanations that treat the according words independently from each other. In this paper, an architecture named CaSE is proposed that either uses Semantic Feature Arrangements or Semantic Interrogations to overcome these drawbacks. Combined with a modified version of Local interpretable model-agnostic explanations (LIME), a state of the art local explanation framework, it is capable of generating meaningful and coherent explanations. The approach utilizes contextual and semantic knowledge from unsupervised topic models in order to enable realistic and semantic sampling and based on that generate understandable explanations for any text classifier. The key concepts of CaSE that are deemed essential for providing humans with high quality explanations are derived from findings of psychology. In a nutshell, CaSE shall enable Semantic Alignment between humans and machines and thus further improve the basis for Interactive Machine Learning. An extensive experimental validation of CaSE is conducted, showing its effectiveness by generating reliable and meaningful explanations whose elements are made of contextually coherent words and therefore are suitable to update human mental models in an appropriate way. In the course of a quantitative analysis, the proposed architecture is evaluated w.r.t. a consistency property and to Local Fidelity of the resulting explanation models. According to that, CaSE generates more realistic explanation models leading to higher Local Fidelity compared to LIME.

Learning hidden topics from data streams has become absolutely necessary but posed challenging problems such as concept drift as well as short and noisy data. Using prior knowledge to enrich a topic model is one of potential solutions to cope with these challenges. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g. Word2vec) is very valuable and useful to help topic models work better. However, in a streaming environment where data arrives continually and infinitely, existing studies are limited to exploiting these resources effectively. Especially, a knowledge graph, that contains meaningful word relations, is ignored. In this paper, to aim at exploiting a knowledge graph effectively, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both a human knowledge graph (Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift. The implementation of GCTM is available at \url{https://github.com/bachtranxuan/GCTM.git}.

Hyperspectral unmixing (HU) has been one of the hot spots in hyperspectral remote sensing research and has great potential in many applications. In recent years, the employment of the probabilistic topic model to mine latent topics in hyperspectral images has been an effective way for spectral unmixing. However, these methods fail to fully exploit the potential of topic models in uncovering image semantics and need extra sparsity constraints, which greatly increases the complexity of the model. In addition, the spatial information which can provide the correlation of features in adjacent pixels is usually ignored in topic model-based HU. To solve these problems, a spectral-spatial unmixing framework of hyperspectral imagery based on a sparse topic relaxation-clustering model (S³TRM) is proposed. In S³TRM, the topic model combined with implied sparse prior constraints are introduced, and the sparse characteristics of S³TRM are used to capture the semantic representation of the spectrum. With the proposed relaxation-clustering strategy, multiple possible spectral representations of features can be obtained, which further alleviates the influence caused by endmember variability. Group clustering is used to locate the endmember quickly and accurately. Moreover, superpixel segmentation is considered to supplement the spatial distribution information of features, thereby improving the fractional abundance. Experiments on a synthetic dataset and three well-known real hyperspectral datasets confirm the excellent performance of the proposed framework in both qualitative assessment and quantitative evaluation, compared with the other state-of-the-art methods.

Blood pressure (BP) is an important indicator of an individual's health status and is closely related to daily behaviors. Thus, a continuous daily measurement of BP is critical for hypertension control. To assist continuous measurement, BP prediction based on non-physiological data (ubiquitous mobile phone data) was studied in the research. An algorithm was proposed that predicts BP based on patients’ daily routine, which includes activities such as sleep, work, and commuting. The aim of the research is to provide insight into the application of mobile data in telemonitoring and the continuous unobtrusive daily measurement of BP. A half-year data set from October 2017 of 320 individuals, including telecom data and BP measurement data, was analyzed. Two hierarchical Bayesian topic models were used to extract individuals’ location-driven daily routine patterns (topics) and calculate probabilities among these topics from their day-level mobile trajectories. Based on the topic probability distribution and patients’ contextual data, their BP were predicted using different models. The prediction model comparison shows that the long short-term memory (LSTM) method exceeds others when the data has a high dependency. Otherwise, the Random Forest regression model outperforms the LSTM method. Also, the experimental results validate the effectiveness of the topics in BP prediction.

Sparse and short news headlines can be arbitrary, noisy, and ambiguous, making it difficult for classic topic model LDA (latent Dirichlet allocation) designed for accommodating long text to discover knowledge from them. Nonetheless, some of the existing research about text-based crude oil forecasting employs LDA to explore topics from news headlines, resulting in a mismatch between the short text and the topic model and further affecting the forecasting performance. Exploiting advanced and appropriate methods to construct high-quality features from news headlines becomes crucial in crude oil forecasting. This paper introduces two novel indicators of topic and sentiment for the short and sparse text data to tackle this issue. Empirical experiments show that AdaBoost.RT with our proposed text indicators, with a more comprehensive view and characterization of the short and sparse text data, outperforms the other benchmarks. Another significant merit is that our method also yields good forecasting performance when applied to other futures commodities.

International audience; We introduce a deep latent recommender system named deepLTRS in order to provide users with high quality recommendations based on observed user ratings \textit{and} texts of product reviews. The underlying motivation is that, when a user scores only a few products, the texts used in the reviews represent a significant source of information, thereby enhancing the predictive ability of the model. Our approach adopts a variational auto-encoder (VAE) architecture as a deep generative latent model for an ordinal matrix encoding ratings and a document-term matrix encoding the reviews. Taking into account both matrices as model inputs, deepLTRS uses a neural network to capture the relationship between latent factors and latent topics. Moreover, a user-majoring encoder and a product-majoring encoder are constructed to jointly capture user and product preferences. Due to the specificity of the model structure, an original row-column alternated mini-batch optimization algorithm is proposed to deal with user-product dependencies and computational burden. Numerical experiments on simulated and real-world data sets demonstrate that deepLTRS outperforms the state-of-the-art, in particular in context of extreme data sparsity.

Researchers and practitioners have long realized the importance of acquiring innovative knowledge from online innovation communities. However, it has still been a challenge to identify valuable information of interest from unstructured and noisy user-generated text. The extant research trying to solve this problem is limited by the generalizability issue and the lack of topic-level investigation. In this article, we proposed an automatic hybrid model to simultaneously extract knowledge topics and knowledge structures from free text in online innovation communities. By employing a topic model, one important contribution of this article is the elimination of manual intervention and addressing of the generalizability issue faced by existing supervised approaches. Another major contribution of this article is the design of a novel solution for knowledge structure construction by combining a topic model with association analysis; this solution addresses the weakness of the traditional topic model. We evaluate the validity of the proposed hybrid model in experiments with a real-world dataset. The results show that by providing a new angle for innovative knowledge acquisition and management, this proposed method is effective in extracting meaningful innovative knowledge and in uncovering knowledge structures.

PurposeDigital voice-of-customer (digital VoC) analysis is gaining much attention in the field of quality management. Digital VoC can be a great source of knowledge about customer needs, habits and expectations. To this end, the most popular approach is based on the application of text mining algorithms named topic modelling. These algorithms can identify latent topics discussed within digital VoC and categorise each source (e.g. each review) based on its content. This paper aims to propose a structured procedure for validating the results produced by topic modelling algorithms.Design/methodology/approachThe proposed procedure compares, on random samples, the results produced by topic modelling algorithms with those generated by human evaluators. The use of specific metrics allows to make a comparison between the two approaches and to provide a preliminary empirical validation.FindingsThe proposed procedure can address users of topic modelling algorithms in validating the obtained results. An application case study related to some car-sharing services supports the description.Originality/valueDespite the vast success of topic modelling-based approaches, metrics and procedures to validate the obtained results are still lacking. This paper provides a first practical and structured validation procedure specifically employed for quality-related applications.

Introducción Amelia SANZ CABRERIZO

PurposeThis study aims to explore knowledge structure and research trends in the domain of digital humanities (DH) in the recent decade. The study identified prevailing topics and then, analyzed trends of such topics over time in the DH field.Design/methodology/approachResearch bibliographic data in the area of DH were collected from scholarly databases. Multiple text mining techniques were used to identify prevailing research topics and trends, such as keyword co-occurrences, bigram analysis, structural topic models and bi-term topic models.FindingsTerm-level analysis revealed that cultural heritage, geographic information, semantic web, linked data and digital media were among the most popular topics in the recent decade. Structural topic models identified that linked open data, text mining, semantic web and ontology, text digitization and social network analysis received increased attention in the DH field.Originality/valueThis study applied existent text mining techniques to understand the research domain in DH. The study collected a large set of bibliographic text, representing the area of DH from multiple academic databases and explored research trends based on structural topic models.

We analyze text data in all the articles published in the top five (T5) economics journals between 2002 and 2019 in order to find gender differences in their research approach. We implement an unsupervised machine learning algorithm: the structural topic model (STM), so as to incorporate gender document-level meta-data into a probabilistic text model. This algorithm characterizes jointly the set of latent topics that best fits our data (the set of abstracts) and how the documents/abstracts are allocated to each topic. Latent topics are mixtures over words where each word has a probability of belonging to a topic after controlling by journal name and publication year (the meta-data). Thus, the topics may capture research fields but also other more subtle characteristics related to the way in which the articles are written. We find that females are unevenly distributed over the estimated latent topics. This and other findings rely on “automatically” generated built-in data given the contents in the abstracts of the articles in the T5 journals, without any arbitrary allocation of texts to particular categories (as JEL codes, or research areas). José Ignacio Conde-Ruiz acknowledges the Spanish Ministry of Science and Innovation for financial support through the project PID2019-105499GB-I00. Manu García and Luis Puch acknowledge the support through the project PID2019-107161GB-C32. Juan-José Ganuza gratefully acknowledges the financial support from the Spanish Agencia Estatal de Investigación, through the Severo Ochoa Programme for Centres of Excellence in R&D (CEX2019-000915-S) and the Spanish Ministry of Science and Innovation through Project PID2020-115044GB-I00.

Introduction. Fertilizer is one of the most important production factors in the world of agriculture. It is crucial to increase the capacity of technology related to fertilizers. Analysis of patent documents can be one way to analyze technological developments, especially fertilizers. Data Collection Methods. The data used in this research are metadata, especially the title and abstract of a patent document in Indonesia. With the keyword "fertilizer," Patent metadata was processed in the 1945-2017 period. Data Analysis. The LDA model can provide a reasonable interpretation regarding topic modeling based on text data. Results and Discussion. The results find that degree of the patent title is better than the abstract of the patent. The LDA approach can adequately separate the topics of fertilizer patent technology so that it does not have multiple interpretations. Conclusion. Based on the findings, there are nine essential topics in the development of fertilizer technology. There is a phenomenon of the lack of technology collaboration between IPC technology sections.

We propose a new approach for tracing value change. Value change may lead to a mismatch between current value priorities in society and the values for which technologies were designed in the past, such as energy technologies based on fossil fuels, which were developed when sustainability was not considered a very important value. Better anticipating value change is essential to avoid a lack of social acceptance and moral acceptability of technologies. While value change can be studied historically and qualitatively, we propose a more quantitative approach that uses large text corpora. It uses probabilistic topic models, which allow us to trace (new) values that are (still) latent. We demonstrate the approach for five types of value change in technology. Our approach is useful for testing hypotheses about value change, such as verifying whether value change has occurred and identifying patterns of value change. The approach can be used to trace value change for various technologies and text corpora, including scientific articles, newspaper articles, and policy documents.

With the massive use of electronic gadgets and the developing fame of web-based media, a great deal of text information is being produced at the rate never observed. It is not feasible for people to pursue all information produced and discover what is being investigated in their area of interest. To determine topics in large textual documents Topic modelling is used. Topic Modeling Algorithms (TMA) are Unsupervised Machine Learning approaches which are widely used and have proven to be successful in the area of Aspect-based Opinion Mining to extract ‘latent’ topics, which are aspects of interest. In this paper, the approaches that are widely used for topic modelling are examined and compared to find their importance in detecting topics based on metrics such as perplexity and coherence. As a result, Latent Dirichlet Allocation is a good topic modelling algorithm compared to Latent Semantic Analysis and Hierarchical Dirichlet Process for aspect extraction process in aspect-based opinion mining. Also, we have proposed an unsupervised aspect extraction algorithm based on topic models for Aspect-based Opinion mining.

Learning from past incidents is critical to achieving and maintaining high process safety performance. Accident and incident records provide one way for learning; however, these are usually in the form of unstructured texts, which makes analysis difficult. Recently, text mining methods based on supervised learning have been proposed for analyzing accident reports; however, they require an impractically large number of labeled records as training examples. This paper proposes an automated, semi-supervised, domain-independent approach for analyzing accident reports. Given a set of user-defined classification topics and domain literature such as handbooks, glossaries, and Wikipedia articles, the method can identify domain-specific keywords and group them into topics with minimal expert involvement. These keywords and topics can then be used for various data mining purposes, including classification. The proposed approach is demonstrated using two different case studies across domains: (1) in aviation to identify the stage of flight when an accident occurs, and (2) in the process industry domain to identify the cause of pipeline accidents. The average classification accuracy of the proposed method was 80% which is comparable to that of supervised learning methods. The key benefits of this approach are that it can generate domain-specific predictive models with limited manual intervention.

This study aims to analyze differences in domestic and international disaster research trends. We first performed topic modeling on 20,477 papers published in three domestic and 12 international journals over the last 21 years (2000-2020) and then visualized the trends. Based on the extracted topics and keywords, we analyzed keyword networks using Gephi. Research in domestic journals mainly revolved around natural disasters like earthquakes, fire, and flooding. In contrast, international journals spotlighted policy-based topics such as disaster governance and community resilience. Meanwhile, globally, building and civil engineering research has shrunk in recent five years (we refer to this as a cold topic). On the other hand, in the past five years, fire and flood research has appeared more frequently in domestic journals, while international journals have presented more articles on community resilience, risk perception, and behavior (we refer to this as a hot topic). Results of this research can provide suggestions about the directions domestic disaster research should develop in the future.

The increasing importance of online distribution channels is paralleled by a rising interest in gaining insights into the customer journey and browsing behavior. We evaluate several machine learning methods (latent Dirichlet allocation, correlated topic model, structural topic model, replicated softmax model) with respect to their ability to reproduce the browsing behavior of households across websites. In addition, we compare these machine learning methods to a related classical technique, singular value decomposition. In our study, the replicated softmax model outperforms latent Dirichlet allocation, but the correlated topic model attains the overall best performance. Compared to singular value decomposition both the correlated topic model and the replicated softmax model lead to a more efficient compression of web browsing data. On the other hand, singular value decomposition surpasses latent Dirichlet allocation. We interpret results of the correlated topic model and the replicated softmax model by determining combinations of topics or hidden variables that are heterogeneous with respect to visited websites. We show that decision makers should not rely on bivariate measures of site visits, as these do not agree with measures of interdependences between sites that can be inferred from the correlated topic model or the replicated softmax model. We investigate how well topics or hidden variables measured by these methods predict yearly household expenditures. The correlated topic model leads to the best predictive performance, followed by the replicated softmax model. We also discuss how the replicated softmax model can be used to support online marketing decisions of websites.

Sometimes the Joint Statistical Meetings (JSM) is frustrating to attend, because multiple sessions on the same topic are scheduled at the same time. This article uses seeded latent Dirichlet alloca...

Yönetim Bilişim Sistemlerinin etkilendiği branşlar, alanda geliştirilen teoriler ve var olan konu çeşitliliklerinin belirlenmesi bir disiplin olarak tanımlandığı ilk günden bu yana akademik literatürde yoğun olarak tartışılan bir konudur. Bu araştırmada, Türkiye’de hızlı ve dinamik bir gelişim gösteren YBS anabilim dalında 2002-2020 yılları arasında tamamlanan 574 lisansüstü tez özeti metin madenciliği yöntemi olan Gizli Dirichlet Ayrımı algoritmasıyla analiz edilmiştir. Analiz sonucunda tezlerin; e-Ticaret ve Pazarlama, Sistem Geliştirme ve Etkileri, Bilişim Sistemleri’nin Organizasyonlara Etkileri, Veri Madenciliği, İnsan Kaynakları Yönetimi, Organizasyonel Değişim, Alana Özel Çalışmalar I, Alana Özel Çalışmalar II, Güvenlik, Eğitim ve Öğretim, Tahmin ve Karar Destek olmak üzere 11 farklı konu başlığı altında kümelenebileceği öngörülmektedir. Araştırma kapsamında, tahmin sonuçlarının ulusal ve uluslararası literatürle göstermiş olduğu benzerlikler ve farklılıklar tartışılmıştır. Araştırma sonucunda yönetim bilişim sistemleri üzerine çalışma yapmak isteyen araştırmacılara fikir vermek ve yol göstermek hedeflenmiştir.

Dynamic text documents, including news articles, user reviews, and blogs, are now commonly encountered in many fields. Accordingly, the topics underlying text streams also change over time. To grasp the topic changes in the increasing accumulation of text documents, there is a great need to develop automatic text analysis models to find the key changes in topics. To this end, this study proposes a topic change point detection (Topic-CD) model. Different from previous studies, we define the change point of topics from the perspective of hyperparameters associated with topic-word distributions. This allows the model to detect change points underlying the whole topic set. Under this definition, the topic modeling and change point detection are combined in a unified framework and then performed simultaneously using a Markov chain Monte Carlo algorithm. In addition, the Topic-CD model is free from setting the number of change points in advance, which makes it more convenient for practical use. We investigate the performance of the Topic-CD model numerically using synthetic data and three real datasets. The results show that the Topic-CD model can well identify the change points in topics when compared with several state-of-the-art methods.

Topic models have been widely used for learning the latent explainable representation of documents, but most of the existing approaches discover topics in a flat structure. In this study, we propose an effective hierarchical neural topic model with strong interpretability. Unlike the previous neural topic models, we explicitly model the dependency between layers of a network, and then combine latent variables of different layers to reconstruct documents. Utilizing this network structure, our model can extract a tree-shaped topic hierarchy with low redundancy and good explainability by exploiting dependency matrices. Furthermore, we introduce manifold regularization into the proposed method to improve the robustness of topic modeling. Experiments on real-world datasets validate that our model outperforms other topic models in several widely used metrics with much fewer computation costs.

This article studies the effects and implications of sybils (secondary accounts created by a person in an online platform) through the game World of Tanks from interdisciplinary, mixed-methods perspectives. Considering sybils allows us to access a "person based'' network, instead of an "account based" network, revealing formerly undetected patterns. We move on to behavioral differences between the "parent" (the initial account) and "child" (those created afterwards) accounts in a sybil relationship. We explore the behavioral patterns of sybils using network, chat, and gameplay data. We find that sybils represent players experimenting with new roles or features without damaging their play record. We find that there are significant behavioral differences between different sybil accounts, and we leverage them to build a machine learning classifier to differentiate sybils. This classifier is able to identify sybil accounts with over 95% accuracy for sybil/non-sybil, 61% for parent/child. This study demonstrates the underexplored but rich potential for sybils to improve research and industry practitioners' understandings of user practices and experiences.

When considering a joint venture or merger, it is essential for firms to explore innovators with complementary technology to compensate for any internal limitations in R&D resources. In this article, we provide a framework for exploring the technology complementarity between enterprises in a quantitative manner based on text-mining patent data. A hierarchical latent Dirichlet allocation topic model identifies the technology topics hidden in patent documents along with the hierarchical structure of those topics. The technology complementarity between broad classes of technology and their subclassifications across enterprises is then measured with an improved formulation. An empirical study on three-dimensional printing illustrates the validity, reliability, and practicality of this method and the measurement formula used, endorsed by technical experts. This method can be used to identify R&D opportunities, to find appropriate acquisition targets and potential collaborators, and to support managerial decision-making with quantified information on technology complementarity.

In this article, the fourth industrial revolution (4th IR) has features that are distinct from the previous three industrial revolutions in terms of a paradigm shift. Although 4th IR relevant technologies may have positive impacts in many fields, they may also create harmful impacts, just as every previous industrial revolution has. Most studies dealing with the 4th IR have focused on its positive impacts; the present research concentrates on its negative impacts and suggests technological opportunities to hedge against expected risks in the future. To this end, future societal topics are first identified through latent Dirichlet allocation based topic modeling using textual information of futuristic data, which is retrieved from the website, including information on future-related insights. Second, the risk topics related to the future are selected with the help of a weighting score using predefined risk keywords and sentiment analysis. Finally, safe technological opportunities are identified based on textual information derived from patent data related to risk topics with the perspectives on risk analysis that have prior (prediction, prevention) and posterior perspectives (response, recovery). To illustrate technology relevant to the 4th IR, artificial intelligence (AI) is selected. As a result, 36 topics related to future society and six future risk-related topics are suggested that reflects security-relevant properties of AI and three types of threats. Opportunities to ensure or safety through technology are also proposed in four perspectives. Technological opportunities related to safety are explored in two perspectives. One is to expand existing threats, and the other is to suggest new threats. This research can be utilized to plan future safety technology by taking a data-driven approach under the huge paradigm shift of the 4th IR and proposing potential technological solutions that are not future societal scenarios and policy solutions suggested in the previous studies.

The COVID-19 pandemic fueled one of the most rapid vaccine developments in history. However, misinformation spread through online social media often leads to negative vaccine sentiment and hesitancy. To investigate COVID-19 vaccine-related discussion in social media, we conducted a sentiment analysis and Latent Dirichlet Allocation topic modeling on textual data collected from 13 Reddit communities focusing on the COVID-19 vaccine from Dec 1, 2020, to May 15, 2021. Data were aggregated and analyzed by month to detect changes in any sentiment and latent topics. ty analysis suggested these communities expressed more positive sentiment than negative regarding the vaccine-related discussions and has remained static over time. Topic modeling revealed community members mainly focused on side effects rather than outlandish conspiracy theories. Covid-19 vaccine-related content from 13 subreddits show that the sentiments expressed in these communities are overall more positive than negative and have not meaningfully changed since December 2020. Keywords indicating vaccine hesitancy were detected throughout the LDA topic modeling. Public sentiment and topic modeling analysis regarding vaccines could facilitate the implementation of appropriate messaging, digital interventions, and new policies to promote vaccine confidence., 8 pages, 4 Figures, 2 Tables

In recent years, online learning platforms (e.g., Coursera, edX) have experienced massive growth and have reached nearly 200 million learners. Although their reach is quite large, the impact of these platforms is constrained by a low level of learner engagement. In traditional face-to-face classrooms, educators aim to engage learners by asking them to participate in class discussions and share information about their identity and ideas. However, the effectiveness of these strategies in online learning platforms is uncertain. The authors examine this issue by assessing the impact of two different types of content sharing on learner engagement. The authors conduct a textual analysis of over 12,000 text postings during an 18-month period (Study 1) and a field experiment among over 2,000 learners (Study 2) in a popular Coursera offering by a large U.S. university. The results indicate that asking learners to share ideas (vs. their identity) has a stronger effect on their video consumption and assessment completion. The authors explain this “idea advantage” by suggesting that learners who share ideas (vs. identity) exhibit a greater degree of elaboration. This idea advantage is strongest for learners from English-speaking countries and those new to online learning.

The ubiquity of social media platforms facilitates free flow of online chatter related to customer experience. Twitter is a prominent social media platform for sharing experiences, and e-retail firms are rapidly emerging as the preferred shopping destination. This study explores customers’ online shopping experience tweets. Customers tweet about their online shopping experience based on moments of truth shaped by encounters across different touchpoints. We aggregate 25,173 such tweets related to six e-retailers tweeted over a 5-year period. Grounded on agency theory, we extract the topics underlying these customer experience tweets using unsupervised latent Dirichlet allocation. The output reveals five topics which manifest into customer experience tweets related to online shopping—ordering, customer service interaction, entertainment, service outcome failure, and service process failure. Topics extracted are validated through inter-rater agreement with human experts. The study, thus, derives topics from tweets about e-retail customer experience and thereby facilitates prioritization of decision-making pertaining to critical service encounter touchpoints.

Deciding when and which products to recommend to whom is always an essential issue for retailers. In this study, we propose a mixed framework with two components to capture customer buying behavior and its changes over time and visualize these results to better help retailers choose and target products strategically for marketing. In this framework, a topic model is first used to extract customer’s purchase behavior instead of association rules or K-means as mainly used in market field. To automatically choose the optimal number of topics, we implement an approach proposed by Koltcov et al. on point-of-sale (POS) data in the supermarket. Meanwhile, to grasp the change of topics over time, we divided monthly POS data in half and applied the topic model with Renyi entropy separately. The results suggest that splitting data might be a better way to understand customer behavior. Second, we consider how to develop an effective way to visualize the results of the topic model, which is essential, because in a supermarket context, simply knowing which product categories are included under which topics is not enough to support how a supermarket promotes their products. To address this, we design a three-layer visualization approach to better interpret the topic model results and to help retailers design target promotion strategies. The design of visualization was overlooked by studies related to the use of topic models on supermarket data. Finally, to demonstrate the usefulness of our proposed framework, we conduct a simple scenario-based analysis between our framework and other models, such as Latent Dirichlet Allocation (LDA) and the Dynamic Topic Model (DTM). The results show that for most periods, our proposed framework outperforms LDA and DTM.

Virus Corona atau Covid-19 menjadi perhatian khusus diseluruh dunia. Banyak masyarakat membicarakan virus ini melalui unggahan komentar dan opini di Media Sosial.Twitter merupakan salah satu media sosial yang saat ini masih banyak digunakan masyarakat untuk menyampaikan opini berupa kumpulan kata yang disebut tweets. Tweets yang berkaitan dengan topik covid-19 ini dapat di klasifikasikan menggunakan metode Topic Modeling untuk menghasilkan sebuah data topic yang sering dibicarakan pengguna twitter. Salah satu algoritma yang digunakan untuk melakukan Topic Modeling adalah menggunakan Latent Dirichlet Alocation (LDA). Pada penelitian ini LDA digunakan untuk mengetahui kata-kata apa saja yang banyak muncul pada data tweets tentang Covid-19 yang telah di unggah masyarakat melalui twitter. Sebelum data tweet dimodelkan dengan LDA, dilakukan terlebih dahulu analisis sentiment dengan Naïve Bayes Classifier untuk menghasilkan sentiment Positif, Negati dan Netral. Terdapat 5000 tweets dijadikan dataset untuk diklasifikasikan menggunakan Topic Modeling. Semua tweets yang yang di tambang masih perlu dilakukan preprocessing text yang bertujuan untuk menghapus kata-kata yang tidak baku, menghapus tanda baca, dan menghapus kata penyambung. Tweets yang sudah dilakukan preprocessing text lalu diberikan nilai bobot sehingga diketahui kata apa saja yang banyak muncul dalam tweets yang berkaitan dengan Covid-19. Kata-kata yang banyak muncul dan sudah diberikan bobot akan divisualisasikan menggunakan World Cloud sehingga dapat dilihat pemetaan kata apa saja yang banyak muncul dalam bentuk gambar.