Search

Personalizing the analysis for web search potentially improves the search experience. A good analytical model for web search should leverage not only collective wisdom but also individual characteristics. Most of the existing analytical models, however, focus on how to utilize the collective wisdom, from a crowd, for instance. In this paper, we address the problem of user-specific web search analysis by considering the so-called burstiness in web search, which captures the behavior of rare words appearing many times in a single document. We go beyond click graph and propose two probabilistic topic models, Topic Independence Model and Topic Dependence Model. The former adopts the assumption that the generation of query terms and URLs are topically independent, and the latter captures the coupling between search queries and URLs. We also capture the temporal burstiness of topics by utilizing continuous Beta distribution. Through a large-scale analysis of a real-life search query log, we observe that each user's web search trail enjoys multiple kinds of user-based unique characteristics. On a massive search query log, the new models achieve a better held-out likelihood than existing baselines, and they can also effectively reveal the latent evolution of topics on the corpus level and user-based level. IEEE

This thesis considers the analysis of app reviews written on mobile app platforms using Structural Topic Modeling (STM). Analyzing reviews can help developers increase their app’s popularity and revenues. Due to a large number of app reviews being available, manual analysis can be difficult. STM offers a way to discover the topics that are discussed in large sets of texts and can provide a way to quickly analyze the contents of the reviews. Since the reviews often have low quality or contain irrelevant information, the resulting STM models can contain much noise, making analysis difficult. Conventional solutions to remove uninformative texts from the data often include supervised learning methods and other approaches that still require human labor, which goes against the advantages of STM’s unsupervised nature. This work uses four review datasets from the Google Play Store to show that filtering reviews using their helpfulness/thumbs-up rating, along with some other filtering and pre-processing techniques, can improve the usage of STM for app reviews., by Alexander W. de Jong, Masterarbeit Universität Innsbruck 2022

Digital technologies significantly improved people's lives, and result in the concept of digital economy. Recently, virtual reality technology has been developed continuously, which helps to promote the digital economy. In this context, there have been growing online e-commerce platforms that are designed for VR merchants, e.g., Oculus1. On these platforms, merchants can present their VR applications for attracting consumers to purchase. In this paper, we aim at exploring and interpreting the relationships among VR applications in these e-commerce platforms. To this end, we collect a dataset from the Oculus and use a classical topic model, namely Link-LDA, to jointly model user purchase records and user reviews for the VR applications. Through extensive experiments, we find that the Link-LDA not only identifies the associations among VR applications well but also provides an explanation for these associations. Our research provides insights into VR consumer behavior analysis and VR application recommendations on e-commerce platforms. © 2022 IEEE.

Recently, with the development of stereo display and 3D graph-ics technology, virtual environments and applications are growing rapidly, e.g., video conferencing, and virtual reality (VR) applications. These are several E-commerce platforms that are designed for the transactions of VR applications. Consumers on these platforms can comment on a VR application after purchase. In this paper, we attempt to predict the helpfulness of these VR application reviews. To this end, we propose a topic-causal regression model that explores the influence of topic features in VR application reviews and nu-merical information on helpfulness. Specifically, we first apply the most classic topic model, Latent Dirichlet Allocation, to extract the topic signals in VR application reviews. Then, we construct a topic regression for causal inference. We perform extensive experiments on a real-world dataset collected from Oculus. The experimental results demonstrate that our model can estimate regression weights for topic factors and analyze their influence on the helpfulness of VR application reviews. © 2022 IEEE.

Prompted by the increasing public focus on environmental policy and the continuous inability of States to reach environmental targets agreed upon in the context of the United Nations and the European Union, this thesis explores the development of discourses within the Danish public agenda regarding nature management 2016-2021. This is done through a mixed-methods framework of discourse analysis and structural topic modeling, which based on documents containing questions and answers about nature management asked in the Danish Parliament’s Environmental committee 2016-2021, estimates the changes in topic prevalence, an analyzes the discourses within each topic. This resulted in a qualitative overview of 21 identified topics and their associated discourses and a quantitative overview of how the different topic proportions changed over time. Two specific changes in the public agenda were found: A change from discussions about untouched forest focused on tradeoff between timber extraction and biodiversity, to a discussion about how animal welfare should be understood in regard to the large grazers in nature national parks which are kept under less supervision than other grazers in Denmark. The other change was from a focus on the environmental commitments made on a United Nations-level to an European Union-level.

Prompted by the increasing public focus on environmental policy and the continuous inability of States to reach environmental targets agreed upon in the context of the United Nations and the European Union, this thesis explores the development of discourses within the Danish public agenda regarding nature management 2016-2021. This is done through a mixed-methods framework of discourse analysis and structural topic modeling, which based on documents containing questions and answers about nature management asked in the Danish Parliament’s Environmental committee 2016-2021, estimates the changes in topic prevalence, an analyzes the discourses within each topic. This resulted in a qualitative overview of 21 identified topics and their associated discourses and a quantitative overview of how the different topic proportions changed over time. Two specific changes in the public agenda were found: A change from discussions about untouched forest focused on tradeoff between timber extraction and biodiversity, to a discussion about how animal welfare should be understood in regard to the large grazers in nature national parks which are kept under less supervision than other grazers in Denmark. The other change was from a focus on the environmental commitments made on a United Nations-level to an European Union-level.

The pace of digitized collective knowledge accumulation has become increasingly rapid in the last few years. That means we have tremendous amounts of information content to be organized, searched, and understood that can be arranged only by employing automatic methods. In the case of textual data analysis, topic modeling, a machine learning method, is definitely the most famous framework to uncover latent topics from text documents. Adopting topic modeling approaches for studying textual sources is a well-established practice in many scientific and humanities studies fields, including the historical research scope. In this paper, we present a benchmark corpus for topic models, a dataset containing an annotated real-world collection of texts focused on the antisemitism theme in 19th century France. The benchmark corpus has been developed to address a specific machine learning task but it can also support the enhancement of other natural language processing-based studies, in particular, those concerning the historical sphere.

It is beneficial for both investors and companies to avoid the detrimental consequences of overpricing during an initial public offering (IPO). Prospectuses are an important source of information for potential investors. Through Latent Dirichlet Allocation (LDA) we extract topics from the summary section of prospectuses S-1 for companies holding an IPO in the U.S. in 2019-2020. We represent the uniqueness of the companies through the topic proportions each document is composed of and use them, together with the initial offering price, to predict the outcome of the IPO. For the best performing model, we obtain an AUC of 0.80. In line with signalling theory, we argue that prospectuses may indeed send signals able to influence potential investors.

Individual passenger travel patterns have significant value in understanding passenger’s behavior, such as learning the hidden clusters of locations, time, and passengers. The learned clusters further enable commercially beneficial actions such as customized services, promotions, data-driven urban-use planning, peak hour discovery, and so on. However, the individualized passenger modeling is very challenging for the following reasons: 1) The individual passenger travel data are multi-dimensional spatiotemporal big data, including at least the origin, destination, and time dimensions; 2) Moreover, individualized passenger travel patterns usually depend on the external environment, such as the distances and functions of locations, which are ignored in most current works. This work proposes a multi-clustering model to learn the latent clusters along the multiple dimensions of Origin, Destination, Time, and eventually, Passenger (ODT-P). We develop a graph-regularized tensor Latent Dirichlet Allocation (LDA) model by first extending the traditional LDA model into a tensor version and then applies to individual travel data. Then, the external information of stations is formulated as semantic graphs and incorporated as the Laplacian regularizations; Furthermore, to improve the model scalability when dealing with massive data, an online stochastic learning method based on tensorized variational Expectation-Maximization algorithm is developed. Finally, a case study based on passengers in the Hong Kong metro system is conducted and demonstrates that a better clustering performance is achieved compared to state-of-the-arts with the improvement in point-wise mutual information index and algorithm convergence speed by a factor of two. © 2022, The Author(s).

Automated sentiment analysis is becoming increasingly recognized due to the growing importance of social media and e-commerce platform review websites. Deep neural networks outperform traditional lexicon-based and machine learning methods by effectively exploiting contextual word embeddings to generate dense document representation. However, this representation model is not fully adequate to capture topical semantics and the sentiment polarity of words. To overcome these problems, a novel sentiment analysis model is proposed that utilizes richer document representations of word-emotion associations and topic models, which is the main computational novelty of this study. The sentiment analysis model integrates word embeddings with lexicon-based sentiment and emotion indicators, including negations and emoticons, and to further improve its performance, a topic modeling component is utilized together with a bag-of-words model based on a supervised term weighting scheme. The effectiveness of the proposed model is evaluated using large datasets of Amazon product reviews and hotel reviews. Experimental results prove that the proposed document representation is valid for the sentiment analysis of product and hotel reviews, irrespective of their class imbalance. The results also show that the proposed model improves on existing machine learning methods., Automatizovaná analýza sentimentu je stále více uznávaná vzhledem k rostoucímu významu sociálních médií a webových stránek s recenzemi platforem elektronického obchodování. Hluboké neuronové sítě překonávají tradiční metody založené na lexikonu a strojovém učení tím, že efektivně využívají kontextové vložené slovo k vytvoření husté reprezentace dokumentu. Tento model reprezentace však není plně adekvátní pro zachycení aktuální sémantiky a polarity sentimentu slov. K překonání těchto problémů je navržen nový model analýzy sentimentu, který využívá bohatší reprezentace dokumentů asociací slov a emocí a tematických modelů, což je hlavní výpočetní novinkou této studie. Model analýzy sentimentu integruje vložená slova s lexikonem založeným na indikátorech sentimentu a emocí, včetně negací a emotikonů, a pro další zlepšení jeho výkonu je využita komponenta tematického modelování spolu s modelem bag-of-words založeným na kontrolovaném schématu vážení termů. Účinnost navrženého modelu je hodnocena na rozsáhlých souborech dat recenzí produktů Amazon a recenzí hotelů. Výsledky experimentů dokazují, že navržená reprezentace dokumentů je platná pro analýzu sentimentu recenzí produktů a hotelů bez ohledu na nevyváženost jejich tříd. Výsledky také ukazují, že navržený model je lepší než stávající metody strojového učení.

It is beneficial for both investors and companies to avoid the detrimental consequences of overpricing during an initial public offering (IPO). Prospectuses are an important source of information for potential investors. Through Latent Dirichlet Allocation (LDA) we extract topics from the summary section of prospectuses S-1 for companies holding an IPO in the U.S. in 2019-2020. We represent the uniqueness of the companies through the topic proportions each document is composed of and use them, together with the initial offering price, to predict the outcome of the IPO. For the best performing model, we obtain an AUC of 0.80. In line with signalling theory, we argue that prospectuses may indeed send signals able to influence potential investors.

This thesis considers the analysis of app reviews written on mobile app platforms using Structural Topic Modeling (STM). Analyzing reviews can help developers increase their app’s popularity and revenues. Due to a large number of app reviews being available, manual analysis can be difficult. STM offers a way to discover the topics that are discussed in large sets of texts and can provide a way to quickly analyze the contents of the reviews. Since the reviews often have low quality or contain irrelevant information, the resulting STM models can contain much noise, making analysis difficult. Conventional solutions to remove uninformative texts from the data often include supervised learning methods and other approaches that still require human labor, which goes against the advantages of STM’s unsupervised nature. This work uses four review datasets from the Google Play Store to show that filtering reviews using their helpfulness/thumbs-up rating, along with some other filtering and pre-processing techniques, can improve the usage of STM for app reviews., by Alexander W. de Jong, Masterarbeit Universität Innsbruck 2022

We propose a novel text-analytic approach for incorporating textual information into structural economic models and apply this to study the effects of tax news. We first develop a novel semi-supervised two-step topic model that automatically extracts specific information regarding future tax policy changes from text. We also propose an approach for transforming such textual information into an economically meaningful time series to be included in a structural econometric model as variable of interest or instrument. We apply our method to study the effects of fiscal foresight, in particular the informational content in speeches of the U.S. president about future tax reforms, and find that our semi-supervised topic model can successfully extract information about the direction of tax changes. The extracted information predicts (exogenous) future tax changes and contains signals that are not present in previously considered (narrative) measures of (exogenous) tax changes. We find that tax news triggers a significant yet delayed response in output.

Neural Topic Models are recent neural models that aim at extracting the main themes from a collection of documents. The comparison of these models is usually limited because the hyperparameters are held fixed. In this paper, we present an empirical analysis and comparison of Neural Topic Models by finding the optimal hyperparameters of each model for four different performance measures adopting a single-objective Bayesian optimization. This allows us to determine the robustness of a topic model for several evaluation metrics. We also empirically show the effect of the length of the documents on different optimized metrics and discover which evaluation metrics are in conflict or agreement with each other.

Many data sets (e.g., reviews, forums, news, etc.) exist parallelly in multiple languages. They all cover the same content, but the linguistic differences make it impossible to use traditional, bag-of-word-based topic models. Models have to be either single-language or suffer from a huge, but extremely sparse vocabulary. Both issues can be addressed by transfer learning. In this paper, we introduce a zero-shot cross-lingual topic model. Our model learns topics on one language (here, English), and predicts them for unseen documents in different languages (here, Italian, French, German, and Portuguese). We evaluate the quality of the topic predictions for the same document in different languages. Our results show that the transferred topics are coherent and stable across languages, which suggests exciting future research directions.

OCTIS is an open-source framework for training, evaluating and comparing Topic Models. This tool uses single-objective Bayesian Optimization (BO) to optimize the hyper-parameters of the models and thus guarantee a fairer comparison. Yet, a single-objective approach disregards that a user may want to simultaneously optimize multiple objectives. We therefore propose OCTIS 2.0: the extension of OCTIS that addresses the problem of estimating the optimal hyper-parameter configurations for a topic model using multi-objective BO. Moreover, we also release and integrate two pre-processed Italian datasets, which can be easily used as benchmarks for the Italian language.

Probabilistic topic models are a versatile class of models for discovering latent themes in document collections through unsupervised learning. Conventional inferential methods lack the scaling capabilities necessary for extensions to large-scale applications. In recent years Stochastic Expectation Maximization has proven scalable for the simplest topic model: Latent Dirichlet Allocation. Performing analytical maximization is unfortunately not possible for many more complex topic models. With the rise of probabilistic programming languages, the ability to infer flexibly specified probabilistic models using sophisticated numerical optimization procedures has become widely available. These frameworks have however mainly been developed for optimization of continuous parameters, often prohibiting direct optimization of discrete parameters. This thesis explores the potential of utilizing probabilistic programming for generic topic modeling using Stochastic Expectation Maximization with numerical maximization of discrete parameters reparameterized to unconstrained space. The method achieves results of similar quality as other methods for Latent Dirichlet Allocation in simulated experiments. Further application is made to infer a Dirichlet-multinomial Regression model with metadata covariates. A real dataset is used and the method produces interpretable topics.

Probabilistic topic models are a versatile class of models for discovering latent themes in document collections through unsupervised learning. Conventional inferential methods lack the scaling capabilities necessary for extensions to large-scale applications. In recent years Stochastic Expectation Maximization has proven scalable for the simplest topic model: Latent Dirichlet Allocation. Performing analytical maximization is unfortunately not possible for many more complex topic models. With the rise of probabilistic programming languages, the ability to infer flexibly specified probabilistic models using sophisticated numerical optimization procedures has become widely available. These frameworks have however mainly been developed for optimization of continuous parameters, often prohibiting direct optimization of discrete parameters. This thesis explores the potential of utilizing probabilistic programming for generic topic modeling using Stochastic Expectation Maximization with numerical maximization of discrete parameters reparameterized to unconstrained space. The method achieves results of similar quality as other methods for Latent Dirichlet Allocation in simulated experiments. Further application is made to infer a Dirichlet-multinomial Regression model with metadata covariates. A real dataset is used and the method produces interpretable topics.

We propose a novel text-analytic approach for incorporating textual information into structural economic models and apply this to study the effects of tax news. We first develop a novel semi-supervised two-step topic model that automatically extracts specific information regarding future tax policy changes from text. We also propose an approach for transforming such textual information into an economically meaningful time series to be included in a structural econometric model as variable of interest or instrument. We apply our method to study the effects of fiscal foresight, in particular the informational content in speeches of the U.S. president about future tax reforms, and find that our semi-supervised topic model can successfully extract information about the direction of tax changes. The extracted information predicts (exogenous) future tax changes and contains signals that are not present in previously considered (narrative) measures of (exogenous) tax changes. We find that tax news triggers a significant yet delayed response in output.

Public discourse about the COVID-19 that appears on Twitter and other social media platforms provides useful insights into public concerns and responses to the pandemic. However, acknowledging that public discourse around COVID-19 is multi-faceted and evolves over time poses both analytical and ontological challenges. Studies that use text-mining approaches to analyse responses to major events commonly treat public discourse on social media as an undifferentiated whole, without systematically examining the extent to which that discourse consists of distinct sub-discourses or which phases characterize its development. They also confound structured behavioural data (i.e., tagging) with unstructured user-generated data (i.e., content of tweets) in their sampling methods. The present study aims to demonstrate how one might go about addressing both of these sets of challenges by combining corpus linguistic methods with a data-driven text-mining approach to gain a better understanding of how the public discourse around COVID-19 developed over time and what topics combine to form this discourse in the Australian Twittersphere over a period of nearly four months. By combining text mining and corpus linguistics, this study exemplifies how both approaches can complement each other productively.

The demand for Natural Language Processing has been thriving rapidly due to the various emerging Internet services such as social networks, e-commerce, intelligent assistant, etc. The large volume of natural language data and the new market requirements have brought significant challenges and opportunities for academia and industry. The challenges include (1) efficiently processing a massive volume of data and devising a highly scalable architecture, (2) extending the horizon of the current natural language processing to tackle problems that cannot be handled well. This thesis demonstrates several advanced techniques to tackle the challenges by boosting scalability and efficiency or extending the horizon of existing core NLP techniques on different granularities. This thesis introduces (i) a neural-based lexical simplification algorithm that utilizes contextualized word embedding to capture the meaning more accurately, (ii) a sample-efficient algorithm that solves extreme large-output classification for contrastive representation learning. (iii) a scalable and efficient algorithm for the probabilistic topic model LDA.First, we propose amplified negative sampling and rigorous mathematical analysis of the famous negative sampling method for the word level distributed representations. Specifically, we distinguish the impact of various loss functions for the contrastive representation learning process. Our theoretical analysis provides a better understanding of a few empirical observations that are well known among practitioners when negative sampling is employed. We also propose the amplified negative sampling based on our analysis. It's a simple yet efficient method that can effectively improve the negative sampling efficiency and boost the downstream tasks' performance. Second, we propose the neural simplicity ranking (NSR) model that utilizes the contextualized word representations in an unconventional way for lexical simplification for the sentence level tasks. Thi

Topic models for focused analysis aim to capture topics within the limiting scope of a targeted aspect (which could be thought of as some inner topic within a certain domain). To serve their analytic purposes, topics are expected to be semantically-coherent and closely aligned with human intuition – this in itself poses a major challenge for the more common topic modeling algorithms which, in a broader sense, perform a full analysis that covers all aspects and themes within a collection of texts. The paper attempts to construct a viable focused-analysis topic model which learns topics from Twitter data written in a closely related group of non-standardized varieties of Arabic widely spoken in the Levant region (i.e Levantine Arabic). Results are compared to a baseline model as well as another targeted topic model designed precisely to serve the purpose of focused analysis. The model is capable of adequately capturing topics containing terms which fall within the scope of the targeted aspect when judged overall. Nevertheless, it fails to produce human-friendly and semantically-coherent topics as several topics contained a number of intruding terms while others contained terms, while still relevant to the targeted aspect, thrown together seemingly at random.

Topic models for focused analysis aim to capture topics within the limiting scope of a targeted aspect (which could be thought of as some inner topic within a certain domain). To serve their analytic purposes, topics are expected to be semantically-coherent and closely aligned with human intuition – this in itself poses a major challenge for the more common topic modeling algorithms which, in a broader sense, perform a full analysis that covers all aspects and themes within a collection of texts. The paper attempts to construct a viable focused-analysis topic model which learns topics from Twitter data written in a closely related group of non-standardized varieties of Arabic widely spoken in the Levant region (i.e Levantine Arabic). Results are compared to a baseline model as well as another targeted topic model designed precisely to serve the purpose of focused analysis. The model is capable of adequately capturing topics containing terms which fall within the scope of the targeted aspect when judged overall. Nevertheless, it fails to produce human-friendly and semantically-coherent topics as several topics contained a number of intruding terms while others contained terms, while still relevant to the targeted aspect, thrown together seemingly at random.

テキストマイニングとデジタルヒューマニティーズ

In modern retail contexts, retailers sell products from vast product assortments to a large and heterogeneous customer base. Understanding purchase behavior in such a context is very important. Standard models cannot be used due to the hig

The demand for Natural Language Processing has been thriving rapidly due to the various emerging Internet services such as social networks, e-commerce, intelligent assistant, etc. The large volume of natural language data and the new market requirements have brought significant challenges and opportunities for academia and industry. The challenges include (1) efficiently processing a massive volume of data and devising a highly scalable architecture, (2) extending the horizon of the current natural language processing to tackle problems that cannot be handled well. This thesis demonstrates several advanced techniques to tackle the challenges by boosting scalability and efficiency or extending the horizon of existing core NLP techniques on different granularities. This thesis introduces (i) a neural-based lexical simplification algorithm that utilizes contextualized word embedding to capture the meaning more accurately, (ii) a sample-efficient algorithm that solves extreme large-output classification for contrastive representation learning. (iii) a scalable and efficient algorithm for the probabilistic topic model LDA.First, we propose amplified negative sampling and rigorous mathematical analysis of the famous negative sampling method for the word level distributed representations. Specifically, we distinguish the impact of various loss functions for the contrastive representation learning process. Our theoretical analysis provides a better understanding of a few empirical observations that are well known among practitioners when negative sampling is employed. We also propose the amplified negative sampling based on our analysis. It's a simple yet efficient method that can effectively improve the negative sampling efficiency and boost the downstream tasks' performance. Second, we propose the neural simplicity ranking (NSR) model that utilizes the contextualized word representations in an unconventional way for lexical simplification for the sentence level tasks. Thi

This dissertation contains four studies in the mapping of scholarly knowledge with special reference to the social sciences. Chapter one theorizes three roles within universities, archivists, professionals, and educators, and explains how their collusions and collisions over knowledge as a valued resource shape knowledge terrains over time. Chapter two uses advanced network visualization techniques to model how the assignment of discipline subject labels to journals by digital archivists can be analyzed to reveal an implicit cognitive map of disciplines. Chapter three uses a topic model to conduct an expansive literature review covering thousands of articles to understand what the term genre means in different disciplines, and it invents several diagnostic techniques to validate model quality. Chapter four uncovers the historical origins of social science in the United States from the end of the Civil War until the Great Depression, and it uses a topic model to track the development of the genre structure of anthropology and sociology as each discipline contends with the establishment of the university system and with the dawning of World War I. Together these studies establish an agenda for the systematic cartography of social science knowledge that will help scholars reach a deeper understanding of the history of their fields.

Gaining a complete picture of the activity in a city using vast data sources is challenging yet potentially very valuable. One such source of data is Twitter which generates millions of short spatio-temporally localized messages that, as a collection, have information on city regions and many forms of city activity. The quantity of data, however, necessitates summarization in a way that makes consumption by an observer efficient, accurate, and comprehensive. We present a two-step process for analyzing geotagged twitter data within a localized urban environment. The first step involves an efficient form of latent Dirichlet allocation, using an expectation maximization, for topic content summarization of the text information in the tweets. The second step involves spatial and temporal analysis of information within each topic using two complimentary metrics. These proposed metrics characterize the distributional properties of tweets in time and space for all topics. We integrate the second step into a graphical user interface that enables the user to adeptly navigate through the space of hundreds of topics. We present results of a case study of the city of Madrid, Spain, for the year 2011 in which both large-scale protests and elections occurred. Our data analysis methods identify these important events, as well as other classes of more mundane routine activity and their associated locations in Madrid.

ObjectiveRecently, many tumor sequencing studies have inferred and reported on mutational signatures, short nucleotide patterns at which particular somatic base substitutions appear more often. A number of signatures reflect biological processes in the patient and factors associated with cancer risk. Our goal is to infer mutational signatures appearing in colon cancer, a cancer for which environmental risk factors vary by cancer subtype, and compare the signatures to those in adult stem cells from normal colon. We also compare the mutational signatures to others in the literature.ResultsWe apply a probabilistic mutation signature model to somatic mutations previously reported for six adult normal colon stem cells and 431 colon adenocarcinomas. We infer six mutational signatures in colon cancer, four being specific to tumors with hypermutation. Just two signatures explained the majority of mutations in the small number of normal aging colon samples. All six signatures are independently identified in a series of 295 Chinese colorectal cancers.

With the thriving of the social media (eg, Sina Microblog and Twitter), the public is keen on expressing their opinions or views on entities as celebrities and products. Emotion mining on social media can be applied in diverse areas, such as helping government or organizations understand people's attitudes so as to make right decisions for business services or political campaigns. One kind of the mainstream approaches for emotion mining are based on topic models, while most of these existing approaches aim at analyzing entire sentiments for the whole Microblog, rather than explicitly assigning the relevant sentiments to the specific entities. In addition, topic model is mainly based on the bag-of-words model but ignores the semantic relations of entity-word in the modeling process, which brings low accuracy and poor interpretability to the sentiment analysis. To overcome the aforementioned difficulties, an Entity Sentiment Topic Model (ESTM) is proposed, which carries out entity-dependent sentiment analysis. To improve the accuracy of sentiment analysis and enhance the interpretability of the results, ESTM is integrated with relations of entity-word and a six-dimensional emotion lexicon as weakly supervised information. Experiments have shown promising results on sentiment classification accuracy, interpretability, and quality of coherent topics for entities. © 2019 John Wiley & Sons, Ltd.

Topic modeling has been widely applied in a variety of industrial applications. Training a high-quality model usually requires massive amount of in-domain data, in order to provide comprehensive co-occurrence information for the model to learn. However, industrial data such as medical or financial records are often proprietary or sensitive, which precludes uploading to data centers. Hence training topic models in industrial scenarios using conventional approaches faces a dilemma: a party (i.e., a company or institute) has to either tolerate data scarcity or sacrifice data privacy. In this paper, we propose a novel framework named Federated Topic Modeling (FTM), in which multiple parties collaboratively train a high-quality topic model by simultaneously alleviating data scarcity and maintaining immune to privacy adversaries. FTM is inspired by federated learning and consists of novel techniques such as private Metropolis Hastings, topic-wise normalization and heterogeneous model integration. We conduct a series of quantitative evaluations to verify the effectiveness of FTM and deploy FTM in an Automatic Speech Recognition (ASR) system to demonstrate its utility in real-life applications. Experimental results verify FTM's superiority over conventional topic modeling. © 2019 Association for Computing Machinery.

Topic modeling has been widely applied in a variety of industrial applications. Training a high-quality model usually requires massive amount of in-domain data, in order to provide comprehensive co-occurrence information for the model to learn. However, industrial data such as medical or financial records are often proprietary or sensitive, which precludes uploading to data centers. Hence training topic models in industrial scenarios using conventional approaches faces a dilemma: a party (i.e., a company or institute) has to either tolerate data scarcity or sacrifice data privacy. In this paper, we propose a novel framework named Federated Topic Modeling (FTM), in which multiple parties collaboratively train a high-quality topic model by simultaneously alleviating data scarcity and maintaining immune to privacy adversaries. FTM is inspired by federated learning and consists of novel techniques such as private Metropolis Hastings, topic-wise normalization and heterogeneous model integration. We conduct a series of quantitative evaluations to verify the effectiveness of FTM and deploy FTM in an Automatic Speech Recognition (ASR) system to demonstrate its utility in real-life applications. Experimental results verify FTM's superiority over conventional topic modeling. © 2019 Association for Computing Machinery.

In the materials science domain the data-driven science paradigm has become the focus since the beginning of the 2000s. A large number of research groups and communities are building and developing data-driven workflows. However, much of the data and knowledge is stored in different heterogeneous data sources maintained by different groups. This leads to a reduced availability of the data and poor interoperability between systems in this domain. Ontology-based techniques are an important way to reduce these problems and a number of efforts have started. In this paper we investigate efforts in the materials science, and in particular in the nanotechnology domain, and show how such ontologies developed by domain experts, can be improved. We use a phrase-based topic model approach and formal topical concept analysis on unstructured text in this domain to suggest additional concepts and axioms for the ontology that should be validated by a domain expert. We describe the techniques and show the usefulness of the approach through an experiment where we extend two nanotechnology ontologies using approximately 600 titles and abstracts.

In the materials science domain the data-driven science paradigm has become the focus since the beginning of the 2000s. A large number of research groups and communities are building and developing data-driven workflows. However, much of the data and knowledge is stored in different heterogeneous data sources maintained by different groups. This leads to a reduced availability of the data and poor interoperability between systems in this domain. Ontology-based techniques are an important way to reduce these problems and a number of efforts have started. In this paper we investigate efforts in the materials science, and in particular in the nanotechnology domain, and show how such ontologies developed by domain experts, can be improved. We use a phrase-based topic model approach and formal topical concept analysis on unstructured text in this domain to suggest additional concepts and axioms for the ontology that should be validated by a domain expert. We describe the techniques and show the usefulness of the approach through an experiment where we extend two nanotechnology ontologies using approximately 600 titles and abstracts.

In the materials science domain the data-driven science paradigm has become the focus since the beginning of the 2000s. A large number of research groups and communities are building and developing data-driven workflows. However, much of the data and knowledge is stored in different heterogeneous data sources maintained by different groups. This leads to a reduced availability of the data and poor interoperability between systems in this domain. Ontology-based techniques are an important way to reduce these problems and a number of efforts have started. In this paper we investigate efforts in the materials science, and in particular in the nanotechnology domain, and show how such ontologies developed by domain experts, can be improved. We use a phrase-based topic model approach and formal topical concept analysis on unstructured text in this domain to suggest additional concepts and axioms for the ontology that should be validated by a domain expert. We describe the techniques and show the usefulness of the approach through an experiment where we extend two nanotechnology ontologies using approximately 600 titles and abstracts.

In the materials science domain the data-driven science paradigm has become the focus since the beginning of the 2000s. A large number of research groups and communities are building and developing data-driven workflows. However, much of the data and knowledge is stored in different heterogeneous data sources maintained by different groups. This leads to a reduced availability of the data and poor interoperability between systems in this domain. Ontology-based techniques are an important way to reduce these problems and a number of efforts have started. In this paper we investigate efforts in the materials science, and in particular in the nanotechnology domain, and show how such ontologies developed by domain experts, can be improved. We use a phrase-based topic model approach and formal topical concept analysis on unstructured text in this domain to suggest additional concepts and axioms for the ontology that should be validated by a domain expert. We describe the techniques and show the usefulness of the approach through an experiment where we extend two nanotechnology ontologies using approximately 600 titles and abstracts.

In the materials science domain the data-driven science paradigm has become the focus since the beginning of the 2000s. A large number of research groups and communities are building and developing data-driven workflows. However, much of the data and knowledge is stored in different heterogeneous data sources maintained by different groups. This leads to a reduced availability of the data and poor interoperability between systems in this domain. Ontology-based techniques are an important way to reduce these problems and a number of efforts have started. In this paper we investigate efforts in the materials science, and in particular in the nanotechnology domain, and show how such ontologies developed by domain experts, can be improved. We use a phrase-based topic model approach and formal topical concept analysis on unstructured text in this domain to suggest additional concepts and axioms for the ontology that should be validated by a domain expert. We describe the techniques and show the usefulness of the approach through an experiment where we extend two nanotechnology ontologies using approximately 600 titles and abstracts.

Topic modeling has been widely applied in a variety of industrial applications. Training a high-quality model usually requires massive amount of in-domain data, in order to provide comprehensive co-occurrence information for the model to learn. However, industrial data such as medical or financial records are often proprietary or sensitive, which precludes uploading to data centers. Hence training topic models in industrial scenarios using conventional approaches faces a dilemma: a party (i.e., a company or institute) has to either tolerate data scarcity or sacrifice data privacy. In this paper, we propose a novel framework named Federated Topic Modeling (FTM), in which multiple parties collaboratively train a high-quality topic model by simultaneously alleviating data scarcity and maintaining immune to privacy adversaries. FTM is inspired by federated learning and consists of novel techniques such as private Metropolis Hastings, topic-wise normalization and heterogeneous model integration. We conduct a series of quantitative evaluations to verify the effectiveness of FTM and deploy FTM in an Automatic Speech Recognition (ASR) system to demonstrate its utility in real-life applications. Experimental results verify FTM's superiority over conventional topic modeling. © 2019 Association for Computing Machinery.

by Pirmin Lechner, BA, Arbeit gesperrt, Universität Innsbruck, Masterarbeit, 2019, (VLID)4566168

Topic models discover latent topics in documents and summarize documents at a high level. To improve topic models' topic quality and extrinsic performance, external knowledge is often incorporated as part of the generative story. One form of external knowledge is weighted text links that indicate similarity or relatedness between the connected objects. This dissertation 1) uncovers the latent structures in observed weighted links and integrates them into topic modeling, and 2) learns latent weighted links from other external knowledge to improve topic modeling. We consider incorporating links at three different levels: documents, words, and topics. We first look at binary document links, e.g., citation links of papers. Document links indicate topic similarity of the connected documents. Past methods model the document links separately, ignoring the entire link density. We instead uncover latent document blocks in which documents are densely connected and tend to talk about similar topics. We introduce LBH-RTM, a relational topic model with lexical weights, block priors, and hinge loss. It extracts informative topic priors from the document blocks for documents' topic generation. It predicts unseen document links with block and lexical features and hinge loss, in addition to topical features. It outperforms past methods in link prediction and gives more coherent topics. Like document links, words are also linked, but usually with real-valued weights. Word links are known as word associations and indicate the semantic relatedness of the connected words. They provide more information about word relationships in addition to the co-occurrence patterns in the training corpora. To extract and incorporate the knowledge in word associations, we introduce methods to find the most salient word pairs. The methods organize the words in a tree structure, which serves as a prior (i.e., tree prior) for tree LDA. The methods are straightforward but effective, yielding more coherent

This dissertation contains four studies in the mapping of scholarly knowledge with special reference to the social sciences. Chapter one theorizes three roles within universities, archivists, professionals, and educators, and explains how their collusions and collisions over knowledge as a valued resource shape knowledge terrains over time. Chapter two uses advanced network visualization techniques to model how the assignment of discipline subject labels to journals by digital archivists can be analyzed to reveal an implicit cognitive map of disciplines. Chapter three uses a topic model to conduct an expansive literature review covering thousands of articles to understand what the term genre means in different disciplines, and it invents several diagnostic techniques to validate model quality. Chapter four uncovers the historical origins of social science in the United States from the end of the Civil War until the Great Depression, and it uses a topic model to track the development of the genre structure of anthropology and sociology as each discipline contends with the establishment of the university system and with the dawning of World War I. Together these studies establish an agenda for the systematic cartography of social science knowledge that will help scholars reach a deeper understanding of the history of their fields.

Gaining a complete picture of the activity in a city using vast data sources is challenging yet potentially very valuable. One such source of data is Twitter which generates millions of short spatio-temporally localized messages that, as a collection, have information on city regions and many forms of city activity. The quantity of data, however, necessitates summarization in a way that makes consumption by an observer efficient, accurate, and comprehensive. We present a two-step process for analyzing geotagged twitter data within a localized urban environment. The first step involves an efficient form of latent Dirichlet allocation, using an expectation maximization, for topic content summarization of the text information in the tweets. The second step involves spatial and temporal analysis of information within each topic using two complimentary metrics. These proposed metrics characterize the distributional properties of tweets in time and space for all topics. We integrate the second step into a graphical user interface that enables the user to adeptly navigate through the space of hundreds of topics. We present results of a case study of the city of Madrid, Spain, for the year 2011 in which both large-scale protests and elections occurred. Our data analysis methods identify these important events, as well as other classes of more mundane routine activity and their associated locations in Madrid.

ObjectiveRecently, many tumor sequencing studies have inferred and reported on mutational signatures, short nucleotide patterns at which particular somatic base substitutions appear more often. A number of signatures reflect biological processes in the patient and factors associated with cancer risk. Our goal is to infer mutational signatures appearing in colon cancer, a cancer for which environmental risk factors vary by cancer subtype, and compare the signatures to those in adult stem cells from normal colon. We also compare the mutational signatures to others in the literature.ResultsWe apply a probabilistic mutation signature model to somatic mutations previously reported for six adult normal colon stem cells and 431 colon adenocarcinomas. We infer six mutational signatures in colon cancer, four being specific to tumors with hypermutation. Just two signatures explained the majority of mutations in the small number of normal aging colon samples. All six signatures are independently identified in a series of 295 Chinese colorectal cancers.

Product defects concern various groups of people, such as customers, manufacturers, government officials, etc. Thus, defect-related knowledge and information are essential. In keeping with the growth of social media, online forums, and Internet commerce, people post a vast amount of feedback on products, which forms a good source for the automatic acquisition of knowledge about defects. However, considering the vast volume of online reviews, how to automatically identify critical product defects and summarize the related information from the huge number of user reviews is challenging, even when we target only the negative reviews. As a kind of opinion mining research, existing defect discovery methods mainly focus on how to classify the type of product issues, which is not enough for users. People expect to see defect information in multiple facets, such as product model, component, and symptom, which are necessary to understand the defects and quantify their influence. In addition, people are eager to seek problem resolutions once they spot defects. These challenges cannot be solved by existing aspect-oriented opinion mining models, which seldom consider the defect entities mentioned above. Furthermore, users also want to better capture the semantics of review text, and to summarize product defects more accurately in the form of natural language sentences. However, existing text summarization models including neural networks can hardly generalize to user review summarization due to the lack of labeled data. In this research, we explore topic models and neural network models for product defect discovery and summarization from user reviews. Firstly, a generative Probabilistic Defect Model (PDM) is proposed, which models the generation process of user reviews from key defect entities including product Model, Component, Symptom, and Incident Date. Using the joint topics in these aspects, which are produced by PDM, people can discover defects which are represented by th

Web search analysis plays a critical role in improving the performance of cutting-edge search engines. Most of the existing models, such as the click graph and its variants, focus on utilizing the wisdom of the crowd. However, how to design a model supporting both the collective wisdom as well as the unique characteristic of individuals is rarely studied. In this paper, our goal is to solve the new problem of user-specific web search analysis. We go beyond click graph and propose two probabilistic topic models, Topic Independence Model(TIM) and Topic Dependence Model (TDM). TIM adopts an assumption that the generation of query terms and URLs are topically independent; TDM captures the coupling between search queries and URLs. We also capture the temporal burstiness of topics by utilizing the continuous Beta distribution. Through a large-scale analysis of a real-life search query log, we observe that each user's web search trail enjoys multiple kinds of user-based unique characteristics. On a massive search query log, the new models achieve a better held-out likelihood than standard LDA, DCMLDA and TOT, and they can also effectively reveal the latent evolutions of topics on the corpus level and user-based level. © 2018 IEEE.

Web search analysis plays a critical role in improving the performance of cutting-edge search engines. Most of the existing models, such as the click graph and its variants, focus on utilizing the wisdom of the crowd. However, how to design a model supporting both the collective wisdom as well as the unique characteristic of individuals is rarely studied. In this paper, our goal is to solve the new problem of user-specific web search analysis. We go beyond click graph and propose two probabilistic topic models, Topic Independence Model(TIM) and Topic Dependence Model (TDM). TIM adopts an assumption that the generation of query terms and URLs are topically independent; TDM captures the coupling between search queries and URLs. We also capture the temporal burstiness of topics by utilizing the continuous Beta distribution. Through a large-scale analysis of a real-life search query log, we observe that each user's web search trail enjoys multiple kinds of user-based unique characteristics. On a massive search query log, the new models achieve a better held-out likelihood than standard LDA, DCMLDA and TOT, and they can also effectively reveal the latent evolutions of topics on the corpus level and user-based level. © 2018 IEEE.

Web search analysis plays a critical role in improving the performance of cutting-edge search engines. Most of the existing models, such as the click graph and its variants, focus on utilizing the wisdom of the crowd. However, how to design a model supporting both the collective wisdom as well as the unique characteristic of individuals is rarely studied. In this paper, our goal is to solve the new problem of user-specific web search analysis. We go beyond click graph and propose two probabilistic topic models, Topic Independence Model(TIM) and Topic Dependence Model (TDM). TIM adopts an assumption that the generation of query terms and URLs are topically independent; TDM captures the coupling between search queries and URLs. We also capture the temporal burstiness of topics by utilizing the continuous Beta distribution. Through a large-scale analysis of a real-life search query log, we observe that each user's web search trail enjoys multiple kinds of user-based unique characteristics. On a massive search query log, the new models achieve a better held-out likelihood than standard LDA, DCMLDA and TOT, and they can also effectively reveal the latent evolutions of topics on the corpus level and user-based level. © 2018 IEEE.

Identification of disease-related alterations in molecular and cellular mechanisms may reveal useful biomarkers for human diseases including cancers. High-throughput omic technologies for identifying and quantifying multi-level biological molecules (e.g., proteins, glycans, and metabolites) have facilitated the advances in biological research in recent years. Liquid (or gas) chromatography coupled with mass spectrometry (LC/GC-MS) has become an essential tool in such large-scale omic studies. Appropriate LC/GC-MS data preprocessing pipelines are needed to detect true differences between biological groups. Challenges exist in several aspects of MS data analysis. Specifically for biomarker discovery, one fundamental challenge in quantitation of biomolecules is owing to the heterogeneous nature of human biospecimens. Although this issue has been a subject of discussion in cancer genomic studies, it has not yet been rigorously investigated in mass spectrometry based omic studies. Purification of mass spectometric data is highly desired prior to subsequent differential analysis. In this research dissertation, we majorly target at addressing the purification problem through probabilistic modeling. We propose an intensity-level purification model (IPM) to computationally purify LC/GC-MS based cancerous data in biomarker discovery studies. We further extend IPM to scan-level purification model (SPM) by considering information from extracted ion chromatogram (EIC, scan-level feature). Both IPM and SPM belong to the category of topic modeling approach, which aims to identify the underlying "topics" (sources) and their mixture proportions in composing the heterogeneous data. Additionally, denoise deconvolution model (DMM) is proposed to capture the noise signals in samples based on purified profiles. Variational expectation-maximization (VEM) and Markov chain Monte Carlo (MCMC) methods are used to draw inference on the latent variables and estimate the model parameters. Before