Search

In this paper, we develop an approach for semi-supervised document clustering based on Latent Dirichlet Allocation (LDA), namely LLDA. A small amount of labeled documents are used to indicate user's document grouping preference. A generative model is investigated to jointly model documents and the small amount of document labels. A variational inference algorithm is developed to infer the document collection structure. We explore the performance of our proposed approach on both a synthetic dataset and realistic document datasets. Our experiments indicate that our proposed approach performs well on grouping documents based on different user grouping preferences. The comparison between our proposed approach and state-of-the-art semi-supervised clustering algorithms using labeled instance shows that our approach is effective. clustering model designed with the LDA model. Considering the effectiveness of the LDA model on the document clustering problem, in this paper, we investigate a LDA-based model for semi-supervised document clustering, namely LLDA. Labeled documents are used as the type of supervised-information and are used to indicate user's document grouping preferences. A generative model is investigated by using which documents are partitioned by maximizing the joint generative likelihood of text documents and the user-provided document labels. These labels were treated as variables which obey normal distribution and are regressed on the topic proportions. The computational cost of LLDA parameter estimation is also a problem for developing the LLDA model for the semi-supervised document clustering. Traditionally, there are two algorithms to infer LLDA parameters, in particular, the variational inference algorithm and the Gibbs sampling algorithm. Compared with the Gibbs sampling algorithm, the variational inference algorithm shows better computational performance due to the high dimensional representation of text documents. In this paper, we also derived a variational inference algorithm for the LLDA model. We have conducted extensive experiments on our proposed LLDA model by using both synthetic and realistic datasets. We also compared our approach with state-of-the-art semi-supervised document clustering algorithms with labeled documents as supervised information. Experimental results show that the LLDA model is effective for semi-supervised document clustering.

Finding the appropriate number of clusters to which documents should be partitioned is crucial in document clustering. In this paper, we propose a novel approach, namely DPMFP, to discover the latent cluster structure based on the DPM model without requiring the number of clusters as input. Document features are automatically partitioned into two groups, in particular, discriminative words and nondiscriminative words, and contribute differently to document clustering. A variational inference algorithm is investigated to infer the document collection structure as well as the partition of document words at the same time. Our experiments indicate that our proposed approach performs well on the synthetic data set as well as real data sets. The comparison between our approach and state-of-the-art document clustering approaches shows that our approach is robust and effective for document clustering.

Topic modeling has been widely used to mine topics. However, users' individual needs are seldom considered, which is against the trend that individuation becomes more and more important. In this work, we propose a user-oriented probabilistic topic model based on Latent Dirichlet Allocation. Interested and uninterested words are used as supervised information to take users' preferences into account. A self-learning algorithm increasing the quantity of supervised information effectively are also presented. As a semi-supervised model, data with or without supervised information attached are treated differently. In the parameters inference, we integrate the Polya urn model into the Gibbs sampling process to utilize different kinds of supervised information efficiently. Experiments conducted on real datasets show the model outperforms the state-of-the-art models.

One essential issue of document clustering is to estimate the appropriate number of clusters for a document collection to which documents should be partitioned. In this paper, we propose a novel approach, namely DPMFS, to address this issue. The proposed approach is designed 1) to group documents into a set of clusters while the number of document clusters is determined by the Dirichlet process mixture model automatically; 2) to identify the discriminative words and separate them from irrelevant noise words via stochastic search variable selection technique. We explore the performance of our proposed approach on both a synthetic dataset and several realistic document datasets. The comparison between our proposed approach and stage-of-the-art document clustering approaches indicates that our approach is robust and effective for document clustering.

This paper investigates a framework that discovers pair-wise constraints for semi-supervised text document clustering. An active learning approach is proposed to select informative document pairs for obtaining user feedbacks. A gain directed document pair selection method that measures how much we can learn by revealing the relationships between pairs of documents is designed. Three different models, namely, uncertainty model, generation error model, and objective function model are proposed. Language modeling is investigated for representing clusters in the semi-supervised document clustering approach.

This paper investigates the idea of incorporating incremental user feedbacks and a small amount of sample documents for some, not necessarily all, clusters into text clustering. For the modeling of each cluster, we make use of a local weight metric to reflect the importance of the features for a particular cluster. The local weight metric is learned using both the unlabeled data and the constraints generated automatically from user feedbacks and sample documents. The quality of local metric is improved by incorporating more precise constraints. Improving the quality of local metric will in return enhance the clustering performance. We have conducted extensive experiments on real-world news documents. The results demonstrate that user feedback information coupled with local metric learning can dramatically improve the clustering performance.