Search

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm. [ABSTRACT FROM AUTHOR]

Most recent approaches for Text Simplification (TS) have drawn on insights from machine translation to learn simplification rewrites from the monolingual parallel corpus of complex and simple sentences, yet their effectiveness strongly relies on large amounts of parallel sentences. However, there has been a serious problem haunting TS for decades, that is, the availability of parallel TS corpora is scarce or not fit for the learning task. In this paper, we will focus on one especially useful and challenging problem of unsupervised TS without a single parallel sentence. To the best of our knowledge, we present the first unsupervised text simplification system based on phrase-based machine translation system, which leverages a careful initialization of phrase tables and language models. On the widely used WikiLarge and WikiSmall benchmarks, our system respectively obtains 39.08 and 25.12 SARI points, even outperforms some supervised baselines. [ABSTRACT FROM AUTHOR]

Dirichlet Multinomial Mixture (DMM) models have been successful in clustering short texts. However, the word co-occurrence information that can be captured by these models is limited to the short text corpus itself. If two words have strong relatedness but rarely co-occurring in short texts, these models can not fully capture the semantic relatedness between the two words. In this paper, we propose a novel model by incorporating word-word correlation into DMM, called WDMM. By constructing a sparse graph using word-word relationship, our model expands each short text using their neighboring words in each text that can help to solve the problem of sparseness in short texts. Therefore, the cluster label of each text is not only influenced by its words, but decided by their similar words in this corpus. Experimental results on real-world datasets demonstrated the substantial superiority of our WDMM model over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Although powerful pretrained language models generate high-quality output text, they bring new concerns about the potential misuse of such models for malicious purposes. Natural language watermarking (NLW) is a technique that is desgined to help tracing the provenance of texts for againsting possible attacks, where the watermark signals are embedded into cover texts using synonym substitutions. The up-to-date BERT-based NLW methods have made remarkable progress on performance improvement of watermarking through generating substitutes for a masked target word. Yet, the BERT-based NLWs focus on the context of texts rather than the meaning of target words, which might make the capacity of watermark embeddings being lower. To address the limitations, this study proposes a novel NLW method by incorporating a paraphraser-based lexical substitution method. Under the promise of paraphrase preservation, the proposed NLW method utilizes the knowledge of paraphrase modeling to generate the substitute candidates to replace the words in original sentences capable of carrying the watermark signal in local contexts. We empirically show that our NLW method not only has a better meaning-preserved, but improves the payload more than 2 times compared with the BERT-based NLW method. Besides, compared with previous state-of-the-art method Compared with other state-of-the-art baselines, the experimental results show that the proposed LS method improves the Precision@1 score from 51.7% to 58.3% and from 50.5% to 62.6% on LS07 and CoInCo benchmarks, respectively. [ABSTRACT FROM AUTHOR]

In multi-view learning, there are a large number of small sample size problems in which the number of training samples is less than the dimension of feature vectors. For such problems, generalized multi-view linear discriminant analysis (GMLDA) usually fails to achieve good learning performance for many classification tasks. In this paper, we propose a new multi-view feature extraction method via fractional spectral modeling, namely, fractional generalized multi-view discriminant analysis (FGMDA), which is able to subsume GMLDA as a special case. In addition, an uncorrelated FGMDA (UFGMDA) is further proposed, where the extracted features are mutually uncorrelated in each view and thus have minimum redundancy. Extensive experimental results on visual recognition demonstrate the superiority of our proposed methods over the state-of-the-art methods in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

Short text classification has provoked a vast amount of attention and research in recent decades. However, most existing methods only focus on the short texts that contain dozens of words like Twitter and Microblog, while pay far less attention to the extreme short texts like news headline and search snippets. Meanwhile, contemporary short text classification methods that extend the features via external knowledge sources always introduce lots of useless concepts, which may be detrimental to classification performance. Moreover, unlike traditional short text classification methods, the classification results of extreme short texts are often determined by a few even one or two keywords. To address these problems, we propose a novel hybrid classification method via Keywords Screening and Attention Mechanisms in extreme short text, called KSAM. More specifically, firstly, the attention-based BiLSTM is introduced in our method to enhance the role of keywords. Secondly, we screen the keywords in the extreme short text for obtaining the true class label, and the concepts concerning the keywords are retrieved from external open knowledge sources like DBpedia. Thirdly, the attention mechanisms are introduced to acquire the weight of these retrieved concepts. Finally, conceptual information is utilized to assist the classification of the extreme short text. Extensive experiments have demonstrated the effectiveness of our method compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a novel representation l earning method DSML to achieve a better performance based on deep sparse auto encoder for Multi task Learning. • We propose a new stacked sparse auto encoder for feature reconstruction which can learn higher level and better representations by using the deep learning method and overcome the overfitting problem effectively. • The training of parameters in DSML has lower computational cost than other common deep learning methods. • Experimental studies of compared with seven representation l earning methods show that DSML is superior to traditional and state of the art representation l earning for Multi task Learning. We demonstrate an effective framework to achieve a better performance based on Deep Sparse auto-encoder for Multi-task Learning, called DSML for short. To learn the reconstructed and higher-level features on cross-domain instances for multiple tasks, we combine the labeled and unlabeled data from all tasks to reconstruct the feature representations. Furthermore, we propose the model of Stacked Reconstruction Independence Component Analysis (SRICA for short) for the optimization of feature representations with a large amount of unlabeled data, which can effectively address the redundancy of image data. Our proposed SRICA model is developed from RICA and is based on deep sparse auto-encoder. In addition, we adopt a Semi-Supervised Learning method (SSL for short) based on model parameter regularization to build a unified model for multi-task learning. There are several advantages in our proposed framework as follows: 1) The proposed SRICA makes full use of a large amount of unlabeled data from all tasks. It is used to pursue an optimal sparsity feature representation, which can overcome the over-fitting problem effectively. 2) The deep architecture used in our SRICA model is applied for higher-level and better representation learning, which is designed to train on patches for sphering the input data. 3) Training parameters in our proposed framework has lower computational cost compared to other common deep learning methods such as stacked denoising auto-encoders. Extensive experiments on several real image datasets demonstrate our proposed framework outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Over the past few decades, short text classification has emerged as a critical downstream task in natural language processing (NLP). One crucial classification research issue is how to advance semantic understanding considering the short length, feature sparsity, and high ambiguity in short texts. Recently, prompt-tuning has been proposed to insert a template into the input and convert text classification tasks into equivalent cloze-style tasks. However, among most of the previous approaches, either the crafted template methods are time-consuming and labor-intensive, or automatic prompt generation methods cannot achieve satisfied performance. In this paper, we introduce a novel approach called Soft Knowledgeable Prompt-tuning for short text classification. Our method considers both the template generation and classification performance to construct prompts for label prediction. We employ five different strategies to expand the label words space for modifying soft prompts, and the integration of these strategies is used as the final verbalizer. Despite being automatic, experimental results show that our method achieved more desirable performance even than the crafted template methods, outperforming the state-of-the-art by more than 14 Accuracy points on four well-known benchmarks. • Both the automatic prompt generation and text classification performance are achieved. • 5 different strategies are used to construct verbalizer effectively and efficiently. • The comprehensive experiments evaluate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

In the past decades, recommendation systems have provided lots of valuable personalized suggestions for the users to address the problem of information over-loaded. Collaborative Filtering (CF) is one of the most commonly applied and successful recommendation approaches, which refers to using the preferences of groups with similar interests to recommend information to other users. Recently, in addition to the traditional matrix factorization techniques, deep learning methods have been proposed to learn more abstract and higher-level representations for recommendation. However, most previous deep recommendation methods learn the higher-level feature representations of users and items through an identical model structure, which ignores the different characteristics of the user-based and item-based data. In addition, the rating matrix is usually sparse which may result in a significant degradation of recommendation performance. To address these problems, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation (CAPR for short). In this method, user-based and item-based feature representations are learned by two different autoencoders for capturing different features of the data. Meanwhile, items' attributions are combined into the feature representations with semi-autoencoder for alleviating the sparsity problem. Extensive experimental results confirm the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep recommendation methods. • Two different autoencoders are used to capture characteristics for users and items. • Manifold regularization is integrated into autoencoder for user's features learning. • The comprehensive experiments evaluate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Short-text clustering, which has attracted much attention with the rapid development of social media in recent decades, is a great challenge due to the feature sparsity, high ambiguity, and massive quantity. Recently, pre-trained language models (PLMs)-based methods have achieved fairly good results on this task. However, two main problems still hang in the air: (1) the significant gap of objective forms in pretraining and fine-tuning, which restricts taking full advantage of knowledge in PLMs. (2) Most existing methods require a post-processing operation for clustering label learning, potentially leading to label estimation errors for different data distributions. To address these problems, in this paper, we propose an Asymmetric Short-Text Clustering via Prompt (short for ASTCP), the features learned with our ASTCP are denser and constricted for clustering. Specifically, a subset text of the corpus is first selected by an asymmetric prompt-tuning network, which aims to obtain predicted label as a clustering center. Then, by the propagation of predicted-label information, a fine-tuned model is designed for representation learning. Thus, a clustering module, such as K-means, is built to directly output clustering labels on top of these representations. Extensive experiments conducted on three datasets have demonstrated that our ASTCP can significantly and consistently outperform other SOTA clustering methods. The source code is available at https://github.com/zhuyi_yzu/ASTCP. [ABSTRACT FROM AUTHOR]

Abstract: Conventionally, micro-sized irregular polyaniline (PANI) particles were synthesized by dropwise addition of the ammonium persulfate (APS) solution into the aniline (ANI) solution with mechanical stirring. By replacing the mechanical stirring with an ultrasonic irradiation, PANI nanofibers in diameters of ∼50nm and lengths of 200nm to several micrometers were prepared. Transmission electron microscopy (TEM) and scanning electron microscopy (SEM) showed that at the early stage of polymerization, the polymers formed in both the mechanical stirred and ultrasonicated systems are in the form of nanofiber. However, with continuing of the reaction, these primary nanofibers grow and agglomerate into irregular shaped PANI particles in the mechanical stirred system, while in the case of the ultrasonic irradiation, the growth and agglomeration are effectively prevented, preserving thus the PANI nanofibers in the final product. By increasing the APS/ANI molar ratio from 0.5 to 2.5, the aspect ratios of the PANI nanofibers decreased. The PANI nanofibers exhibit higher solubility than the irregular shaped PANI particles. Although the yield, as well as the conductivity of the ultrasonic synthesized PANI nanofibers, was slightly lower than the irregular shaped PANI particles, the ultrasonic synthesis approach is one of the facile and scalable approaches in synthesizing PANI nanofibers in comparison with other ones without use of templates (e.g., the interfacial polymerization and rapid mixing polymerization). UV–Vis and Fourier transformed infrared (FTIR) spectra indicated ultrasound had no significant effect on the chemical structure of the PANI. [Copyright &y& Elsevier]

The workflow of the proposed algorithm DEP-MSB. • The algorithm integrates multi-source biological information to improve the prediction accuracy. • The algorithm establishes a vertex attribute matrix to fuse topological and biological information attributes. • An optimization method is proposed to accelerate the algorithm convergence. • Experiment results show that the proposed method is more effectively and accurately. Mining essential protein is crucial for discovering the process of cellular organization and viability. At present, there are many computational methods for essential proteins detecting. However, these existing methods only focus on the topological information of the networks and ignore the biological information of proteins, which lead to low accuracy of essential protein identification. Therefore, this paper presents a new essential proteins prediction strategy, called DEP-MSB which integrates a variety of biological information including gene expression profiles, GO annotations, and Domain interaction strength. In order to evaluate the performance of DEP-MSB, we conduct a series of experiments on the yeast PPI network and the experimental results have shown that the proposed algorithm DEP-MSB is more superior to the other existing traditional methods and has obviously improvement in prediction accuracy. [ABSTRACT FROM AUTHOR]