Text Categorization by Weighted Features

Text representation, which represent texts of variable lengths into feature vectors, is a fundamental task in machine learning. Early approaches extract lexical and syntactic features, which are discrete and sparse and can hardly capture the semantic relatedness between words. Recent advances in deep learning, which represents text segments into dense and continuous vectors, has shed light on this problem. However, the main limitation is they are usually based on complex neural network structure, which are resource-consuming to train and make inference. To remedy this, we propose a novel text representation approach which considers word local and global importance via the BM25 weighing schema. We use word vectors pretrained from large text corpus to capture the latent semantic relatedness between words. Experimental results show that our approach is effective and efficient compared with existing feature-based, unsupervised and supervised baselines.