Search

Dimensionality reduction is an important and challenging task in machine learning and data mining. It can facilitate data clustering, classification and information retrieval. As an efficient technique for dimensionality reduction, feature selection is about finding a small feature subset preserving the most relevant information. In this paper, we propose a new criterion, called maximum projection and minimum redundancy feature selection, to address unsupervised learning scenarios. First, the feature selection is formalized with the use of the projection matrices and then characterized equivalently as a matrix factorization problem. Second, an iterative update algorithm and a greedy algorithm are proposed to tackle this problem. Third, kernel techniques are considered and the corresponding algorithm is also put forward. Finally, the proposed algorithms are compared with four state-of-the-art feature selection methods. Experimental results reported for six publicly datasets demonstrate the superiority of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Dimensionality reduction is an important and challenging task in machine learning and data mining. Feature selection and feature extraction are two commonly used techniques for decreasing dimensionality of the data and increasing efficiency of learning algorithms. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper, we propose a new unsupervised feature selection criterion developed from the viewpoint of subspace learning, which is treated as a matrix factorization problem. The advantages of this work are four-fold. First, dwelling on the technique of matrix factorization, a unified framework is established for feature selection, feature extraction and clustering. Second, an iterative update algorithm is provided via matrix factorization, which is an efficient technique to deal with high-dimensional data. Third, an effective method for feature selection with numeric data is put forward, instead of drawing support from the discretization process. Fourth, this new criterion provides a sound foundation for embedding kernel tricks into feature selection. With this regard, an algorithm based on kernel methods is also proposed. The algorithms are compared with four state-of-the-art feature selection methods using six publicly available datasets. Experimental results demonstrate that in terms of clustering results, the proposed two algorithms come with better performance than the others for almost all datasets we experimented with here. [ABSTRACT FROM AUTHOR]

Like the traditional machine learning, the multi-label learning is faced with the curse of dimensionality. Some feature selection algorithms have been proposed for multi-label learning, which either convert the multi-label feature selection problem into numerous single-label feature selection problems, or directly select features from the multi-label data set. However, the former omit the label dependency, or produce too many new labels leading to learning with significant difficulties; the latter, taking the global label dependency into consideration, usually select a few redundant or irrelevant features, because actually not all labels depend on each other, which may confuse the algorithm and degrade its classification performance. To select a more relevant and compact feature subset as well as explore the label dependency, a granular feature selection method for multi-label learning is proposed with a maximal correlation minimal redundancy criterion based on mutual information. The maximal correlation minimal redundancy criterion makes sure that the selected feature subset contains the most class-discriminative information, while in the meantime exhibits the least intra-redundancy. Granulation can help explore the label dependency. We study the relation of the label granularity and the performance on four data sets, and compare the proposed method with other three multi-label feature selection methods. The experimental results demonstrate that the proposed method can select compact and specific feature subsets, improve the classification performance and performs better than other three methods on the widely-used multi-label learning evaluation criteria. [ABSTRACT FROM AUTHOR]

Restricted Boltzmann machines and deep belief networks have been shown to perform effectively in many applications such as supervised and unsupervised learning, dimensionality reduction and feature learning. Implementing networks, which use contrastive divergence as the learning algorithm on neuromorphic hardware, can be beneficial for real-time hardware interfacing, power efficient hardware and scalability. Neuromorphic hardware which uses memristors as synapses is one of the most promising areas to achieve the above-mentioned goals. This paper presents a restricted Boltzmann machine which uses a two memristor model to emulate synaptic weights and achieves learning using contrastive divergence. [ABSTRACT FROM AUTHOR]

Abstract: Feature selection (attribute reduction) from large-scale incomplete data is a challenging problem in areas such as pattern recognition, machine learning and data mining. In rough set theory, feature selection from incomplete data aims to retain the discriminatory power of original features. To address this issue, many feature selection algorithms have been proposed, however, these algorithms are often computationally time-consuming. To overcome this shortcoming, we introduce in this paper a theoretic framework based on rough set theory, which is called positive approximation and can be used to accelerate a heuristic process for feature selection from incomplete data. As an application of the proposed accelerator, a general feature selection algorithm is designed. By integrating the accelerator into a heuristic algorithm, we obtain several modified representative heuristic feature selection algorithms in rough set theory. Experiments show that these modified algorithms outperform their original counterparts. It is worth noting that the performance of the modified algorithms becomes more visible when dealing with larger data sets. [Copyright &y& Elsevier]

Estimating interest rates is a typical multivariate prediction problem that has garnered considerable attention in the finance industry. However, the rising complexity of the prediction environment has presented significant obstacles to the methodology and application of traditional single-scale rational prediction models. Considering this challenge, the paper presents a new multivariate fusion prediction system that aims to tackle the following issues: (1) How to mine prediction information from multi-scale information systems (MSISs)? (2) How to describe the irrational behavior of decision-makers (DMs) in prediction processes? (3) How to improve the prediction performance and generalization capability of the models? Specifically, the score function of each feature under MSISs is primarily established by applying scale rules, regret theory (RT), and dominance relations to thoroughly explore prediction information across varying scales. Second, an adaptive S3WD model is developed for feature selection, which leverages the feature score distribution and the sequential three-way decision (S3WD) concept to avoid the problem of dimensional disasters, thereby improving the accuracy of selected features and reducing the learning cost. Furthermore, to enhance the effectiveness and rationality of the kernel extreme learning machine (KELM), an improved version known as IKELM is developed, which combines the regret-based comprehensive score of each sample. Finally, a multivariate fusion prediction system is constructed and abbreviated as an S3WD-IKELM-PSO system, where the particle swarm optimization (PSO) algorithm is employed to determine optimal parameters. It is worth noting that actual interest rate estimation datasets are utilized to verify the feasibility and validity of the proposed prediction system. According to the experimental results, the S3WD-IKELM-PSO system exhibits superior prediction performance and generalization ability compared to existing machine learning models. • A regret theory-based multivariate prediction system in MSISs is explored. • A regret-based feature evaluation system in an MSIS is designed. • An adaptive S3WD model for feature selection is proposed. • An improved KELM model is proposed. [ABSTRACT FROM AUTHOR]

As we aim at alleviating the curse of high-dimensionality, subspace learning is becoming more popular. Existing approaches use either information about global or local structure of the data, and few studies simultaneously focus on global and local structures as the both of them contain important information. In this paper, we propose a global and local structure preserving sparse subspace learning (GLoSS) model for unsupervised feature selection. The model can simultaneously realize feature selection and subspace learning. In addition, we develop a greedy algorithm to establish a generic combinatorial model, and an iterative strategy based on an accelerated block coordinate descent is used to solve the GLoSS problem. We also provide whole iterate sequence convergence analysis of the proposed iterative algorithm. Extensive experiments are conducted on real-world datasets to show the superiority of the proposed approach over several state-of-the-art unsupervised feature selection approaches. [ABSTRACT FROM AUTHOR]

Abstract: Measures of relevance between features play an important role in classification and regression analysis. Mutual information has been proved an effective measure for decision tree construction and feature selection. However, there is a limitation in computing relevance between numerical features with mutual information due to problems of estimating probability density functions in high-dimensional spaces. In this work, we generalize Shannon’s information entropy to neighborhood information entropy and propose a measure of neighborhood mutual information. It is shown that the new measure is a natural extension of classical mutual information which reduces to the classical one if features are discrete; thus the new measure can also be used to compute the relevance between discrete variables. In addition, the new measure introduces a parameter delta to control the granularity in analyzing data. With numeric experiments, we show that neighborhood mutual information produces the nearly same outputs as mutual information. However, unlike mutual information, no discretization is required in computing relevance when used the proposed algorithm. We combine the proposed measure with four classes of evaluating strategies used for feature selection. Finally, the proposed algorithms are tested on several benchmark data sets. The results show that neighborhood mutual information based algorithms yield better performance than some classical ones. [Copyright &y& Elsevier]