Search

Relief-like algorithms have been widely used as feature selection to reduce the dimension of high-dimensional data which involves thousands of irrelevant variables because of their low computational cost and high accuracy. Classical Relief algorithms have not exactly shown the dynamic procedure that updates weight iteratively. This paper proposes an innovative feature weight estimation method, called dynamic representation and neighbor sparse reconstruction-based Relief (DRNSR-Relief). Similar to the classical Relief algorithms, the goal of DRNSR-Relief is to maximize the expected margin in the weighted feature space. A dynamic representation framework is introduced to show the dynamic relationship between the expected margin vector and the weight vector. To achieve better neighbor reconstruction, DRNSR-Relief decomposes a nonlinear problem into a set of locally linear ones through local hyperplane with l 1 regularization and then estimates feature weights in a large margin framework. With the help of gradient ascent method, we can guarantee the convergence of DRNSR-Relief. To demonstrate the validity and the effectiveness of our formulation for feature selection in supervised learning, we perform extensive experiments on synthetic and real-world datasets. Experimental results indicate that DRNSR-Relief is very promising. [ABSTRACT FROM AUTHOR]

Feature selection is an important part of classifier design. There are many possible methods for searching and evaluating feature subsets, but little consensus on which methods are best. This paper examines a number of filter-based feature subset evaluation measures with the goal of assessing their performance with respect to specific classifiers. This work tests 16 common filter measures for use with K -nearest neighbors and support vector machine classifiers. The measures are tested on 20 real and 20 artificial data sets, which are designed to probe for specific challenges. The strengths and weaknesses of each measure are discussed with respect to the specific challenges and correlation with classifier accuracy. The results highlight several challenging problems with a number of common filter measures. The results indicate that the best filter measure is classifier-specific. K -nearest neighbors classifiers work well with subset-based RELIEF, correlation feature selection or conditional mutual information maximization, whereas Fisher׳s interclass separability criterion and conditional mutual information maximization work better for support vector machines. Despite the large number and variety of feature selection measures proposed in the literature, no single measure is guaranteed to outperform the others, even within a single classifier, and the overall performance of a feature selection method cannot be characterized independently of the subsequent classifier. [ABSTRACT FROM AUTHOR]

• Feature Selection (FS) plays an important role in learning and classification tasks. • Large-scale online feature selection problems with big data. • Intelligent online feature selection system; Analysis step in a pattern recognition system. • Exploring the recent advances of online machine learning techniques and a conflict resolution technique (Automated Negotiation). • Enhancing the classification performance of ultra-high dimensional databases. Feature Selection (FS) plays an important role in learning and classification tasks. Its objective is to select the relevant and non-redundant features. Considering the huge number of features in real-world applications, FS methods using batch learning technique cannot resolve big data problems especially when data arrive sequentially. In this paper, we proposed an online feature selection system which resolves this problem. The proposed OFS system called MOANOFS (Multi-Objective Automated Negotiation based Online Feature Selection) explore the recent advances of online machine learning techniques and a conflict resolution technique (Automated Negotiation) for the purpose of enhancing the classification performance of ultra-high dimensional databases. MOANOFS uses two decision levels. In the first level, we decided which k(s) among the learners (or OFS methods) are the trustful ones (with high confidence or trust value). These elected k learners would participate in the second level where we integrated our proposed Multilateral Automated Negotiation based OFS (MANOFS) method. This would enable us to finally decide which features are the most relevant. We showed that MOANOFS system achieves high accuracy with several real text classification datasets as 20Newsgroups, RCV1. [ABSTRACT FROM AUTHOR]