Search

Relief-like algorithms have been widely used as feature selection to reduce the dimension of high-dimensional data which involves thousands of irrelevant variables because of their low computational cost and high accuracy. Classical Relief algorithms have not exactly shown the dynamic procedure that updates weight iteratively. This paper proposes an innovative feature weight estimation method, called dynamic representation and neighbor sparse reconstruction-based Relief (DRNSR-Relief). Similar to the classical Relief algorithms, the goal of DRNSR-Relief is to maximize the expected margin in the weighted feature space. A dynamic representation framework is introduced to show the dynamic relationship between the expected margin vector and the weight vector. To achieve better neighbor reconstruction, DRNSR-Relief decomposes a nonlinear problem into a set of locally linear ones through local hyperplane with l 1 regularization and then estimates feature weights in a large margin framework. With the help of gradient ascent method, we can guarantee the convergence of DRNSR-Relief. To demonstrate the validity and the effectiveness of our formulation for feature selection in supervised learning, we perform extensive experiments on synthetic and real-world datasets. Experimental results indicate that DRNSR-Relief is very promising. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) is a popular machine learning method which can flexibly simulate the relationships of real-world classification applications. When facing problems ( i.e. , data sets) with a smaller number of samples ( i.e. , instances), ELM may often result in the overfitting trouble. In this paper, we propose a new Instance Cloned Extreme Learning Machine (IC-ELM for short) which can handle numerous different classification problems. IC-ELM uses an instance cloning method to balance the input data’s distribution and extend the training data set, which alleviates the overfitting issue and enhances the testing classification accuracy. Experiments and comparisons on 20 UCI data sets, and validations on image and text classification applications, demonstrate that IC-ELM is able to achieve superior results compared to the original ELM algorithm and its variants, as well as several other classical machine learning algorithms. [ABSTRACT FROM AUTHOR]

A popular procedure for identifying which one of two classification algorithms has a better performance is to test them on multiple data sets, and the accuracies resulting from k-fold cross validation are aggregated to draw a conclusion. Several nonparametric methods have been proposed for this purpose, while parametric methods will be a better choice to determine the superior algorithm when the assumptions for deriving sampling distributions can be satisfied. In this paper, we consider every accuracy estimate resulting from the instances in a fold or a data set as a point estimator instead of a fixed value to derive the sampling distribution of the point estimator for comparing the performance of two classification algorithms. The test statistics for both data-set and fold averaging levels are proposed, and the ways to calculate their degrees of freedom are also presented. Twelve data sets are chosen to demonstrate that our parametric methods can be used to effectively compare the performance of two classification algorithms on multiple data sets. Several critical issues in using our parametric methods and the nonparametric ones proposed in a previous study are then discussed. [ABSTRACT FROM AUTHOR]

The large-scale images and videos are one kind of the main source of big data. Dynamic texture (DT) is essential for understanding the video sequences with spatio-temporal similarities. This paper presents a powerful tool called dynamic fractal analysis to DT description and classification, which integrates rich description of DT with strong robustness to environmental changes. The proposed dynamic fractal spectrum (DFS) for DT sequences is composed of two components. The first one is a volumetric dynamic fractal spectrum component (V-DFS) that captures the stochastic self-similarities of DT sequences by treating them as 3D volumes; the second one is a multi-slice dynamic fractal spectrum component (S-DFS) that encodes fractal structures of repetitive DT patterns on 2D slices along different views of the 3D volume. To fully exploit various types of dynamic patterns in DT, five measurements of DT pixels are collected for the analysis on DT sequences from different perspectives. We evaluated our method on four publicly available benchmark datasets. All the experimental results have demonstrated the excellent performance of our method in comparison with state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Feature selection is an important part of classifier design. There are many possible methods for searching and evaluating feature subsets, but little consensus on which methods are best. This paper examines a number of filter-based feature subset evaluation measures with the goal of assessing their performance with respect to specific classifiers. This work tests 16 common filter measures for use with K -nearest neighbors and support vector machine classifiers. The measures are tested on 20 real and 20 artificial data sets, which are designed to probe for specific challenges. The strengths and weaknesses of each measure are discussed with respect to the specific challenges and correlation with classifier accuracy. The results highlight several challenging problems with a number of common filter measures. The results indicate that the best filter measure is classifier-specific. K -nearest neighbors classifiers work well with subset-based RELIEF, correlation feature selection or conditional mutual information maximization, whereas Fisher׳s interclass separability criterion and conditional mutual information maximization work better for support vector machines. Despite the large number and variety of feature selection measures proposed in the literature, no single measure is guaranteed to outperform the others, even within a single classifier, and the overall performance of a feature selection method cannot be characterized independently of the subsequent classifier. [ABSTRACT FROM AUTHOR]

Abstract: Classification with imbalanced data-sets has become one of the most challenging problems in Data Mining. Being one class much more represented than the other produces undesirable effects in both the learning and classification processes, mainly regarding the minority class. Such a problem needs accurate tools to be undertaken; lately, ensembles of classifiers have emerged as a possible solution. Among ensemble proposals, the combination of Bagging and Boosting with preprocessing techniques has proved its ability to enhance the classification of the minority class. In this paper, we develop a new ensemble construction algorithm (EUSBoost) based on RUSBoost, one of the simplest and most accurate ensemble, which combines random undersampling with Boosting algorithm. Our methodology aims to improve the existing proposals enhancing the performance of the base classifiers by the usage of the evolutionary undersampling approach. Besides, we promote diversity favoring the usage of different subsets of majority class instances to train each base classifier. Centered on two-class highly imbalanced problems, we will prove, supported by the proper statistical analysis, that EUSBoost is able to outperform the state-of-the-art methods based on ensembles. We will also analyze its advantages using kappa-error diagrams, which we adapt to the imbalanced scenario. [Copyright &y& Elsevier]

• Feature Selection (FS) plays an important role in learning and classification tasks. • Large-scale online feature selection problems with big data. • Intelligent online feature selection system; Analysis step in a pattern recognition system. • Exploring the recent advances of online machine learning techniques and a conflict resolution technique (Automated Negotiation). • Enhancing the classification performance of ultra-high dimensional databases. Feature Selection (FS) plays an important role in learning and classification tasks. Its objective is to select the relevant and non-redundant features. Considering the huge number of features in real-world applications, FS methods using batch learning technique cannot resolve big data problems especially when data arrive sequentially. In this paper, we proposed an online feature selection system which resolves this problem. The proposed OFS system called MOANOFS (Multi-Objective Automated Negotiation based Online Feature Selection) explore the recent advances of online machine learning techniques and a conflict resolution technique (Automated Negotiation) for the purpose of enhancing the classification performance of ultra-high dimensional databases. MOANOFS uses two decision levels. In the first level, we decided which k(s) among the learners (or OFS methods) are the trustful ones (with high confidence or trust value). These elected k learners would participate in the second level where we integrated our proposed Multilateral Automated Negotiation based OFS (MANOFS) method. This would enable us to finally decide which features are the most relevant. We showed that MOANOFS system achieves high accuracy with several real text classification datasets as 20Newsgroups, RCV1. [ABSTRACT FROM AUTHOR]