Search

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

Feature selection refers to the process of identifying the most relevant and valuable features or characteristics in a dataset, and it is an important step that can improve the performance and accuracy of predictive models for machine learning and data mining. Feature selection is crucial in efficiently reducing the number of features, thereby enhancing classification accuracy. This, in turn, alleviates computational burdens and elevates the overall performance of machine learning algorithms. This paper extended the conventional manta ray foraging optimization (MRFO) algorithm to an improved binary MRFO (IBMFO) algorithm. IBMRFO serves as a search strategy for designing a wrapper-based feature selection approach. To commence, we introduce a novel mechanism, termed the chaotic tent map (CTM), aimed at initializing the population to fortify both the algorithm's exploitation capabilities and population diversity. Second, an adaptive somersault foraging (ASF) factor is tuned to avoid premature convergence. Finally, a binary mechanism is introduced to adapt the algorithm for binary feature selection dilemmas. Experiments are performed on 30 renowned UC Irvine machine learning repository datasets. The results of the student's t-test show that the classification algorithm has statistical significance. The findings unequivocally exhibit the superior performance of the proposed IBMRFO compared to other algorithms under scrutiny. Notably, the efficiency of the enhanced factors is assessed through rigorous testing. For post-publications supporting this work, readers can refer to https://github.com/ELF-CHEUNG/IBMRFO. • Novel chaotic tent map for population initialization from evolutionary computation. • Optimization methods based on enhanced evolutionary computation. • Adaptive factor enhances convergence rate and search efficiency of algorithm. • Extensive numerical and scientific simulations for verifications with UCI dataset. [ABSTRACT FROM AUTHOR]

In multiword expression (MWE) recognition, there exist many studies where different learning methods are employed to decide whether given word combination is a multiword expression. The recognition methods commonly utilize a number of features that are extracted from a data source, frequently from the given text. Though the recognition methods and the features are well studied, we believe that to achieve the best possible performance with a learning method, different subsets of features should also be considered and the best performing subset must be selected. In this paper, we propose a procedure that covers the performance comparison of well-known feature selection methods to obtain the best feature subset in MWE recognition. The evaluation tests are performed on a Turkish MWE data set and the performance is measured by precision, recall and F1 values. The highest F1 value =0.731 is obtained by C4.5 classifier employing either wrapper or filtering method in feature selection. In the regarding setting(s), it is examined that the performance is increased by 1.11% compared to the setting where all features are employed in classification. Based on the experimental results, it may be stated that feature selection improves the performance of MWE recognition by eliminating the noisy/non-effective features. Moreover, it is obvious that proposed feature selection method contributes to the overall MWE recognition system by reducing the measurement and storage requirements due to the lower number of features in classification, providing a faster and more-cost effective learning model. [ABSTRACT FROM AUTHOR]

In Data Mining, during the preprocessing step, there is a considerable diversity of candidate algorithms to select important features, according to some criteria. This broad availability of algorithms that perform the Feature Selection task gives rise to the difficulty of choosing, a priori , between the algorithms at hand, the most promising one for a particular problem. In this paper, we present the proposal and evaluation of a new architecture for the recommendation of Feature Selection algorithms based on the use of Metalearning. Our framework is very flexible since the user can adapt it to its proper needs. This flexibility is one of the main advantages of our proposal over other approaches in the literature, which involve steps that cannot be adapted to the user’s local requirements. Furthermore, it combines several concepts of intelligent systems, including Machine Learning and Data Mining, with topics derived from expert systems, as user and data-driven knowledge, with meta-knowledge. This set of solutions coupled with leading-edge technologies allows our architecture to be integrated into any information system, which impact on the automation of services and in reducing human effort during the process. Regarding the Metalearning process, our framework considers several types of properties inherent to the data sets, as well as, Feature Selection algorithms based on many information, distance, dependence and consistency measures. The quality of the methods for Feature Selection was estimated according to a multicriteria performance measure, which guided the ranking process of these algorithms for the construction of data metabases. Proposed by the authors of this work, this multicriteria performance measure combines any three measurements on a single one, creating an interesting and powerful tool to evaluate not only FS algorithms but also to assess any context where it is necessary a combination to maximize a measure or minimize it. The recommendation models, represented by decision trees and induced from the training metabases, allowed us to see in what circumstances a Feature Selection algorithm outperforms the other and what aspects of the data present greater influence in determining the performance of these algorithms. Nevertheless, if the user wishes, any other learning algorithm may be used to induce the recommendation model. This versatility is another strong point of this proposal. Results show that with the characterization of data, through statistical, information and complexity measures, it is possible to reach an accuracy higher than 90%. Besides yielding recommendation models that are interpretable and robust to overfitting, the developed architecture is less computationally expensive than approaches recently proposed in the literature. [ABSTRACT FROM AUTHOR]

Feature selection is a crucial preprocessing technique in data mining and machine learning and has attracted increasing attentions. However, the relevance of existing methods only contains limited information. To enhance the classification ability of filter and information theory-based feature selection methods and reduce the redundancy of the selected subset, a relevant-redundant weight-based feature criterion (FSRRW) is proposed. In this paper, a feature relevant-redundant weight (RRW) is constructed to extract the important relevant and redundant information. Then, a novel feature relevance is defined based on the weight, which contains more comprehensive information from the dynamically changing features. Additionally, a feature evaluation criterion is presented via maximizing the feature relevance and minimizing the feature redundancy. The proposed algorithm and seven compared methods are tested on 20 benchmark datasets. Extensive experiments demonstrate that the proposed criterion exhibits better feature screening abilities, effectively facilitates classification, and has preferable applicability and robustness. • Design a feature relevant-redundant weight and define feature relevance. • Construct a novel feature evaluation criterion named FSRRW. • FSRRW is evaluated on 20 benchmark datasets under three classifiers. • The proposed method outperforms seven popular feature selection methods. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

• A data-driven product failure detection framework was developed. • Sampling strategies improved the performance of the models. • ANOVA selection technique improved the performance. • The RUSBoosted Tree algorithm provided satisfactory results. • The proposed methodology is flexible to be applied in other sectors. For a healthy production line, it is essential to ensure a low failure rate of products. Product quality in production lines can be inspected using several techniques at the end of a production process, including a manual inspection. Different methods are applied to inspect the product quality at the end of the production process and sometimes during the production. This is often done using manual inspection, but this is less efficient, expensive, and time-consuming. Machine learning algorithms have the potential for evaluating and predicting product quality in a production line. In this paper, a novel product failure detection model that applies ANOVA (Analysis of Variance) feature selection method, Min-Max Scaling normalization method, mean imputation technique, Random Forest classification algorithm, a data sampling technique, and Grid Search parameter optimization approach is proposed and validated. For the comparison of the proposed model, several experiments have been performed using five classification algorithms, including RUSBoosted Tree. Experimental results demonstrated that the proposed model using the Random Forest algorithm, ANOVA feature selection, and sampling method achieves the best performance among other models and detects the faulty products effectively. It was also shown that the RUSBoosted Tree algorithm can be considered by practitioners for building the faulty product prediction model when data sampling and feature selection techniques are not integrated into the prediction model. [ABSTRACT FROM AUTHOR]

In the past two decades, the dimensionality of datasets involved in machine learning and data mining applications has increased explosively. Therefore, feature selection has become a necessary step to make the analysis more manageable and to extract useful knowledge about a given domain. A large variety of feature selection techniques are available in literature, and their comparative analysis is a very difficult task. So far, few studies have investigated, from a theoretical and/or experimental point of view, the degree of similarity/dissimilarity among the available techniques, namely the extent to which they tend to produce similar results within specific application contexts. This kind of similarity analysis is of crucial importance when two or more methods are combined in an ensemble fashion: indeed the ensemble paradigm is beneficial only if the involved methods are capable of giving different and complementary representations of the considered domain. This paper gives a contribution in this direction by proposing an empirical approach to evaluate the degree of consistency among the outputs of different selection algorithms in the context of high dimensional classification tasks. Leveraging on a proper similarity index, we systematically compared the feature subsets selected by eight popular selection methods, representatives of different selection approaches, and derived a similarity trend for feature subsets of increasing size. Through an extensive experimentation involving sixteen datasets from three challenging domains (Internet advertisements, text categorization and micro-array data classification), we obtained useful insight into the pattern of agreement of the considered methods. In particular, our results revealed how multivariate selection approaches systematically produce feature subsets that overlap to a small extent with those selected by the other methods. [ABSTRACT FROM AUTHOR]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

• Feature selection improves performance of machine learning algorithms. • Feature selection with more n-tier techniques is simpler and more stable. • A feature selection model that is not specific to any data set is widely applied. Feature Selection and classification have previously been widely applied in various areas like business, medical and media fields. High dimensionality in datasets is one of the main challenges that has been experienced in classifying data, data mining and sentiment analysis. Irrelevant and redundant attributes have also had a negative impact on the complexity and operation of algorithms for classifying data. Consequently, the algorithms record poor results or performance. Some existing work use all attributes for classification, some of which are insignificant for the task, thereby leading to poor performance. This paper therefore develops a hybrid filter model for feature selection based on principal component analysis and information gain. The hybrid model is then applied to support classification using machine learning techniques e.g. the Naïve Bayes technique. Experimental results demonstrate that the hybrid filter model reduces data dimensions, selects appropriate feature sets, and reduces training time, hence providing better classification performance as measured by accuracy, precision and recall.. [ABSTRACT FROM AUTHOR]

• A new unsupervised feature selection method is proposed for heterogeneous dataset. • Relevant and non-redundant features are selected without prior data transformation. • The proposed algorithm is suitable for high dimensional data. • The proposed algorithm is scalable with respect to any size of data. • Rigorous comparative study carried out to prove efficacy of proposed mechanism. Unsupervised feature selection (UFS) is utilized in various application domains, such as data mining, pattern recognition, machine learning, etc. UFS follows three basic approaches, namely filter, wrapper, and hybrid (that is, a combination of both filter and wrapper) to select the relevant and non-redundant features. It has been observed that a filter method does not guarantee an optimal solution. However, a wrapper approach is computationally expensive. The hybrid method are known to give a better trade-off between filter and wrapper strategies. But, the practical applicability of schemes mentioned above are preferably restricted only to a numerical dataset and are not so suitable for a mixed dataset. Therefore, there is a need for a UFS scheme which can handle both the numerical and non-numerical features directly. In this paper, a robust and efficient two-phase (i.e., feature ranking (FR) and feature selection (FS)) UFS method is proposed. The proposed FR utilizes entropy and mutual information to produce maximum informative and non-redundant ranked features from a high-dimensional mixed dataset. Further, the proposed FS follows k-prototype clustering algorithm with improved Callinski-Harasbaz criteria-based selection methodology to choose optimal features. Experiments on real-life dataset substantiate that the proposed approach provides a better subset of features compared to the existing state of the art approaches. [ABSTRACT FROM AUTHOR]