Search

Feature selection (FS) is an optimisation problem that reduces the dimension of the dataset and increases the performance of the machine learning algorithms and classification through the selection of the optimal subset features and elimination of the redundant features. However, the huge search space is an important challenge in the FS problem. Due to their satisfactory capabilities to handle high-dimension search spaces, meta-heuristic search algorithms have recently gained much attention and become popular in the FS problem. For these algorithms, choosing a proper fitness function plays an important role. The fitness function orients the searching strategy of the algorithms to obtain best solutions. Appropriate fitness functions will help the algorithms with exploring the search space more effectively and efficiently. In this work, firstly the efficiency of two of the most outstanding and successful heuristic algorithms in the FS domain, namely genetic algorithm (GA) and grey wolf optimiser (GWO), are investigated and analysed with a single-objective fitness function. Secondly, two recent feature selection techniques based on GA and GWO, namely feature selection, weight, and parameter optimisation (FWP) and binary GWO (BGWO) with their fitness function are investigated and analysed. Thirdly, in order to remove the detected drawbacks and weaknesses of the FS algorithms and to enhance their efficiency, a new multi-objective weighted fitness function based on multiple predominant criteria has been presented. The effectiveness of the proposed fitness function on the FS algorithms is evaluated by using SVM and associative classification on 11 different large and small datasets. The experimental results show the superiority of proposed fitness function (where features were reduced and the classification performance has been improved) over single-objective fitness function and other existing fitness functions. Furthermore, another key aim of this study is to present a comprehensive study about the strengths and weaknesses of the FS algorithms which can be used as guidelines for future possible works to more improve the developments of these algorithms. [ABSTRACT FROM AUTHOR]

Feature transformation methods can be used effectively to improve the performance of the machine learning techniques, such as classification and clustering, via enhancing the discrimination among the samples. Additionally, binary representation of features may be necessary in several circumstances. For example, distributed systems may have limited bandwidth, storage, and energy that necessitate rough quantization of the measurements. This paper proposes a random projection-based feature transformation method that maps the data points from the original space into a new binary space. In the proposed transformation method, the random projection process is formulated as an optimization problem. A modified chaotic version of the sine–cosine algorithm is used to find the optimal random projector. The performance of the proposed transformation method is validated using 15 standard UCI benchmark datasets against the linear discernment analysis. Moreover, it is compared to other metaheuristic-based feature reduction methods including the gray wolf optimizer, the genetic algorithm, and the antlion optimizer. The obtained results assure the superiority of the proposed transformation method in terms of classification accuracy and storage requirements. [ABSTRACT FROM AUTHOR]

The performance of a model in machine learning problems highly depends on the dataset and training algorithms. Choosing the right training algorithm can change the tale of a model. While some algorithms have a great performance in some datasets, they may fall into trouble in other datasets. Moreover, by adjusting hyperparameters of an algorithm, which controls the training processes, the performance can be improved. This study contributes a method to tune hyperparameters of machine learning algorithms using Grey Wolf Optimization (GWO) and Genetic algorithm (GA) metaheuristics. Also, 11 different algorithms including Averaged Perceptron, FastTree, FastForest, Light Gradient Boost Machine (LGBM), Limited memory Broyden Fletcher Goldfarb Shanno algorithm Maximum Entropy (LbfgsMxEnt), Linear Support Vector Machine (LinearSVM), and a Deep Neural Network (DNN) including four architectures are employed on 11 datasets in different biological, biomedical, and nature categories such as molecular interactions, cancer, clinical diagnosis, behavior related predictions, RGB images of human skin, and X-rays images of Covid19 and cardiomegaly patients. Our results show that in all trials, the performance of the training phases is improved. Also, GWO demonstrates a better performance with a p-value of 2.6E-5. Moreover, in most experiment cases of this study, the metaheuristic methods demonstrate better performance and faster convergence than Exhaustive Grid Search (EGS). The proposed method just receives a dataset as an input and suggests the best-explored algorithm with related arguments. So, it is appropriate for datasets with unknown distribution, machine learning algorithms with complex behavior, or users who are not experts in analytical statistics and data science algorithms. [Display omitted] • The proposed method can improve the performance of the machine learning model independent from the composition of datasets. • It assists to gain a better performance faster than tuning by blindly and randomly chosen hyperparameters. • The idea of this method is general and it is not specific to a limited number of training algorithms and optimizers. • This method is convenient to use for users who are not experts in algorithm designing, also for real-world problems. • Generally, metaheuristic methods give better result and converge faster than blind approaches like Exhaustive Grid Search. [ABSTRACT FROM AUTHOR]