Search

The neural activities of the brain are detected by Electroencephalography (EEG) that allows for analysis and classification of epileptic disease. The existing methods fail to capture high-dimensional data between adjacent sequences which make it difficult for the classifier to process and maximize the classification errors. This research proposes Hierarchical Long Short-Term Memory (H-LSTM) with a skip connection-based epileptic seizure classification method. The H- LSTM captures both short and long-term dependencies between adjacent sequences in the high-dimensional data. The skip connection is introduced between H- LSTM layers, facilitating the flow of data across adjacent sequences to improve the classification performance of epileptic seizures. The datasets used to evaluate the proposed H- LSTM with skip connection-based classification method are BONN-EEG and CHB-MIT EEG. The proposed H-LSTM with skip connection method attains 99.81% accuracy on BONN - EEG while attaining 99.34% accuracy on the CHB-MIT EEG dataset which is more effective than the existing methods namely, Bidirectional Gated Recurrent Unit (Bi-GRU) and Graph Convolutional Network (GCN). [ABSTRACT FROM AUTHOR]

The use of feature selection approaches is frequently required for handling high-dimensional datasets. Applying the particle swarm optimization (PSO) algorithm to high-dimensional datasets with thousands of features is still a major challenge in data mining and machine learning. Considering the limitations of the feature model, the key difficulty is choosing usable features. To address this challenge, a two-phase approach is suggested to optimize the feature selection used for high-dimensional data classification. First, the feature ranking metric (FR) is used to select the relevant and non-redundant feature set suitable for classification. In the second phase, the geometric particle swarm optimization (GPSO) algorithm is used to find the most informative feature subset from the original attributes using the selected features acquired from the first phase. To further improve the classification accuracy and to make the proposed approach more accurate, a new fitness function is designed with classification accuracy, feature relevancy, and feature reduction rate to calculate the goodness of selected features. The efficiency of FR-GPSO is evaluated on five microarray datasets and six benchmark datasets from the UCI machine learning repository, evaluated by a k-nearest neighbor (kNN) classifier with 10-fold cross-validation and compared with the commonly used feature selection methods. The results show that the proposed method not only selects a smaller number of attributes but also increases the classification accuracy compared with other approaches. Furthermore, the statistical test shows that the proposed method is statistically significant over the other competitors' algorithms. [ABSTRACT FROM AUTHOR]

In the paradigm of online active classification, the learner not only has to predict the label of each incoming instance, but also must decide whether the true label of that instance should be supplied, or not. The overall goal is to minimize the number of prediction mistakes with few label queries. In this paper, we focus on a novel framework for online active learning, with the aim of handling high dimensional classification problems. The key component of our framework is to exploit both the margin-based predictive uncertainty and the feature-based discriminative information of the current instance, in order to determine whether it should be labeled. Based on this labeling strategy, we propose several online active learning algorithms, for both binary classification tasks and multiclass ones. For these algorithms, which use adaptive subgradient methods for updating their linear model, expected mistake bounds are provided. Experiments on high-dimensional (binary and multiclass) classification datasets reveal the benefit of our label query strategy, and show the superiority of our algorithms over the existing ones. [ABSTRACT FROM AUTHOR]

Objective: Nowadays, "high dimension low sample size (HDLSS) settings" is very popular in many areas such as genetics, bioinformatics, medical imaging. In this study, it was aimed to investigate the classification performance of Random Ferns, which is a relatively new classifier, on HDLSS settings with different characteristics. Material and Methods: By simulation studies, artificial data sets that have different characteristics in terms of dimension, sample size, correlation structure, noise ratio and prevalence, were generated. Each scenario was iterated for 1,000 times and, classification performances of Random Ferns has been compared with support vector machines (SVM), which stand out with its high classification performance. Results: The performance variation of Random Ferns differs from SVM. It performed better at small sample size (n=20). When the F values are examined, it is seen that the classification performance of Random Ferns in the case of imbalanced distribution does not change much according to the balanced distribution situation and is higher than SVM. It has also been observed that high success as was expected in the classical data structure cannot be achieved. Conclusion: It is noteworthy that Random Ferns outperforms, especially in balanced distribution and small sample sizes. It is thought that this method, which does not have many applications in the field of health, will contribute to studies where the number of observations is quite low. [ABSTRACT FROM AUTHOR]

Selecting a subset of candidate features is one of the important steps in the data mining process. The ultimate goal of feature selection is to select an optimal number of high-quality features that can maximize the performance of the learning algorithm. However, this problem becomes challenging when the number of features increases in a dataset. Hence, advanced optimization techniques are used these days to search for the optimal feature combinations. Whale Optimization Algorithm (WOA) is a recent metaheuristic that has successfully applied to different optimization problems. In this work, we propose a new variant of WOA (SBWOA) based on spatial bounding strategy to play the role of finding the potential features from the high-dimensional feature space. Also, a simplified version of SBWOA is introduced in an attempt to maintain a low computational complexity. The effectiveness of the proposed approach was validated on 16 high-dimensional datasets gathered from Arizona State University, and the results are compared with the other eight state-of-the-art feature selection methods. Among the competitors, SBWOA has achieved the highest accuracy for most datasets such as TOX_171, Colon, and Prostate_GE. The results obtained demonstrate the supremacy of the proposed approaches over the comparison methods. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of classification of high dimensional queries to high dimensional classes from discrete alphabets where the probabilistic model that relates data to the classes is known. This problem has applications in various fields including the database search problem in mass spectrometry. The problem is analogous to the nearest neighbor search problem, where the goal is to find the data point in a database that is the most similar to a query point. The state of the art method for solving an approximate version of the nearest neighbor search problem in high dimensions is locality sensitive hashing (LSH). LSH is based on designing hash functions that map near points to the same buckets with a probability higher than random (far) points. To solve our high dimensional classification problem, we introduce distribution sensitive hashes that map jointly generated pairs to the same bucket with probability higher than random pairs. We design distribution sensitive hashes using a forest of decision trees and we analytically derive the complexity of search. We further show that the proposed hashes perform faster than state of the art approximate nearest neighbor search methods for a range of probability distributions, in both theory and simulations. Finally, we apply our method to the spectral library search problem in mass spectrometry, and show that it is an order of magnitude faster than the state of the art methods. [ABSTRACT FROM AUTHOR]

Background: Chronic kidney disease (CKD), characterized by progressive loss of renal function, is becoming a growing problem in the general population. New analytical technologies such as “omics”-based approaches, including metabolomics, provide a useful platform for biomarker discovery and improvement of CKD management. In metabolomics studies, not only prediction accuracy is attractive, but also variable importance is critical because the identified biomarkers reveal pathogenic metabolic processes underlying the progression of chronic kidney disease. We aimed to use k-important neighbors (KIN), for the analysis of a high dimensional metabolomics dataset to classify patients into mild or advanced progression of CKD. Methods: Urine samples were collected from CKD patients (n=73). The patients were classified based on metabolite biomarkers into the two groups: mild CKD (glomerular filtration rate (GFR)> 60 mL/min per 1·73 m2) and advanced CKD (GFR2). Accordingly, 48 and 25 patients were in mild (class 1) and advanced (class 2) groups respectively. Recently, KIN was proposed as a novel approach to high dimensional binary classification settings. Through employing a hybrid dissimilarity measure in KIN, it is possible to incorporate information of variables and distances simultaneously. Results: The proposed KIN not only selected a few number of biomarkers, it also reached a higher accuracy compared to traditional k-nearest neighbors (61.2% versus 60.4%) and random forest (61.2% versus 58.5%) which are currently known as the best classifieres. Conclusion: Real metabolomics dataset demonstrate the superiority of proposed KIN versus KNN in terms of both classification accuracy and variable importance.

The main purpose of discriminant analysis is to enable classification of new observations into one of g classes or populations. Discriminant methods suffer when applied to high dimensional data because the sample covariance matrix is singular. In this study, we propose two new discriminant methods for high dimensional data under the multivariate normal population with a block diagonal covariance matrix structure. As the first method, we approximate the sample covariance matrix as a singular matrix based on the idea of reducing the dimensionality of the observations to get a well-conditioned covariance matrix. As the second method, we use a block diagonal sample covariance matrix instead. The performances of these two methods are compared with some of the existing methods in a simulation study. The results show that both proposed methods outperform other comparative methods in various situations. In addition, the two new proposed methods for discriminant analysis are applied to a real dataset.

Datasets produced in modern research, such as biomedical science, pose a number of challenges for machine learning techniques used in binary classification due to high dimensionality. Feature selection is one of the most important statistical techniques used for dimensionality reduction of the datasets. Therefore, techniques are needed to find an optimal number of features to obtain more desirable learning performance. In the machine learning context, gene selection is treated as a feature selection problem, the objective of which is to find a small subset of the most discriminative features for the target class. In this paper, a gene selection method is proposed that identifies the most discriminative genes in two stages. Genes that unambiguously assign the maximum number of samples to their respective classes using a greedy approach are selected in the first stage. The remaining genes are divided into a certain number of clusters. From each cluster, the most informative genes are selected via the lasso method and combined with genes selected in the first stage. The performance of the proposed method is assessed through comparison with other state-of-the-art feature selection methods using gene expression datasets. This is done by applying two classifiers i.e., random forest and support vector machine, on datasets with selected genes and training samples and calculating their classification accuracy, sensitivity, and Brier score on samples in the testing part. Boxplots based on the results and correlation matrices of the selected genes are thenceforth constructed. The results show that the proposed method outperforms the other methods.

Background. Model building is a crucial part of omics based biomedical research to transfer classifications and obtain insights into underlying mechanisms. Feature selection is often based on minimizing error between model predictions and given classification (maximizing accuracy). Human ratings/classifications, however, might be error prone, with discordance rates between experts of 5-15%. We therefore evaluate if a feature pre-filtering step might improve identification of features associated with true underlying groups. Methods. Data was simulated for up to 100 samples and up to 10,000 features, 10% of which were associated with the ground truth comprising 2-10 normally distributed populations. Binary and semi-quantitative ratings with varying error probabilities were used as classification. For feature preselection standard cross-validation (V2) was compared to a novel heuristic (V1) applying univariate testing, multiplicity adjustment and cross-validation on switched dependent (classification) and independent (features) variables. Preselected features were used to train logistic regression/linear models (backward selection, AIC). Predictions were compared against the ground truth (ROC, multiclass-ROC). As use case, multiple feature selection/classification methods were benchmarked against the novel heuristic to identify prognostically different G-CIMP negative glioblastoma tumors from the TCGA-GBM 450 k methylation array data cohort, starting from a fuzzy umap based rough and erroneous separation. Results. V1 yielded higher median AUC ranks for two true groups (ground truth), with smaller differences for true graduated differences (3-10 groups). Lower fractions of models were successfully fit with V1. Median AUCs for binary classification and two true groups were 0.91 (range: 0.54-1.00) for V1 (Benjamini-Hochberg) and 0.70 (0.28-1.00) for V2, 13% (nD616) of V2 models showed AUCs

We propose a flexible ensemble classication framework, Random Subspace Ensemble (RaSE), for sparse classication. In the RaSE algorithm, we aggregate many weak learners, where each weak learner is a base classifier trained in a subspace optimally selected from a collection of random subspaces. To conduct subspace selection, we propose a new criterion, ratio information criterion (RIC), based on weighted Kullback-Leibler divergence. The theoretical analysis includes the risk and Monte-Carlo variance of the RaSE classifier, establishing the screening consistency and weak consistency of RIC, and providing an upper bound for the misclassification rate of the RaSE classifier. In addition, we show that in a high-dimensional framework, the number of random subspaces needs to be very large to guarantee that a subspace covering signals is selected. Therefore, we propose an iterative version of the RaSE algorithm and prove that under some specifi conditions, a smaller number of generated random subspaces are needed to find a desirable subspace through iteration. An array of simulations under various models and real-data applications demonstrate the eectiveness and robustness of the RaSE classifier and its iterative version in terms of low misclassification rate and accurate feature ranking. The RaSE algorithm is implemented in the R package RaSEn on CRAN. [ABSTRACT FROM AUTHOR]

Random Forest is a supervised classification method based on bagging (Bootstrap aggregating) Breiman and random selection of features. The choice of features randomly assigned to the Random Forest makes it possible that the selected feature is not necessarily informative. So it is necessary to select features in the Random Forest. The purpose of choosing this feature is to select an optimal subset of features that contain valuable information in the hope of accelerating the performance of the Random Forest method. Mainly for the execution of high-dimensional datasets such as the Parkinson, CNAE-9, and Urban Land Cover dataset. The feature selection is done using the Correlation-Based Feature Selection method, using the BestFirst method. Tests were carried out 30 times using the K-Cross Fold Validation value of 10 and dividing the dataset into 70% training and 30% testing. The experiments using the Parkinson dataset obtained a time difference of 0.27 and 0.28 seconds faster than using the Random Forest method without feature selection. Likewise, the trials in the Urban Land Cover dataset had 0.04 and 0.03 seconds, while for the CNAE-9 dataset, the difference time was 2.23 and 2.81 faster than using the Random Forest method without feature selection. These experiments showed that the Random Forest processes are faster when using the first feature selection. Likewise, the accuracy value increased in the two previous experiments, while only the CNAE-9 dataset experiment gets a lower accuracy. This research's benefits is by first performing feature selection steps using the Correlation-Base Feature Selection method can increase the speed of performance and accuracy of the Random Forest method on high-dimensional data. [ABSTRACT FROM AUTHOR]

Feature subset selection for microarray data aims at reducing the number of genes so that useful information can be extracted from the samples. At the same time, selecting the relevant genes (features) from the high dimensional data can improve the classification accuracy of the learning algorithm. This paper proposes a feature selection algorithm, which is fit for high dimensional and small sample size microarray data. Feature selection is performed in two phases. In the first phase, Random Forest is used to identifying the importance of each feature, so that the features with high relevance can be given priority over less relevant ones. In the second phase, feature clustering is performed around the relevant features to yield the reduced feature set. A statistical method is used to create the clusters that aid in giving the genes specifically representing the disease. The effectiveness of the proposed algorithm has been compared with three state-of-the-art feature selection algorithms viz. FastCorrelation Based Filter (FCBF), a Fast Clustering-Based Feature Selection Algorithm (FAST) and Random Forest (RF) on nine real-world cancer microarray datasets. Empirically, the algorithms have been evaluated through three well-known classifiers viz. probability based Naïve Bayes, Tree-based C4.5, and the Instance-based IB1. The stated result shows that the proposed algorithm can be helpful in finding the smaller set of features for cancer microarray datasets with better classification accuracy.

Linear discriminant analysis (LDA) is widely used for various binary classification problems. In contrast to the LDA that estimates the precision matrix Ω and the mean difference vector δ in the classification rule separately, the linear programming discriminant (LPD) rule estimates the product Ωδ directly through a constrained ℓ1 minimization. The LPD rule has very good classification performance on many high‐dimensional binary classification problems. However, to estimate β* = Ωδ, the LPD rule uses equal weights for all the elements of β* in the constrained ℓ1 minimization. It may not deliver the optimal estimate of β*, and therefore the estimated discriminant direction can be suboptimal. In order to obtain better estimates of β* and the discriminant direction, we can heavily penalize βj in the constrained ℓ1 minimization if we suspect the jth feature is useless for the classification while moderately penalize βj if we suspect the jth feature is useful. In this paper, based on the LPD rule and some popular feature screening methods, we propose a new weighted linear programming discriminant (WLPD) rule for the high‐dimensional binary classification problem. The screening statistics used in the marginal two‐sample t‐test screening, Kolmogorov–Smirnov filter, and the maximum marginal likelihood screening will be used to construct appropriate weights for different elements of β* flexibly. Besides the linear programming algorithm, we develop a new alternating direction method of multipliers algorithm to solve the high‐dimensional constrained ℓ1 minimization problem efficiently. Our numerical studies show that our proposed WLPD rule can outperform LPD and serve as an effective binary classification tool. [ABSTRACT FROM AUTHOR]

We propose a robust and sparse classification method based on the optimal scoring approach. It is also applicable if the number of variables exceeds the number of observations. The data are first projected into a low dimensional subspace according to an optimal scoring criterion. The projection only includes a subset of the original variables (sparse modeling) and is not distorted by outliers (robust modeling). In this low dimensional subspace classification is performed by minimizing a robust Mahalanobis distance to the group centers. The low dimensional representation of the data is also useful for visualization purposes. We discuss the algorithm for the proposed method in detail. A simulation study illustrates the properties of robust and sparse classification by optimal scoring compared to the non-robust and/or non-sparse alternative methods. Three real data applications are given. [ABSTRACT FROM AUTHOR]

Handwritten character recognition is the most widely used branch of study in image pattern recognition. Tamil, the official language of Tamil Nadu in South India, Sri Lanka, Singapore and Malaysia, has a script which contains many loops and compound characters, with small differences between character classes. Most of the research on offline Tamil handwritten character recognition system was done only on few character classes as it is very difficult to distinguish between minute dissimilarities of large character classes. It is important to design a complete recognition system that can process all character classes of Tamil and distinguish natural variability between inter-class images. Unlike conventional machine learning approaches for pattern recognition problems, we have proposed a nearest interest point classifier, which can choose sufficient and necessary subset of features from a variable length high dimensional feature vector. Since this is a practical problem, in this work, a study on image to image matching is included through feature analysis without using machine learning approaches. The proposed algorithm gave a good recognition accuracy for all the character classes on the standard database available for Tamil, HP Labs offline Tamil handwritten character database. Our proposed classifier produced a recognition accuracy of 90.2% while including the whole dataset. The method has been compared with the standard classifiers and has been proved to be a state-of-the-art performance in recognition of accuracy over the previous results given in the literature. [ABSTRACT FROM AUTHOR]

Background: Chronic kidney disease (CKD), characterized by progressive loss of renal function, is becoming a growing problem in the general population. New analytical technologies such as "omics"-based approaches, including metabolomics, provide a useful platform for biomarker discovery and improvement of CKD management. In metabolomics studies, not only prediction accuracy is attractive, but also variable importance is critical because the identified biomarkers reveal pathogenic metabolic processes underlying the progression of chronic kidney disease. We aimed to use k-important neighbors (KIN), for the analysis of a high dimensional metabolomics dataset to classify patients into mild or advanced progression of CKD. Methods: Urine samples were collected from CKD patients (n=73). The patients were classified based on metabolite biomarkers into the two groups: mild CKD (glomerular filtration rate (GFR)> 60 mL/min per 1·73 m2) and advanced CKD (GFR<60 mL/min per 1·73 m2). Accordingly, 48 and 25 patients were in mild (class 1) and advanced (class 2) groups respectively. Recently, KIN was proposed as a novel approach to high dimensional binary classification settings. Through employing a hybrid dissimilarity measure in KIN, it is possible to incorporate information of variables and distances simultaneously. Results: The proposed KIN not only selected a few number of biomarkers, it also reached a higher accuracy compared to traditional k-nearest neighbors (61.2% versus 60.4%) and random forest (61.2% versus 58.5%) which are currently known as the best classifieres. Conclusion: Real metabolomics dataset demonstrate the superiority of proposed KIN versus KNN in terms of both classification accuracy and variable importance. [ABSTRACT FROM AUTHOR]

High dimensional data classification with very limited labeled training data is a challenging task in the area of data mining. In order to tackle this task, we first propose a feature selection-based semi-supervised classifier ensemble framework (FSCE) to perform high dimensional data classification. Then, we design an adaptive semi-supervised classifier ensemble framework (ASCE) to improve the performance of FSCE. When compared with FSCE, ASCE is characterized by an adaptive feature selection process, an adaptive weighting process (AWP), and an auxiliary training set generation process (ATSGP). The adaptive feature selection process generates a set of compact subspaces based on the selected attributes obtained by the feature selection algorithms, while the AWP associates each basic semi-supervised classifier in the ensemble with a weight value. The ATSGP enlarges the training set with unlabeled samples. In addition, a set of nonparametric tests are adopted to compare multiple semi-supervised classifier ensemble (SSCE)approaches over different datasets. The experiments on 20 high dimensional real-world datasets show that: 1) the two adaptive processes in ASCE are useful for improving the performance of the SSCE approach and 2) ASCE works well on high dimensional datasets with very limited labeled training data, and outperforms most state-of-the-art SSCE approaches. [ABSTRACT FROM AUTHOR]

With hundreds or thousands of features in high dimensional data, computational workload is challenging. In classification process, features which do not contribute significantly to prediction of classes, add to the computational workload. Therefore the aim of this paper is to use feature selection to decrease the computation load by reducing the size of high dimensional data. Selecting subsets of features which represent all features were used. Hence the process is two-fold; discarding irrelevant data and choosing one feature that representing a number of redundant features. There have been many studies regarding feature selection, for example backward feature selection and forward feature selection. In this study, a k-means clustering based feature selection is proposed. It is assumed that redundant features are located in the same cluster, whereas irrelevant features do not belong to any clusters. In this research, two different high dimensional datasets are used: 1) the Human Activity Recognition Using Smartphones (HAR) Dataset, containing 7352 data points each of 561 features and 2) the National Classification of Economic Activities Dataset, which contains 1080 data points each of 857 features. Both datasets provide class label information of each data point. Our experiment shows that k-means clustering based feature selection can be performed to produce subset of features. The latter returns more than 80% accuracy of classification result.

A high-dimensional feature selection having a very large number of features with an optimal feature subset is an NP-complete problem. Because conventional optimization techniques are unable to tackle large-scale feature selection problems, meta-heuristic algorithms are widely used. In this paper, we propose a particle swarm optimization technique while utilizing regression techniques for feature selection. We then use the selected features to classify the data. Classification accuracy is used as a criterion to evaluate classifier performance, and classification is accomplished through the use of k-nearest neighbour (KNN) and Bayesian techniques. Various high dimensional data sets are used to evaluate the usefulness of the proposed approach. Results show that our approach gives better results when compared with other conventional feature selection algorithms.

Never before in history is the data growing at such a high volume, variety and velocity. It not only provides multi-sources of information but also generates high dimensional data sets. As the dimensionality of the input data space (i.e., the number of predictors) increases, it becomes exponentially more difficult to find global optima for the parameter space, i.e., to fit models. This is one phenomenon of the “curse of dimensionality". Intuitively, data reduction provides a promising way, feature selection is a one method of data reduction. This paper proposes an effective model for feature selection and classification. It takes advantage of the divide-and-conquer idea combined with MapReduce paradigm to handle high dimensional big data sets. Several well-known classification methods have been used and compared in our experiment and the result illustrates that the proposed model outperforms all selected methods. And this model shows great power and flexibility for handling non-big data as well. [ABSTRACT FROM AUTHOR]

Four binary discrimination methods are studied in the context of high-dimension, low sample size data with an asymptotic geometric representation, when the dimension increases while the sample sizes of the classes are fixed. We show that the methods support vector machine, mean difference, distance-weighted discrimination, and maximal data piling have the same asymptotic behavior as the dimension increases. We study the consistent, inconsistent, and strongly inconsistent cases in terms of angles between the normal vectors of the separating hyperplanes of the methods and the optimal direction for classification. A simulation study is done to assess the theoretical results. [ABSTRACT FROM AUTHOR]

We consider a regularized D-classification rule for high dimensional binary classification, which adapts the linear shrinkage estimator of a covariance matrix as an alternative to the sample covariance matrix in the D-classification rule (D-rule in short). We find an asymptotic expression for misclassification rate of the regularized D-rule, when the sample size n and the dimension p both increase and their ratio p ∕ n approaches a positive constant γ . In addition, we compare its misclassification rate to the standard D-rule under various settings via simulation. [ABSTRACT FROM AUTHOR]

With hundreds or thousands of features in high dimensional data, computational workload is challenging. In classification process, features which do not contribute significantly to prediction of classes, add to the computational workload. Therefore the aim of this paper is to use feature selection to decrease the computation load by reducing the size of high dimensional data. Selecting subsets of features which represent all features were used. Hence the process is two-fold; discarding irrelevant data and choosing one feature that representing a number of redundant features. There have been many studies regarding feature selection, for example backward feature selection and forward feature selection. In this study, a k-means clustering based feature selection is proposed. It is assumed that redundant features are located in the same cluster, whereas irrelevant features do not belong to any clusters. In this research, two different high dimensional datasets are used: 1) the Human Activity Recognition Using Smartphones (HAR) Dataset, containing 7352 data points each of 561 features and 2) the National Classification of Economic Activities Dataset, which contains 1080 data points each of 857 features. Both datasets provide class label information of each data point. Our experiment shows that k-means clustering based feature selection can be performed to produce subset of features. The latter returns more than 80% accuracy of classification result. [ABSTRACT FROM AUTHOR]

Predicting customer defection is an important subject for companies producing cloud based software. The studied company sell three products (High, Medium and Low Price), in which the consumer has choice to defect or retain the product after certain period of time. The fact that the company collected very large dataset leads to inapplicability of standard statistical models due to the curse of dimensionality. Parametric statistical models will tend to produce very big standard error which may lead to inaccurate prediction results. This research examines a machine learning approach developed for high dimensional data namely logistic regression ensemble (LORENS). Using computational approaches, LORENS has prediction ability as good as standard logistic regression model i.e. between 66% to 77% prediction accuracy. In this case, LORENS is preferable as it is more reliable and free of assumptions. [ABSTRACT FROM AUTHOR]

Many high dimensional classification techniques have been proposed in the literature based on sparse linear discriminant analysis. To use them efficiently, sparsity of linear classifiers is a prerequisite. However, this might not be readily available in many applications, and rotations of data are required to create the sparsity needed. We propose a family of rotations to create the sparsity required. The basic idea is to use the principal components of the sample covariance matrix of the pooled samples and its variants to rotate the data first and then to apply an existing high dimensional classifier. This rotate-and-solve procedure can be combined with any existing classifiers and is robust against the level of sparsity of the true model. We show that these rotations do create the sparsity that is needed for high dimensional classifications and we provide theoretical understanding why such a rotation works empirically. The effectiveness of the method proposed is demonstrated by several simulated and real data examples, and the improvements of our method over some popular high dimensional classification rules are clearly shown. [ABSTRACT FROM AUTHOR]

Gene expression data typically contain fewer samples (as each experiment is costly) and thousands of expression values (or features) captured by automatic robotic devices. Feature selection is one of the important and challenging tasks for this kind of data where many traditional methods failed and evolutionary based methods were succeeded. In this study, the initial datasets are preprocessed using a quartile based fast heuristic technique to reduce the crude domain features which are less relevant in categorizing the samples of either group. Hamming distance is introduced as a proximity measure to update the velocity of particle(s) in binary PSO framework to select the important feature subsets. The experimental results on three benchmark datasets vis-á-vis colon cancer, defused B-cell lymphoma and leukemia data are evaluated by means of classification accuracies and validity indices as well. Detailed comparative studies are also made to show the superiority and effectiveness of the proposed method. The present study clearly reveals that by choosing proper preprocessing method, fine tuned by HDBPSO with Hamming distance as a proximity measure, it is possible to find important feature subsets in gene expression data with better and competitive performances. [ABSTRACT FROM AUTHOR]

A high-dimensional feature selection having a very large number of features with an optimal feature subset is an NP-complete problem. Because conventional optimization techniques are unable to tackle large-scale feature selection problems, meta-heuristic algorithms are widely used. In this paper, we propose a particle swarm optimization technique while utilizing regression techniques for feature selection. We then use the selected features to classify the data. Classification accuracy is used as a criterion to evaluate classifier performance, and classification is accomplished through the use of k-nearest neighbour (KNN) and Bayesian techniques. Various high dimensional data sets are used to evaluate the usefulness of the proposed approach. Results show that our approach gives better results when compared with other conventional feature selection algorithms. [ABSTRACT FROM AUTHOR]

The concept of any method to resolve feature selection issues is to identify a subset of the original features. However, determining a minimal feature subset is considered an NP-hard problem. Many existing feature selection methods, namely particle swarm optimization (PSO), grey wolf optimization (GWO) etc., are complex and consume more time. This article proposes a new wrapper-based method that employs Binary Golden Eagle Optimizer with Time Varying Flight Length (BGEO-TVFL) to reduce the feature selection issues. The proposed method provides optimal feature selection based on Golden Eagle Optimizer (GEO). GEO is a recently presented swarm-based meta-heuristics algorithm that imitates the smartness of Golden Eagles (GEs). GEO algorithm performs on continuous space; however, the feature selection is in discrete space. Hence, the continuous space has transformed into a discrete space by employing transfer functions. Here, eight different transfer functions have been applied to determine the best transfer function and test the BGEO-TVFL. Moreover, a time-varying flight length (TVFL) has been employed to balance the exploration and exploitation in GEO. Finally, the performance of the proposed BGEO-TVFL has been determined by focusing on various metrics under different UC Irvine (UCI) datasets. The best variant has been selected and analyzed the performance with existing standard filter feature selection approaches and wrapper based feature selection methods. The existing methods include Bat Algorithm (BAT), Ant Colony Optimization (ACO) algorithm, PSO, GWO, Genetic Algorithm (GA), Cuckoo Search (CS), Golden Eagle Optimizer (GEO), ReliefF, Information Gain (IG), Correlation-based feature selection (CFS), and Gain Ratio (GR) respectively. In addition, the performance of BGEO-TVFL is also compared and analyzed with state-of-art methods to determine its superiority. Furthermore, the proposed BGEO-TVFL is analyzed with existing algorithms by employing 30 functions from CEC'2017, CEC'2018 and CEC'2020 benchmarks. As a result, the proposed BGEO-TVFL has achieved a better result than existing methods and is suitable for dealing with dimensionality reduction issues. [ABSTRACT FROM AUTHOR]

The article presents the proposed similarity-dissimilarity plot for the visualization of high dimensional data in biomedical pattern classification. Also cited are the importance of feature extraction in solving pattern classification problems, the proposed set of techniques based on matrices called scatter plot matrices, and the hierarchical clustering algorithms for the visualization of clustering in the feature space.

Abstract: One of the challenges which must be faced in the field of the information processing is the need to cope with huge amounts of data. There exist many different environments in which large quantities of information are produced. For example, in a command-line interface, a computer user types thousands of commands which can hide information about the behavior of her/his. However, processing this kind of streaming data on-line is a hard problem. This paper addresses the problem of the classification of streaming data from a dimensionality reduction perspective. We propose to learn a lower dimensionality input model which best represents the data and improves the prediction performance versus standard techniques. The proposed method uses maximum dependence criteria as distance measurement and finds the transformation which best represents the command-line user. We also make a comparison between the dimensionality reduction approach and using the full dataset. The results obtained give some deeper understanding in advantages and drawbacks of using both perspectives in this user classifying environment. [Copyright &y& Elsevier]

Abstract: This paper proposes, focusing on random forests, the increasingly used statistical method for classification and regression problems introduced by Leo Breiman in 2001, to investigate two classical issues of variable selection. The first one is to find important variables for interpretation and the second one is more restrictive and try to design a good parsimonious prediction model. The main contribution is twofold: to provide some experimental insights about the behavior of the variable importance index based on random forests and to propose a strategy involving a ranking of explanatory variables using the random forests score of importance and a stepwise ascending variable introduction strategy. [Copyright &y& Elsevier]

The lasso and elastic net methods are the popular technique for parameter estimation and variable selection. Moreover, the adaptive lasso and elastic net methods use the adaptive weights on the penalty function based on the lasso and elastic net estimates. The adaptive weight is related to the power order of the estimator. Normally, these methods focus to estimate parameters in terms of linear regression models that are based on the dependent variable and independent variable as a continuous scale. In this paper, we compare the lasso and elastic net methods and the higher-order of the adaptive lasso and adaptive elastic net methods for classification on high dimensional data. The classification is used to classify the categorical data for dependent variable dependent on the independent variables, which is called the logistic regression model. The categorical data are considered a binary variable, and the independent variables are used as the continuous variable. The high dimensional data are represented when the number of independent variables is higher than the sample sizes. For this research, the simulation of the logistic regression is considered as the binary dependent variable and 20, 30, 40, and 50 as the independent variables when the sample sizes are less than the number of the independent variables. The independent variables are generated from normal distribution on several variances, and the dependent variables are obtained from the probability of logit function and transforming it to predict the binary data. For application in real data, we express the classification of the type of leukemia as the dependent variables and the subset of gene expression as the independent variables. The criterion of these methods is to compare by the average percentage of predicted accuracy value. The results are found that the higher-order of adaptive lasso method is satisfied with large dispersion, but the higher-order of adaptive elastic net method outperforms on small dispersion. [ABSTRACT FROM AUTHOR]

• Feature selection approach based on Binary Crow Search Algorithm is proposed. • Time varying flight length is used to enhance balance between exploration & exploitation. • 8 variants of the proposed method based on different transfer functions are tested. • Performance of proposed method is evaluated on 20 benchmark datasets. • Proposed approach outperformed other feature selection approaches. Crow Search Algorithm (CSA) is a simple yet effective meta-heuristic algorithm that has been applied to solve many engineering problems. In CSA, fl parameter controls the search capability of crows and AP parameter balances the trade-off between exploration and exploitation. The parameter fl is initialized to a constant value in CSA. However, CSA faces the problem of being trapped in local minima. This work proposes the solution to this problem by introducing the new concept of time varying flight length in CSA. The value of fl should be large in initial stages of algorithm in order to support random exploration and it should gradually decrease in later iterations to encourage the exploitation of good solutions found so far. The proposed approach, Binary Crow Search Algorithm with Time Varying Flight Length (BCSA-TVFL) is applied to feature selection problems in wrapper mode. Eight variants of BCSA-TVFL based on eight different transfer functions are tested. The best performing variant is then selected and compared with other state-of-the-art wrapper feature selection techniques and standard filter feature selection techniques. Performance of proposed approach is tested on 20 standard UCI datasets. Experimental result comparison shows that the proposed feature selection technique performs better than other competitors. [ABSTRACT FROM AUTHOR]