Search

Support vector machine (SVM) is one of the most studied paradigms in the realm of machine learning for classification and regression problems. It relies on vectorized input data. However, a significant portion of the real-world data exists in matrix format, which is given as input to SVM by reshaping the matrices into vectors. The process of reshaping disrupts the spatial correlations inherent in the matrix data. Also, converting matrices into vectors results in input data with a high dimensionality, which introduces significant computational complexity. To overcome these issues in classifying matrix input data, support matrix machine (SMM) is proposed. It represents one of the emerging methodologies tailored for handling matrix input data. SMM preserves the structural information of the matrix data by using the spectral elastic net property which is a combination of the nuclear norm and Frobenius norm. This article provides the first in-depth analysis of the development of the SMM model, which can be used as a thorough summary by both novices and experts. We discuss numerous SMM variants, such as robust, sparse, class-imbalance, and multi-class classification models. We also analyze the applications of the SMM and conclude the article by outlining potential future research avenues and possibilities that may motivate researchers to advance the SMM algorithm., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Multiview learning (MVL) seeks to leverage the benefits of diverse perspectives to complement each other, effectively extracting and utilizing the latent information within the dataset. Several twin support vector machine-based MVL (MvTSVM) models have been introduced and demonstrated outstanding performance in various learning tasks. However, MvTSVM-based models face significant challenges in the form of computational complexity due to four matrix inversions, the need to reformulate optimization problems in order to employ kernel-generated surfaces for handling non-linear cases, and the constraint of uniform noise assumption in the training data. Particularly in cases where the data possesses a heteroscedastic error structure, these challenges become even more pronounced. In view of the aforementioned challenges, we propose multiview twin parametric margin support vector machine (MvTPMSVM). MvTPMSVM constructs parametric margin hyperplanes corresponding to both classes, aiming to regulate and manage the impact of the heteroscedastic noise structure existing within the data. The proposed MvTPMSVM model avoids the explicit computation of matrix inversions in the dual formulation, leading to enhanced computational efficiency. We perform an extensive assessment of the MvTPMSVM model using benchmark datasets such as UCI, KEEL, synthetic, and Animals with Attributes (AwA). Our experimental results, coupled with rigorous statistical analyses, confirm the superior generalization capabilities of the proposed MvTPMSVM model compared to the baseline models. The source code of the proposed MvTPMSVM model is available at https://github.com/mtanveer1/MvTPMSVM., Competing Interests: Declaration of competing interest The authors of this work have no competing interests., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Fuzzy membership is an effective approach used in twin support vector machines (SVMs) to reduce the effect of noise and outliers in classification problems. Fuzzy twin SVMs (TWSVMs) assign membership weights to reduce the effect of outliers, however, it ignores the positioning of the input data samples and hence fails to distinguish between support vectors and noise. To overcome this issue, intuitionistic fuzzy TWSVM combined the concept of intuitionistic fuzzy number with TWSVMs to reduce the effect of outliers and distinguish support vectors from noise. Despite these benefits, TWSVMs and intuitionistic fuzzy TWSVMs still suffer from some drawbacks as: 1) the local neighborhood information is ignored among the data points and 2) they solve quadratic programming problems (QPPs), which is computationally inefficient. To overcome these issues, we propose a novel intuitionistic fuzzy weighted least squares TWSVMs for classification problems. The proposed approach uses local neighborhood information among the data points and also uses both membership and nonmembership weights to reduce the effect of noise and outliers. The proposed approach solves a system of linear equations instead of solving the QPPs which makes the model more efficient. We evaluated the proposed intuitionistic fuzzy weighted least squares TWSVMs on several benchmark datasets to show the efficiency of the proposed model. Statistical analysis is done to quantify the results statistically. As an application, we used the proposed model for the diagnosis of Schizophrenia disease.

Imbalanced datasets are prominent in real-world problems. In such problems, the data samples in one class are significantly higher than in the other classes, even though the other classes might be more important. The standard classification algorithms may classify all the data into the majority class, and this is a significant drawback of most standard learning algorithms, so imbalanced datasets need to be handled carefully. One of the traditional algorithms, twin support vector machines (TSVM), performed well on balanced data classification but poorly on imbalanced datasets classification. In order to improve the TSVM algorithm's classification ability for imbalanced datasets, recently, driven by the universum twin support vector machine (UTSVM), a reduced universum twin support vector machine for class imbalance learning (RUTSVM) was proposed. The dual problem and finding classifiers involve matrix inverse computation, which is one of RUTSVM's key drawbacks. In this paper, we improve the RUTSVM and propose an improved reduced universum twin support vector machine for class imbalance learning (IRUTSVM). We offer alternative Lagrangian functions to tackle the primal problems of RUTSVM in the suggested IRUTSVM approach by inserting one of the terms in the objective function into the constraints. As a result, we obtain new dual formulation for each optimization problem so that we need not compute inverse matrices neither in the training process nor in finding the classifiers. Moreover, the smaller size of the rectangular kernel matrices is used to reduce the computational time. Extensive testing is carried out on a variety of synthetic and real-world imbalanced datasets, and the findings show that the IRUTSVM algorithm outperforms the TSVM, UTSVM, and RUTSVM algorithms in terms of generalization performance., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Random Forest is an ensemble of decision trees based on the bagging and random subspace concepts. As suggested by Breiman, the strength of unstable learners and the diversity among them are the ensemble models' core strength. In this paper, we propose two approaches known as oblique and rotation double random forests. In the first approach, we propose rotation based double random forest. In rotation based double random forests, transformation or rotation of the feature space is generated at each node. At each node different random feature subspace is chosen for evaluation, hence the transformation at each node is different. Different transformations result in better diversity among the base learners and hence, better generalization performance. With the double random forest as base learner, the data at each node is transformed via two different transformations namely, principal component analysis and linear discriminant analysis. In the second approach, we propose oblique double random forest. Decision trees in random forest and double random forest are univariate, and this results in the generation of axis parallel split which fails to capture the geometric structure of the data. Also, the standard random forest may not grow sufficiently large decision trees resulting in suboptimal performance. To capture the geometric properties and to grow the decision trees of sufficient depth, we propose oblique double random forest. The oblique double random forest models are multivariate decision trees. At each non-leaf node, multisurface proximal support vector machine generates the optimal plane for better generalization performance. Also, different regularization techniques (Tikhonov regularization, axis-parallel split regularization, Null space regularization) are employed for tackling the small sample size problems in the decision trees of oblique double random forest. The proposed ensembles of decision trees produce trees with bigger size compared to the standard ensembles of decision trees as bagging is used at each non-leaf node which results in improved performance. The evaluation of the baseline models and the proposed oblique and rotation double random forest models is performed on benchmark 121 UCI datasets and real-world fisheries datasets. Both statistical analysis and the experimental results demonstrate the efficacy of the proposed oblique and rotation double random forest models compared to the baseline models on the benchmark datasets., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

The timely identification of significant memory concern (SMC) is crucial for proactive cognitive health management, especially in an aging population. Detecting SMC early enables timely intervention and personalized care, potentially slowing cognitive disorder progression. This study presents a state‐of‐the‐art review followed by a comprehensive evaluation of machine learning models within the randomized neural networks (RNNs) and hyperplane‐based classifiers (HbCs) family to investigate SMC diagnosis thoroughly. Utilizing the Alzheimer's Disease Neuroimaging Initiative 2 (ADNI2) dataset, 111 individuals with SMC and 111 healthy older adults are analyzed based on T1W magnetic resonance imaging (MRI) scans, extracting rich features. This analysis is based on baseline structural MRI (sMRI) scans, extracting rich features from gray matter (GM), white matter (WM), Jacobian determinant (JD), and cortical thickness (CT) measurements. In RNNs, deep random vector functional link (dRVFL) and ensemble dRVFL (edRVFL) emerge as the best classifiers in terms of performance metrics in the identification of SMC. In HbCs, Kernelized pinball general twin support vector machine (Pin‐GTSVM‐K) excels in CT and WM features, whereas Linear Pin‐GTSVM (Pin‐GTSVM‐L) and Linear intuitionistic fuzzy TSVM (IFTSVM‐L) performs well in the JD and GM features sets, respectively. This comprehensive evaluation emphasizes the critical role of feature selection, feature based‐interpretability and model choice in attaining an effective classifier for SMC diagnosis. The inclusion of statistical analyses further reinforces the credibility of the results, affirming the rigor of this analysis. The performance measures exhibit the suitability of this framework in aiding researchers with the automated and accurate assessment of SMC. The source codes of the algorithms and datasets used in this study are available at https://github.com/mtanveer1/SMC. This article is categorized under:Technologies > ClassificationTechnologies > Machine LearningApplication Areas > Health Care [ABSTRACT FROM AUTHOR]

Loss function plays a vital role in supervised learning frameworks. The selection of the appropriate loss function holds the potential to have a substantial impact on the proficiency attained by the acquired model. The training of supervised learning algorithms inherently adheres to predetermined loss functions during the optimization process. In this paper, we present a novel contribution to the realm of supervised machine learning: an asymmetric loss function named wave loss. It exhibits robustness against outliers, insensitivity to noise, boundedness, and a crucial smoothness property. Theoretically, we establish that the proposed wave loss function manifests the essential characteristic of being classification-calibrated. Leveraging this breakthrough, we incorporate the proposed wave loss function into the least squares setting of support vector machines (SVM) and twin support vector machines (TSVM), resulting in two robust and smooth models termed as Wave-SVM and Wave-TSVM, respectively. To address the optimization problem inherent in Wave-SVM, we utilize the adaptive moment estimation (Adam) algorithm, which confers multiple benefits, including the incorporation of adaptive learning rates, efficient memory utilization, and faster convergence during training. It is noteworthy that this paper marks the first instance of Adam's application to solve an SVM model. Further, we devise an iterative algorithm to solve the optimization problems of Wave-TSVM. To empirically showcase the effectiveness of the proposed Wave-SVM and Wave-TSVM, we evaluate them on benchmark UCI and KEEL datasets (with and without feature noise) from diverse domains. Moreover, to exemplify the applicability of Wave-SVM in the biomedical domain, we evaluate it on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The experimental outcomes unequivocally reveal the prowess of Wave-SVM and Wave-TSVM in achieving superior prediction accuracy against the baseline models. The source codes of the proposed models are publicly available at https://github.com/mtanveer1/Wave-SVM. • A new asymmetric, bounded, and smooth loss function termed wave loss is proposed. • Theoretically, we analyzed the classification-calibrated characteristic of the wave loss function. • Two new robust and smooth models, termed Wave-SVM and Wave-TSVM, are proposed. • The Adam algorithm is used to solve the optimization problem of the Wave-SVM. • The optimization problem of the Wave-TSVM is solved by an efficient iterative method. [ABSTRACT FROM AUTHOR]

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and arti- ficial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noisecorrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model. [ABSTRACT FROM AUTHOR]

Breast cancer is a common disease that affects feminine health, making it an active area of research. Also, support vector machine with pinball loss (pin-SVM) is an efficient classification algorithm to address noise sensitivity and re-sampling instability. The pinball loss function uses a loss parameter τ ∈ [ 0 , 1 ] which corresponds to the quantile level. However, the non-negativity condition on τ is not necessary, and it can be extended to the negative values for an improvement in classification accuracy. Also, instead of a positive loss parameter τ , two positive parameters, τ 1 and τ 2 are used in literature, which improve the generalization performance of the pin-SVM. Taking motivation from the aforementioned observations, in this paper, we propose an innovative loss function, termed the flexible pinball loss, which extends the parameters τ 1 and τ 2 to encompass negative values. This extension enables the function to take τ 1 and τ 2 values from − 1 to 1 while preserving convexity. Subsequently, we integrate the proposed flexible pinball loss function into the support vector machine framework and propose a novel model named flexible pinball loss support vector machine (FP-SVM) for the prediction of breast cancer. FP-SVM provides loss to both incorrectly and correctly classified samples, leveraging the parameters τ 1 and τ 2 , respectively. Importantly, FP-SVM strategically traverses the maximum solution path, ensuring the preservation of convexity within the optimization problem. The proposed FP-SVM outperforms the baseline models in terms of accuracy, which is empirically supported by numerical experiments on 30 UCI and KEEL benchmark datasets. Furthermore, to show the efficacy of the proposed FP-SVM in real-world application, we performed experiments on publicly available breast cancer dataset (BreakHis), and the results demonstrate that the proposed FP-SVM outperforms the baseline models. • A novel convex flexible pinball loss SVM (FP-SVM) is proposed for classification. • The proposed FP-SVM employs two parameters to control the loss on both incorrectly and correctly classified samples. • Numerical experiments on 30 benchmark datasets demonstrate that the proposed FP-SVM outperforms the baseline models. • The proposed model has an application in classifying breast cancer dataset (BreakHis). [ABSTRACT FROM AUTHOR]

Abstract The original twin support vector machine (TWSVM) formulation works by solving two smaller quadratic programming problems (QPPs) as compared to the traditional hinge-loss SVM (C-SVM) which solves a single large QPP — this makes the TWSVM training and testing process faster than the C-SVM. However, these TWSVM problems are based on the hinge-loss function and, hence, are sensitive to feature noise and unstable for re-sampling. The pinball-loss function, on the other hand, maximizes quantile distances which grants noise insensitivity but this comes at the cost of losing sparsity by penalizing correctly classified samples as well. To overcome the limitations of TWSVM, we propose a novel sparse pinball twin support vector machines (SPTWSVM) based on the ϵ -insensitive zone pinball loss function to rid the original TWSVM of its noise insensitivity and ensure that the resulting TWSVM problems retain sparsity which makes computations relating to predictions just as fast as the original TWSVM. We further investigate the properties of our SPTWSVM including sparsity, noise insensitivity, and time complexity. Exhaustive testing on several benchmark datasets demonstrates that our SPTWSVM is noise insensitive, retains sparsity and, in most cases, outperforms the results obtained by the original TWSVM. Highlights • A novel twin support vector machine with sparse pinball loss (SPTWSVM) is proposed. • Our proposed SPTWSVM is noise insensitive, retain sparsity and more stable for re-sampling. • The proposed SPTWSVM's time complexity is approximately four times faster than the Pin-SVM. • Numerical results have shown better generalization performance for noise corrupted datasets. • The proposed SPTWSVM can be easily extended to other variants of TWSVM. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) has been used widely for classification of electroencephalogram (EEG) signals for the diagnosis of neurological disorders such as epilepsy and sleep disorders. SVM shows good generalization performance for high dimensional data due to its convex optimization problem. The incorporation of prior knowledge about the data leads to a better optimized classifier. Different types of EEG signals provide information about the distribution of EEG data. To include prior information in the classification of EEG signals, we propose a novel machine learning approach based on universum support vector machine (USVM) for classification. In our approach, the universum data points are generated by selecting universum from the EEG dataset itself which are the interictal EEG signals. This removes the effect of outliers on the generation of universum data. Further, to reduce the computation time, we use our approach of universum selection with universum twin support vector machine (UTSVM) which has less computational cost in comparison to traditional SVM. For checking the validity of our proposed methods, we use various feature extraction techniques for different datasets consisting of healthy and seizure signals. Several numerical experiments are performed on the generated datasets and the results of our proposed approach are compared with other baseline methods. Our proposed USVM and proposed UTSVM show better generalization performance compared to SVM, USVM, Twin SVM (TWSVM) and UTSVM. The proposed UTSVM has achieved highest classification accuracy of 99% for the healthy and seizure EEG signals. [ABSTRACT FROM AUTHOR]

Generalized eigenvalue proximal support vector machine (GEPSVM) has attracted widespread attention due to its simple architecture, rapid execution, and commendable performance. GEPSVM gives equal significance to all samples, thereby diminishing its robustness and efficacy when confronted with real-world datasets containing noise and outliers. In order to reduce the impact of noises and outliers, we propose a novel intuitionistic fuzzy generalized eigenvalue proximal support vector machine (IF-GEPSVM). The proposed IF-GEPSVM assigns the intuitionistic fuzzy score to each training sample based on its location and surroundings in the high-dimensional feature space by using a kernel function. The solution of the IF-GEPSVM optimization problem is obtained by solving a generalized eigenvalue problem. Further, we propose an intuitionistic fuzzy improved generalized eigenvalue proximal support vector machine (IF-IGEPSVM) by solving standard eigenvalue decomposition resulting in simpler optimization problems with less computation cost which leads to an efficient intuitionistic fuzzy-based model. We conduct a comprehensive evaluation of the proposed IF-GEPSVM and IF-IGEPSVM models on UCI and KEEL benchmark datasets. Moreover, to evaluate the robustness of the proposed IF-GEPSVM and IF-IGEPSVM models, label noise is introduced into some UCI and KEEL datasets. The experimental findings showcase the superior generalization performance of the proposed IF-GEPSVM and IF-IGEPSVM models when compared to the existing baseline models, both with and without label noise. Our experimental results, supported by rigorous statistical analyses, confirm the superior generalization abilities of the proposed IF-GEPSVM and IF-IGEPSVM models over the baseline models. Furthermore, we implement the proposed IF-GEPSVM and IF-IGEPSVM models on the USPS recognition dataset, yielding promising results that underscore the models' effectiveness in practical and real-world applications. The source code of the proposed IF-GEPSVM and IF-IGEPSVM models are available at https://github.com/mtanveer1/IF-GEPSVM. [ABSTRACT FROM AUTHOR]

• Proposed a universum based recursive feature selection technique. • Feature selection is performed using knowledge about data distribution. • Alzheimer's data is classified using various feature extraction methods. • Voxel based and volume based morphometry is utilized for analysis. • Better localization of brain regions is obtained using proposed method. Alzheimer's disease is one of the most common causes of death in today's world. Magnetic resonance imaging (MRI) provides an efficient and non-invasive approach for diagnosis of Alzheimer's disease. Efficient feature extraction techniques are needed for accurate classification of MRI images. Motivated by the work on support vector machine based recursive feature elimination (SVM-RFE) [16] , we propose a novel feature selection technique to incorporate prior information about data distribution in the recursive feature elimination process. Our method is termed as universum support vector machine based recursive feature elimination (USVM-RFE). The proposed method provides global information about data in the RFE process as compared to the local approach of feature selection in SVM-RFE. We also present the application of feature selection and classification algorithms on both voxel based as well as volume based morphometry analysis of structural MRI images (ADNI database). Feature selection is performed using MRI data of brain tissues such as gray matter, white matter, and cerebrospinal fluid. USVM-RFE provides improvement over SVM-RFE in classification of control normal (CN), mild cognitive impairment (MCI), and Alzheimer's disease (AD) subjects. Moreover, better accuracy is obtained by USVM-RFE with lesser number of features in comparison to SVM-RFE. This leads to identification of prominent brain regions for feature selection and classification of MRI images. The highest accuracies obtained by our method for classification of CN vs AD, CN vs MCI, and MCI vs AD are 100%, 90%, and 73.68%, respectively. [ABSTRACT FROM AUTHOR]