Search

Twin support vector regression (TSVR) has been widely applied in regression problems. TSVR seeks a pair of ε -insensitive proximal planes by solving two support vector machine type problems. TSVR assumes that the matrices appearing in the dual formulation are positive definite. However, in real-world scenarios, such an assumption may not be fulfilled and, hence, leads to suboptimal performance. ε -Twin support vector regression (ε -TSVR) improved the TSVR by introducing the regularisation term to avoid the singularity issues. Most of the twin support vector machine models involve the computation of matrix inverses. Also, TSVR implements the empirical risk minimization principle. In this paper, we propose an improved twin support vector regression (ITSVR) for brain age estimation by introducing different Lagrangian functions for the primal problems of the TSVR. The proposed ITSVR implements the structural risk minimization principle and avoids the computation of the matrix inverses. To solve the optimization problem more efficiently, we used successive overrelaxation (SOR) technique. We evaluated the proposed ITSVR on cognitively healthy subjects, mild cognitive impairment subjects and Alzheimer's disease subjects. The experimental results demonstrate that the proposed ITSVR has superior performance compared to the baseline models for brain age estimation. [ABSTRACT FROM AUTHOR]

Discovering a hearing disorder at an earlier intervention is critical for reducing the effects of hearing loss and the approaches to increase the remaining hearing ability can be implemented to achieve the successful development of human communication. Recently, the explosive dataset features have increased the complexity for audiologists to decide the proper treatment for the patient. In most cases, data with irrelevant features and improper classifier parameters causes a crucial influence on the audiometry system in terms of accuracy. This is due to the dependent processes of these two, where the classification accuracy performance could be worsened if both processes are conducted independently. Although the filter algorithm is capable of eliminating irrelevant features, it still lacks the ability to consider feature reliance and results in a poor selection of significant features. Improper kernel parameter settings may also contribute to poor accuracy performance. In this paper, an ensemble filters feature selection based on Information Gain (IG), Gain Ratio (GR), Chi-squared (CS), and Relief-F (RF) with harmonize optimization of Particle Swarm Optimization (PSO) and Support Vector Machine (SVM) is presented to mitigate these problems. Ensemble filters are utilized so that the initial top dominant features relevant for classification can be considered. Then, PSO and SVM are optimized simultaneously to achieve the optimal solution. The results on a standard Audiology dataset show that the proposed method produces 96.50% accuracy with optimal solution compared to classical SVM, which signifies the proposed method is effective in handling high dimensional data for hearing disorder prediction. [ABSTRACT FROM AUTHOR]

The experts' decisions and evaluating the patients' data are the most significant parts affecting the breast cancer analysis. For early breast cancer detection, numerous techniques of machine learning not only can assist in examining and diagnosis the medical data quickly but also decrease the potential errors that could be occurred due to inexpert or unskilled decision-makers. Support vector machine is one of the famous classifiers that has already made an important contribution to the field of cancer classification. However, configurations of different kernel function and their parameters can significantly affect the performance of the SVM classifier. To further improve the classification accuracy of the SVM classifier for breast cancer diagnosis, an intelligent cancer classification method is proposed based on selecting a feature subset and optimizing the relevant parameters (i.e., penalty factor parameter (c ) and kernel parameter γ ) of the SVM classifier concurrently through an intelligent algorithm using the Jaya algorithm. Then, this method (Jaya-SVM) was applied to precisely characterize the breast cancer dataset, including 699 samples, which are 458 and 241 for benign and malignant, respectively. Furthermore, to evaluate the effectiveness of the proposed Jaya-SVM classifier, it is compared in terms of the computational complexity and the classification accuracy with several combinatorial metaheuristic classifiers, namely the genetic algorithm (GA), differential evolution (DE), particle swarm optimization (PSO), and cuckoo search (CS) based-SVM. Apart from this, a Breast Cancer Coimbra Dataset taken from the UCI library is used to validate the effectiveness of the proposed method. The results are presented, explained, and conclusions are drawn. [ABSTRACT FROM AUTHOR]

A Gaussian based Particle Swarm Optimization Gravitational Search Algorithm (GPSOGSA) is being proposed for extensive feature selection that serves highly in making effective predictions. GPSOGSA helps to overcome the problem of being stuck into the local optima and influences the local searching ability, thus it aims to bridge the gap of exploration and exploitation. The algorithm also limits the usage of too many parameters like acceleration factors, maximum velocity, inertia weight that plays a vital role in PSO, GSA and PSOGSA. The efficacy of the algorithm has been tested upon unimodal and multimodal benchmark functions. We have also evaluated the performance of the algorithm by applying it on various benchmark datasets. The algorithm uses a wrapper-based approach that includes Support Vector Machine as a learner algorithm, and improves both the execution time and the performance accuracy. The findings show that the proposed algorithm could escape from local optimum and converges faster than the PSO, GSA and PSOGSA algorithms. [ABSTRACT FROM AUTHOR]

The quest to improve performance accuracy and prediction speed in machine learning algorithms cannot be overemphasized, as the need for machines to outperform humans continue to grow. Accordingly, several studies have proposed methods to improve prediction performance and speed particularly for spatio-temporal analysis. This study proposes a novel classifier that leverages ensemble techniques to improve prediction performance and speed. The proposed classifier, Ada-AdaSVM uses an AdaBoost feature selection algorithm to select small features of input datasets for a joint support vector machine (SVM)–AdaBoost classifier. The proposition is evaluated against a selection of existing classifiers (SVM, AdaSVM and AdaBoost) using the Jaffe, Yale, Taiwanese facial expression database (TFEID) and CK + 48 datasets with Haar features as the preferred method for feature extraction. The findings indicated that Ada-AdaSVM outperforms SVM, AdaSVM and AdaBoost classifiers in terms of speed and accuracy. [ABSTRACT FROM AUTHOR]

With the exponential growth of the amount of data being generated, stored and processed on a daily basis in the machine learning, data analytics and decision-making systems, the data preprocessing established itself as the key factor for building reliable high-performance machine learning models. One of the roles in preprocessing is variable reduction using feature selection methods; however, the processing time needed for these methods is a major drawback. This study aims at mitigating this problem by migrating the algorithm to a MapReduce implementation suitable for parallelization on a high number of commodity hardware units. The genetic algorithm-based methods were put in the focus of this study. Hadoop, an open-source MapReduce library, was used as a framework for implementing parallel genetic algorithms within our research. The representative machine learning methods, SVM (support vector machine), ANN (artificial neural network), RT (random tree), logistic regression and Naive Bayes, were embedded into implementation for feature selection. The feature selection methods were applied to four NSL-KDD data sets, and the number of features is reduced from cca 40 to cca 10 data sets with the accuracy of 90.45%. These results have both significant practical and theoretical impact. On the one hand, the genetic algorithm has been parallelized in the MapReduce manner, which has been considered unachievable in a strict sense. Furthermore, the genetic algorithm allows randomness-enhanced feature selection and its parallelization reduces overall data preprocessing and allows larger population count which in turn leads to better feature selection. On the practical side, it has been shown that this implementation outperforms the existing feature selection methods. [ABSTRACT FROM AUTHOR]

The semi-supervised support vector machine (S3VM) for classification is introduced for dealing with quantities of unlabeled data in the real world. Labeled data are utilized to train the algorithm and then were adapted to classify the unlabeled data. However, this algorithm has several drawbacks, such as the non-smooth term of semi-supervised objective function negatively affects the classification precision. Moreover, it is required to endure heavy burden in solving two quadratic programming problems with inversion matrix operation. To cope with this problem, this article puts forward a novel class of Bézier smooth semi-supervised support vector machines (BS4VMs), based on the approximation property of Bézier function to the non-smooth term. Because of this approximation, a fast quasi-Newton method for solving BS4VMs can be used to decrease the calculating time scale. This new kind of algorithm enhances the generalization and robustness of S3VM for nonlinear case as well. Further, to show how the BS4VMs can be practically implemented, experiments on synthetic, UCI dataset, USPS dataset, and large-scale NDC database are offered. The theoretical analysis and experiments comparisons clearly confirm the superiority of BS4VMs in both classification accuracy and calculating time. [ABSTRACT FROM AUTHOR]

Features subset selection was commonly used in data mining and artificial intelligence techniques to produce a model with a minimal set of features that enhances the performance of the classifier. The essential motive for selecting features is to avoid the problem of a number of dimensions trap. This paper introduces a new technique of selection of features dependent on the modified of binary teaching–learning-based optimization and the suggested method called MBTLBO. This algorithm (teaching learning-based optimization TLBO) is one of the present metaheuristic that is been widely utilized to a several of intractable optimization issues in recent times. Such algorithm has been combined with supervised data mining technique (support vector machine) for the implementation of feature subset selection problem in binary identification. The collection of specific risk features with the rheumatic disease was implemented. The findings revealed that the new approach (MBTLBO) increases the accuracy of classification. [ABSTRACT FROM AUTHOR]

A software defect report is a bug in the software system that developers and users submit to the software defect library during software development and maintenance. Managing a software defect report that is overwhelming is a challenging task. The traditional method is manual identification, which is time-consuming and laborious and delays the repair of important software defects. Based on the above background, the purpose of this paper is to study the software defect prediction (SDP) model based on LASSO–SVM. In this paper, the problem of poor prediction accuracy of most SDP models is proposed. A SDP model combining minimum absolute value compression and selection method and support vector machine algorithm is proposed. Firstly, the feature selection ability of the minimum absolute value compression and selection method is used to reduce the dimension of the original data set, and the data set not related to SDP is removed. Then, the optimal value of SVM is obtained by using the parameter optimization ability of cross-validation algorithm. Finally, the SDP is completed by the nonlinear computing ability of SVM. The accuracy of simulation results is 93.25% and 66.67%, recall rate is 78.04%, and f-metric is 72.72%. The results show that the proposed defect prediction model has higher prediction accuracy than the traditional defect prediction model, and the prediction speed is faster. [ABSTRACT FROM AUTHOR]

Among the extensions of twin support vector machine (TSVM), some scholars have utilized K-nearest neighbor (KNN) graph to enhance TSVM's classification accuracy. However, these KNN-based TSVM classifiers have two major issues such as high computational cost and overfitting. In order to address these issues, this paper presents an enhanced regularized K-nearest neighbor-based twin support vector machine (RKNN-TSVM). It has three additional advantages: (1) Weight is given to each sample by considering the distance from its nearest neighbors. This further reduces the effect of noise and outliers on the output model. (2) An extra stabilizer term was added to each objective function. As a result, the learning rules of the proposed method are stable. (3) To reduce the computational cost of finding KNNs for all the samples, location difference of multiple distances-based K-nearest neighbors algorithm (LDMDBA) was embedded into the learning process of the proposed method. The extensive experimental results on several synthetic and benchmark datasets show the effectiveness of our proposed RKNN-TSVM in both classification accuracy and computational time. Moreover, the largest speedup in the proposed method reaches to 14 times. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) is a widely used and reliable machine learning algorithm. It has been successfully applied to many real-world problems, with remarkable results. However, it has also been observed that SVM computational complexity increases with the increase in data size. Although many SVM speed optimization techniques have been proposed in the literature, there is still need for further improvement on the performance speed and accuracy of this algorithm. In this paper, a boundary detection algorithm for SVM speed optimization called ant colony optimization instance selection algorithm (ACOISA) is proposed. ACOISA is inspired by edge selection in ant colony optimization (ACO) algorithm, and it performs two primary functions: boundary detection and boundary instance selection. In the algorithm, ACO is used for boundary detection and k-nearest neighbor algorithm is used for boundary instance selection. Different sets of experiments are carried out to validate the efficiency of the proposed technique. All the experiments were performed on 35 datasets containing three well-known e-fraud types (credit card fraud, email spam and phishing email) and 31 other datasets available at UCI data repository. The experimental results showed that the proposed technique efficiently improved SVM training speed in 100% of the datasets used for evaluation, without significantly affecting SVM classification quality. Furthermore, the Freidman's and Holm's post hoc tests were conducted to statistically validate the credibility of the results. The statistical test results revealed that the proposed technique is significantly faster, compared to the standard SVM and some existing instance selection techniques, in all cases. [ABSTRACT FROM AUTHOR]

Construction of robust regression learning models to fit data with noise is an important and challenging problem of data regression. One of the ways to tackle this problem is the selection of a proper loss function showing insensitivity to noise present in the data. Since Huber function has the property that inputs with large deviations of misfit are penalized linearly and small errors are squared, we present novel robust regularized twin support vector machines for data regression based on Huber and ε-insensitive Huber loss functions in this study. The proposed regression models result in solving a pair of strongly convex minimization problems in simple form in primal whose solutions are obtained by functional and Newton–Armijo iterative algorithms. The finite convergence of Newton–Armijo algorithm is proved. Numerical tests are performed on noisy synthetic and benchmark datasets, and their results are compared with few popular regression learning algorithms. The comparative study clearly shows the robustness of the proposed regression methods and further demonstrates their effectiveness and suitability. [ABSTRACT FROM AUTHOR]

C-Mantec neural network constructive algorithm Ortega (C-Mantec neural network algorithm implementation on MATLAB. https://github.com/IvanGGomez/CmantecPaco, 2015) creates very compact architectures with generalization capabilities similar to feed-forward networks trained by the well-known back-propagation algorithm. Nevertheless, constructive algorithms suffer much from the problem of overfitting, and thus, in this work the learning procedure is first analyzed for networks created by this algorithm with the aim of trying to understand the training dynamics that will permit optimization possibilities. Secondly, several optimization strategies are analyzed for the position of class separating hyperplanes, and the results analyzed on a set of public domain benchmark data sets. The results indicate that with these modifications a small increase in prediction accuracy of C-Mantec can be obtained but in general this was not better when compared to a standard support vector machine, except in some cases when a mixed strategy is used. [ABSTRACT FROM AUTHOR]

Stress is an issue that everyone experiences in today's modern life. Prolonged exposure to stress can cause many mental and physical diseases. Accordingly, the stress management issue has become popular, and the need for personal healthcare devices has increased in recent years. Therefore, the aim of this research is to design and manufacture a portable stress monitoring system, based on photoplethysmography (PPG) and galvanic skin response (GSR) physiological signals, acquired by wearable sensors. To do so, we proposed a novel algorithm for continuous measurement of the stress index (SI) as well as the classification of stress levels. In order to estimate an accurate value for SI, various soft computing algorithms such as support vector regression, artificial neural networks (ANN), and adaptive neuro-fuzzy inference system (ANFIS) were adopted for modeling the stress based on the features extracted from normalized and non-normalized types of PPG and GSR signals and their combinations. Furthermore, K-nearest neighbor (KNN), ANNs, Naive Bayes, and support vector machine (SVM) were utilized to discriminate different levels of stress in subjects. The obtained results indicate that the ANFIS algorithm can estimate the SI training output with the correlation coefficient (CC) of 0.9281 and the average relative error of 0.23 on a subset of the combined features of PPG and GSR signals. Also, the best classification performance was for KNN (K = 3) algorithm, with 85.3% accuracy. To evaluate the developed system, data of 16 subjects, out of the training dataset, participated in the experiment in the presence of the experts and psychologists, were used. The average CC of 0.81 and classification accuracy of 75% were obtained, using the implemented ANFIS model and KNN classifier. [ABSTRACT FROM AUTHOR]

The parameters of support vector machines (SVMs) such as kernel parameters and the penalty parameter have a great influence on the accuracy and complexity of the classification models. In the past, different evolutionary optimization algorithms were employed for optimizing SVMs; in this paper, we propose a social ski-driver (SSD) optimization algorithm which is inspired from different evolutionary optimization algorithms for optimizing the parameters of SVMs, with the aim of improving the classification performance. To cope with the problem of imbalanced data which is one of the challenging problems for building robust classification models, the proposed algorithm (SSD-SVM) was enhanced to deal with imbalanced data. In this study, eight standard imbalanced datasets were used for testing our proposed algorithm. For verification, the results of the SSD-SVM algorithm are compared with grid search, which is a conventional method of searching parameter values, and particle swarm optimization (PSO). The experimental results show that the SSD-SVM algorithm is capable of finding near-optimal values of SVMs parameters. The results also demonstrated high classification performance compared to the PSO algorithm. [ABSTRACT FROM AUTHOR]

The enormous number of articles published daily on the Internet, by a diverse array of authors, often offers misleading or unwanted information, rendering activities such as sports betting riskier. As a result, extracting meaningful and reliable information from these sources becomes a time-consuming and near impossible task. In this context, labeling articles as objective or subjective is not a simple natural language processing task because subjectivity can take several forms. With the rise of online sports betting due to the revolution in Internet and mobile technology, an automated system capable of sifting through all these data and finding relevant sources in a reasonable amount of time presents itself as a desirable and marketable product. In this work, we present a framework for the classification of sports articles composed of three stages: The first stage extracts articles from web pages using text extraction libraries, parses the text and then tags words using Stanford's parts of speech tagger; the second stage extracts unique syntactic and semantic features, and reduces them using our modified cortical algorithm (CA)—hereafter CA*—while the third stage classifies these texts as objective or subjective. Our framework was tested on a database containing 1000 articles, manually labeled using Amazon's crowdsourcing tool, Mechanical Turk; and results using CA, CA*, support vector machines and one of its soft computing variants (LMSVM) as classifiers were reported. A testing accuracy of 85.6% was achieved on a fourfold cross-validation with a 40% reduction in features using CA* that was trained using an entropy weight update rule and a cross-entropy cost function. [ABSTRACT FROM AUTHOR]

In this study we represent malware as opcode sequences and detect it using a deep belief network (DBN). Compared with traditional shallow neural networks, DBNs can use unlabeled data to pretrain a multi-layer generative model, which can better represent the characteristics of data samples. We compare the performance of DBNs with that of three baseline malware detection models, which use support vector machines, decision trees, and the k-nearest neighbor algorithm as classifiers. The experiments demonstrate that the DBN model provides more accurate detection than the baseline models. When additional unlabeled data are used for DBN pretraining, the DBNs perform better than the other detection models. We also use the DBNs as an autoencoder to extract the feature vectors of executables. The experiments indicate that the autoencoder can effectively model the underlying structure of input data and significantly reduce the dimensions of feature vectors. [ABSTRACT FROM AUTHOR]

Twin support vector regression (TSVR) aims at finding 휖-insensitive up- and down-bound functions for the training points by solving a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in the conventional SVR. So TSVR works faster than SVR in theory. However, TSVR gives equal emphasis to the points above the up-bound and below the down-bound, which leads to the same influences on the regression function. In fact, points in different positions have different effects on the regressor. Inspired by it, we propose an asymmetric ν-twin support vector regression based on pinball loss function (Asy- ν-TSVR). The new algorithm can effectively control the fitting error by tuning the parameters ν and p. Therefore, it enhances the generalization ability. Moreover, we study the distribution of samples and give the upper bounds for the samples locating in different positions. Numerical experiments on one artificial dataset, eleven benchmark datasets and a real wheat dataset demonstrate the validity of our proposed algorithm. [ABSTRACT FROM AUTHOR]

An application of classifier fusion technique is presented to improve the performance of automated reservoir facies identification system. The algorithm presented in this study uses three well-known classifiers, namely Bayesian, k-nearest neighbor (kNN), and support vector machine (SVM) to automatically identify four defined facies of Asmari Formation from log-derived amplitude versus offset (AVO) attributes. Fuzzy Sugeno integral (FSI) method is then employed to combine the outputs of three investigated classifiers and increase the consistency of reservoir facies identification process. The experimental results obtained from applying the presented algorithm on data related to three wells drilled in Asmari Formation provide evidence of the effectiveness of the proposed algorithm regarding true positive (TP), false positive (FP), and classification accuracy criteria. [ABSTRACT FROM AUTHOR]

Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and theory of TWSVM, including the theory of proximal support vector machine, generalized eigenvalue proximal support vector machine, and TWSVM. Then, we focus on the various improvements made to TWSVM, mainly including least squares twin support vector machine, smooth twin support vector machine, regularized twin support vector machine, projection twin support vector machine, and modified TWSVM on the model selection problem. These newly emerging algorithms greatly expand the applications of TWSVM. In recent years, there is a lot of research on application of TWSVM. Next, we list some research on application of TWSVM in detail. Finally, we try to provide a comprehensive view of these advances in TWSVM and discuss the direction of future research. [ABSTRACT FROM AUTHOR]

In this paper, we propose a matrix version extension for linear regularization projection twin support vector machine presented by Shao et al. (Knowl Based Syst 37:203-210, ), named as linear projection twin support matrix machine [linear projection twin support matrix machine (PTSMM)], for 2d image data classification. In order to discuss the nonlinear version of PTSMM, a new matrix kernel function is introduced and based on which, we provide a nonlinear PTSMM algorithm with a detailed theoretical derivation. To examine the effectiveness of the presented linear and nonlinear PTSMM, we perform comparative experiments with three linear classifiers support tensor machines, twin support tensor machine and proximal support tensor machine on ORL, YALE and AR databases. Experimental results show that our methods are effective and efficient. [ABSTRACT FROM AUTHOR]

In many classification problems, the class distribution is imbalanced. Learning from the imbalance data is a remarkable challenge in the knowledge discovery and data mining field. In this paper, we propose a scaling kernel-based support vector machine (SVM) approach to deal with the multi-class imbalanced data classification problem. We first use standard SVM algorithm to gain an approximate hyperplane. Then, we present a scaling kernel function and calculate its parameters using the chi-square test and weighting factors. Experimental results on KEEL data sets show the proposed algorithm can resolve the classifier performance degradation problem due to data skewed distribution and has a good generalization. [ABSTRACT FROM AUTHOR]

In many real-world applications in the areas of data mining, the distributions of testing data are different from that of training data. And on the other hand, many data are often represented by multiple views which are of importance to learning. However, little work has been done for it. In this paper, we explored to leverage the multi-view information across different domains for knowledge transfer. We proposed a novel transfer learning model which integrates the domain distance and view consistency into a 2-view support vector machine framework, namely DV2S. The objective of DV2S is to find the optimal feature mapping such that under the projections the classification margin is maximized, while both the domain distance and the disagreement between multiple views are minimized simultaneously. Experiments showed that DV2S outperforms a variety of state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

This paper presents several criteria for partition of classes for the support vector machine based hierarchical classification. Our clustering algorithm combines support vector machine and binary tree, it is a divisive (top-down) approach in which a set of classes is automatically separated into two smaller groups at each node of the hierarchy, it splits the classes based on the normalized cuts clustering algorithm. Our clustering algorithm considers the involved classes rather than the individual data samples. In the new proposed measures, similarity between classes is determined based on boundary complexity. In these measures, concepts such as the upper bound of error and Kolmogorov complexity are used. We reported results on several data sets and five distance/similarity measures. Experimental results demonstrate the superiority of the proposed measures compared to other measures; even when applied to nonlinearly separable data, the new criteria perform well. [ABSTRACT FROM AUTHOR]

Motivated by the support vector data description, a classical one-class support vector machine, and the twin support vector machine classifier, this paper formulates a twin support vector hypersphere (TSVH) classifier, a novel binary support vector machine (SVM) classifier that determines a pair of hyperspheres by solving two related SVM-type quadratic programming problems, each of which is smaller than that of a conventional SVM, which means that this TSVH is more efficient than the classical SVM. In addition, the TSVH successfully avoids matrix inversion compared with the twin support vector machine, which indicates learning algorithms of the SVM can be easily extended to this TSVH. Computational results on several synthetic as well as benchmark data sets indicate that the proposed TSVH is not only faster, but also obtains better generalization. [ABSTRACT FROM AUTHOR]

Most state-of-the-art speaker recognition systems are based on discriminative learning approaches. On the other hand, generative Gaussian mixture models (GMM) have been widely used in speaker recognition during the last decades. In an earlier work, we proposed an algorithm for discriminative training of GMM with diagonal covariances under a large margin criterion. In this paper, we propose an improvement of this algorithm, which has the major advantage of being computationally highly efficient, thus well suited to handle large-scale databases. We also develop a new strategy to detect and handle the outliers that occur in the training data. To evaluate the performances of our new algorithm, we carry out full NIST speaker identification and verification tasks using NIST-SRE'2006 data, in a Symmetrical Factor Analysis compensation scheme. The results show that our system significantly outperforms the traditional discriminative support vector machines (SVM)-based system of SVM-GMM supervectors, in the two speaker recognition tasks. [ABSTRACT FROM AUTHOR]

In this paper, a simple and linearly convergent Lagrangian support vector machine algorithm for the dual of the twin support vector regression (TSVR) is proposed. Though at the outset the algorithm requires inverse of matrices, it has been shown that they would be obtained by performing matrix subtraction of the identity matrix by a scalar multiple of inverse of a positive semi-definite matrix that arises in the original formulation of TSVR. The algorithm can be easily implemented and does not need any optimization packages. To demonstrate its effectiveness, experiments were performed on well-known synthetic and real-world datasets. Similar or better generalization performance of the proposed method in less training time in comparison with the standard and twin support vector regression methods clearly exhibits its suitability and applicability. [ABSTRACT FROM AUTHOR]

A feature-weighted Support Vector Machine regression algorithm is introduced in this paper. We note that the classical SVM is based on the assumption that all the features of the sample points supply the same contribution to the target output value. However, this assumption is not always true in real problems. In the proposed new algorithm, we give different weight values to different features of the samples in order to improve the performance of SVM. In our algorithm, firstly, a measure named grey correlation degree is applied to evaluate the correlation between each feature and the target problem, and then the values of the grey correlation degree are used as weight values assigned to the features. The proposed method is tested on sample stock data sets selected from China Shenzhen A-share market. The result shows that the new version of SVM can improve the accuracy of the prediction. [ABSTRACT FROM AUTHOR]

A novel method, namely ensemble support vector machine with segmentation (SeEn-SVM), for the classification of imbalanced datasets is proposed in this paper. In particular, vector quantization algorithm is used to segment the majority class and hence generates some small datasets that are of less imbalance than original one, and two different weighted functions are proposed to integrate all the results of basic classifiers. The goal of the SeEn-SVM algorithm is to improve the prediction accuracy of the minority class, which is more interesting for people. The SeEn-SVM is applied to six UCI datasets, and the results confirmed its better performance than previously proposed methods for imbalance problem. [ABSTRACT FROM AUTHOR]

For multi-class classification problem, a novel algorithm, called as multiple birth support vector machine (MBSVM), is proposed, which can be considered as an extension of twin support vector machine. Our MBSVM has been compared with the several typical support vector machines. From theoretical point of view, it has been shown that its computational complexity is remarkably low, especially when the class number K is large. Based on our MBSVM, the dual problems of MBSVM are equivalent to symmetric mixed linear complementarity problems to which successive overrelaxation (SOR) can be directly applied. We establish our SOR algorithm for MBSVM. The SOR algorithm handles one data point at a time, so it can process large dataset that need no reside in memory. From practical point of view, its accuracy has been validated by the preliminary numerical experiments. [ABSTRACT FROM AUTHOR]

In this paper, extreme learning machine (ELM) for ε-insensitive error loss function-based regression problem formulated in 2-norm as an unconstrained optimization problem in primal variables is proposed. Since the objective function of this unconstrained optimization problem is not twice differentiable, the popular generalized Hessian matrix and smoothing approaches are considered which lead to optimization problems whose solutions are determined using fast Newton-Armijo algorithm. The main advantage of the algorithm is that at each iteration, a system of linear equations is solved. By performing numerical experiments on a number of interesting synthetic and real-world datasets, the results of the proposed method are compared with that of ELM using additive and radial basis function hidden nodes and of support vector regression (SVR) using Gaussian kernel. Similar or better generalization performance of the proposed method on the test data in comparable computational time over ELM and SVR clearly illustrates its efficiency and applicability. [ABSTRACT FROM AUTHOR]

This paper studies the state-of-the-art classification techniques for electroencephalogram (EEG) signals. Fuzzy Functions Support Vector Classifier, Improved Fuzzy Functions Support Vector Classifier and a novel technique that has been designed by utilizing Particle Swarm Optimization and Radial Basis Function Networks (PSO-RBFN) have been studied. The classification performances of the techniques are compared on standard EEG datasets that are publicly available and used by brain-computer interface (BCI) researchers. In addition to the standard EEG datasets, the proposed classifier is also tested on non-EEG datasets for thorough comparison. Within the scope of this study, several data clustering algorithms such as Fuzzy C-means, K-means and PSO clustering algorithms are studied and their clustering performances on the same datasets are compared. The results show that PSO-RBFN might reach the classification performance of state-of-the art classifiers and might be a better alternative technique in the classification of EEG signals for real-time application. This has been demonstrated by implementing the proposed classifier in a real-time BCI application for a mobile robot control. [ABSTRACT FROM AUTHOR]

This paper presents a new classification method utilizing distance-based decision surface with nearest neighbor projection approach, called DDC. Kernel type of DDC has been extended to take into account the effective nonlinear structure of the data. DDC has some properties: (1) does not need conventional learning procedure (as k-NN algorithm), (2) does not need searching time to locate the k-nearest neighbors, and (3) does not need optimization process unlike some classification methods such as Support Vector Machine (SVM). In DDC, we compute the weighted average of distances to all the training samples. Unclassified sample will be classified as belonging to a class that has the minimum obtained distance. As a result, by such a rule we can derive a formula that can be used as the decision surface. DDC is tested on both synthetic and real-world data sets from the UCI repository, and the results were compared with k-NN, RBF Network, and SVM. The experimental results indicate DDC outperforms k-NN in the most experiments and the results are comparable to or better than SVM with some data sets. [ABSTRACT FROM AUTHOR]

Twin support vector machine (TSVM) is a new machine learning algorithm, which aims at finding two nonparallel planes for each class. In order to do so, one needs to resolve a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one. However, when constructing the classification plane for one class, a large number of samples of this class are considered in the objective function, but only fewer samples in the other class are considered, which easily results in over-fitting problem. In addition, the same penalties are given to each misclassified samples in the TSVM. In fact, the misclassified samples have different effects on the decision of the hyper-plane. In order to overcome these two disadvantages, by introducing the rough set theory into ν-TSVM, we propose a rough margin-based ν-TSVM in this paper. In the proposed algorithm, the different points in the different positions are proposed to have different effects on the separating hyper-plane. We firstly construct rough lower margin, rough upper margin, and rough boundary in the ν-TSVM and then give the different penalties to the different misclassified samples according to their positions. The new classifier can avoid the over-fitting problem to a certain extent. Numerical experiments on one artificial dataset and six benchmark datasets demonstrate the feasibility and validity of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The parameter values of kernel function affect classification results to a certain extent. In the paper, a multiclass classification model based on improved least squares support vector machine (LSSVM) is presented. In the model, the non-sensitive loss function is replaced by quadratic loss function, and the inequality constraints are replaced by equality constraints. Consequently, quadratic programming problem is simplified as the problem of solving linear equation groups, and the SVM algorithm is realized by least squares method. When the LSSVM is used in multiclass classification, it is presented to choose parameter of kernel function on dynamic, which enhances preciseness rate of classification. The Fibonacci symmetry searching algorithm is simplified and improved. The changing rule of kernel function searching region and best shortening step is studied. The best multiclass classification results are obtained by means of synthesizing kernel function searching region and best shortening step. The simulation results show the validity of the model. [ABSTRACT FROM AUTHOR]

When support vector machine (SVM) classifier is applied to image semantic annotation, it usually encounters the problem of excessive training samples. In this paper, we propose a novel method, which is by combining learning vector quantization (LVQ) technique and SVM classifier, to improve annotation accuracy and speed. Affinity propagation algorithm-based LVQ technique is used to optimize the training set, and a few number of optimized representative feature vectors are used to train SVM. This approach not only meets the small sample size characteristic of SVM, but also greatly accelerates the training and annotating process. Comparative experimental studies confirm the validity of the proposed method. [ABSTRACT FROM AUTHOR]

The prediction of secondary structure is an important topic in the field of bioinformatics, even if the methods have matured, and development of the algorithms is a far less active area than a decade ago. Accurate prediction is very useful to biologists in its own right, but it is worth pointing out that it is also an essential component of tertiary structure prediction, which in contrast is far from solved and continues to be a highly active area of research. In addition, sequence comparison methods have more recently incorporated local structure tracks. The extra information utilized by the new methods has led to considerable improvements in fold recognition and alignment accuracy. In this paper, a novel method for protein secondary structure prediction is presented. Using evolutionary information contained in amino acid's physicochemical properties, position-specific scoring matrix generated by PSI-BLAST and HMMER3 profiles as input to hybrid back propagation system, secondary structure can be predicted at significantly increased accuracy. Based on knowledge discovery theory based on inner cognitive mechanism (KDTICM) theory, we have constructed a compound pyramid model approach, which is composed of four layers of the intelligent interface and integrated in several ways, such as hybrid back propagation method (HBP), modified knowledge discovery in databases (KDD*), hybrid SVM method (HSVM) and so on. Experiments on three standard datasets (RS126, CB513 and CASP8) show that CPM is capable of producing the higher Q and SOV scores than that achieved by existing widely used schemes such as PSIPRED, PHD, Predator, as well as previously developed prediction methods. On the RS126 and CB513 datasets, it achieves a Q and SOV99 score are considerably higher than the best reported scores, respectively. It is also tested on target proteins of critical assessment of protein structure prediction experiment (CASP8) and achieves better results than the traditional methods, including the popular PSIPRED method over overall prediction accuracy. Available: . [ABSTRACT FROM AUTHOR]

The Support Vector Machine (SVM) is an interesting classifier with excellent power of generalization. In this paper, we consider applying the SVM to semi-supervised learning. We propose using an additional criterion with the standard formulation of the semi-supervised SVM ( S VM) to reinforce classifier regularization. Since, we deal with nonconvex and combinatorial problem, we use a genetic algorithm to optimize the objective function. Furthermore, we design the specific genetic operators and certain heuristics in order to improve the optimization task. We tested our algorithm on both artificial and real data and found that it gives promising results in comparison with classical optimization techniques proposed in literature. [ABSTRACT FROM AUTHOR]

The present paper investigates the application of the multi-layer perceptron (MLP) to the task of categorizing texts based on their authors’ style. This task is of particular importance for information retrieval applications involving very large document databases. The emphasis of this article is to determine the extent to which the MLP model can be fine-tuned to successfully analyse such data, uncovering the stylistic differences among authors. The MLP-based method is compared and contrasted to statistical techniques, such as discriminant analysis, that are widely used in stylistic studies. The comparison of the methods is based on their classification performance, to provide an objective evaluation of the advantages of each method. A second aim of the study presented here is to compare the effectiveness of distinct features in the task of uncovering the author identity for each method. To evaluate to a greater depth the effectiveness of the entire approach, the results of the proposed MLP-based method are compared to those of established approaches, such as the support vector machines (SVM), using both the original parameters employed by the MLP as well as term frequency–inverse document frequency (TF–IDF) parameters, and the cascade correlation approach. It is found that the proposed MLP-based approach possesses a number of advantages, such as high classification accuracy, broadly comparable to that of the SVM, coupled with the ability to algorithmically reduce the set of parameters used without adversely affecting the classification accuracy. [ABSTRACT FROM AUTHOR]

This paper presents a new method for fault classification in series-compensated transmission line using multiclass support vector machine (MCSVM) and multi class extreme learning machine (MCELM). These methods use the information obtained from the wavelet decomposition of fault current signals for fault classification. Using MATLAB simulink, data set has been generated with different types of fault and system variables, which include the fault resistance, fault distance, load angle and fault inception angle. The proposed method has been tested on a 400-kV, 300-km transmission line under variety of fault conditions. The performance of MCSVM and MCELM is compared in terms of training time and classification accuracy. The comparisons have been made for both One-Versus-One and One-Versus-Rest methods of SVMs and ELMs. Results show that MCELMs need less training time compared to MCSVMs, and the classification accuracy of MCELMs is more or less similar to MCSVMs. The feasibility of the proposed methods is also tested on a practical 220-kV series-compensated transmission line, and the results obtained are quite promising. [ABSTRACT FROM AUTHOR]

The human heart is a complex system that reveals many clues about its condition in its electrocardiogram (ECG) signal, and ECG supervising is the most important and efficient way of preventing heart attacks. ECG analysis and recognition are both important and tempting topics in modern medical research. The purpose of this paper is to develop an algorithm which investigates kernel method, locally linear embedding (LLE), principal component analysis (PCA), and support vector machine(SVM) algorithms for dimensionality reduction, features extraction, and classification for recognizing and classifying the given ECG signals. In order to do so, a nonlinear dimensionality reduction kernel method based LLE is proposed to reduce the high dimensions of the variational ECG signals, and the principal characteristics of the signals are extracted from the original database by means of the PCA, each signal representing a single and complete heart beat. SVM method is applied to classify the ECG data into several categories of heart diseases. Experimental results obtained demonstrated that the performance of the proposed method was similar and sometimes better when compared to other ECG recognition techniques, thus indicating a viable and accurate technique. [ABSTRACT FROM AUTHOR]

Multivariate calibration is a classic problem in the analytical chemistry field and frequently solved by partial least squares (PLS) and artificial neural networks (ANNs) in the previous works. The spaciality of multivariate calibration is high dimensionality with small sample. Here, we apply support vector regression (SVR) as well as ANNs, and PLS to the multivariate calibration problem in the determination of the three aromatic amino acids ( phenylalanine, tyrosine and tryptophan) in their mixtures by fluorescence spectroscopy. The results of the leave-one-out method show that SVR performs better than other methods, and appear to be one good method for this task. Furthermore, feature selection is performed for SVR to remove redundant features and a novel algorithm named Prediction RIsk based FEature selection for support vector Regression (PRIFER) is proposed. Results on the above multivariate calibration data set show that PRIFER is a powerful tool for solving the multivariate calibration problems. [ABSTRACT FROM AUTHOR]

Support Vector Machine (SVM) classifiers are high-performance classification models devised to comply with the structural risk minimization principle and to properly exploit the kernel artifice of nonlinearly mapping input data into high-dimensional feature spaces toward the automatic construction of better discriminating linear decision boundaries. Among several SVM variants, Least-Squares SVMs (LS-SVMs) have gained increased attention recently due mainly to their computationally attractive properties coming as the direct result of applying a modified formulation that makes use of a sum-squared-error cost function jointly with equality, instead of inequality, constraints. In this work, we present a flexible hybrid approach aimed at augmenting the proficiency of LS-SVM classifiers with regard to accuracy/generalization as well as to hyperparameter calibration issues. Such approach, named as Mixtures of Weighted Least-Squares Support Vector Machine Experts, centers around the fusion of the weighted variant of LS-SVMs with Mixtures of Experts models. After the formal characterization of the novel learning framework, simulation results obtained with respect to both binary and multiclass pattern classification problems are reported, ratifying the suitability of the novel hybrid approach in improving the performance issues considered. [ABSTRACT FROM AUTHOR]

The success of support vector machine depends upon its parameters. The leave-one-out (LOO) method provides a quantitative criterion for selecting those parameters. However, one shortcoming of the LOO method is that it is highly time consuming. An effective approach is to approximate the LOO error by an upper bound. This paper is concerned with the support vector ordinal regression machine (SVORM). Two bounds of the LOO error for SVORM are presented. The first bound is based on the geometrical concept of a span. The second one is based on the concept of support vector. Preliminary numerical experiments show the validity of the bounds. [ABSTRACT FROM AUTHOR]

The main aim of this paper is to predict NO and NO2 concentrations 4 days in advance by comparing two artificial intelligence learning methods, namely, multi-layer perceptron and support vector machines, on two kinds of spatial embedding of the temporal time series. Hourly values of NO and NO2 concentrations, as well as meteorological variables were recorded in a cross-road monitoring station with heavy traffic in Szeged, in order to build a model for predicting NO and NO2 concentrations several hours in advance. The prediction of NO and NO2 concentrations was performed partly on the basis of their past values, and partly on the basis of temperature, humidity and wind speed data. Since NO can be predicted more accurately, its values were considered primarily when forecasting NO2. Time series prediction can be interpreted in a way that is suitable for artificial intelligence learning. Two effective learning methods, namely, multi-layer perceptron and support vector regression are used to provide efficient non-linear models for NO and NO2 time series predictions. Multi-layer perceptron is widely used to predict these time series, but support vector regression has not yet been applied for predicting NO and NO2 concentrations. Three commonly used linear algorithms were considered as references: 1-day persistence, average of several day persistence and linear regression. Based on the good results of the average of several day persistence, a prediction scheme was introduced, which forms weighted averages instead of simple ones. The optimization of these weights was performed with linear regression in linear case and with the learning methods mentioned in non-linear case. Concerning the NO predictions, the non-linear learning methods give significantly better predictions than the reference linear methods. In the case of NO2, the improvement of the prediction is considerable, however, it is less notable than for NO. [ABSTRACT FROM AUTHOR]

Three extensions to the Kernel-AdaTron training algorithm for Support Vector Machine classifier learning are presented. These extensions allow the trained classifier to adhere more closely to the constraints imposed by Support Vector Machine theory. The results of these modifications show improvements over the existing Kernel-AdaTron algorithm. A method of parameter optimisation for polynomial kernels is also proposed. [ABSTRACT FROM AUTHOR]