Search

In modern network security and management architecture, Classification of Traffic plays a major role in recent years. In particular, the process of intrusion detection and QOS control is considered as a essential components in traffic classification. Recent method uses statistical feature related classification approach with machine learning techniques for Traffic classification. Earlier method used several machine learning classifiers for classification purpose. Due to the lack of classifier performance in each aspect, the overall classification of traffic affected while least size of training data are used. To deal with this process, this paper proposes an efficient classifier called adaptive nearest cluster based classifier (ANCC-classifier). The proposed classifier classifies the traffic by collecting the statistical feature based correlated information. Such information is obtained by analysing the normal and abnormal flow of network. The present system is analysed in theoretical and experiential perspectives. Experimental result provides improved performance when compared with the other state of art methods.

In multi-label learning, each training example is associated with a set of labels and the task is to predict the proper label set for each unseen instance. The widely known binary relevance method (BR) for multi-label classification considers each label as an independent binary problem. It is ignored in the literature due to inadequacy of not considering label correlations. In this paper, we present our double layer based classifier chains method (DCC), which overcomes dis- advantages of BR and inherits the benefit of classifier chain method (CC). This algorithm decomposes the multi-label classification problem into two classification processes to generate classifier chain. Each classifier in the chain is respon- sible for learning and predicting the binary association of the label given the attribute space expanded by all prior binary relevance predictions in the chain. This chaining allows DCC to take into account correlations in the label space. We also extend this approach further in an ensemble framework. An extensive evaluation covers a broad range of multi-label datasets with a variety of evaluation measures specifically designed for multi-label classification. Experiments on bench- mark datasets validate the effectiveness of proposed approach comparing with state-of-art methods in terms of average ranking. �

Twin-hypersphere support vector machine (THSVM) for binary pattern recognition aims at generating two hyperspheres in the feature space such that each hypersphere contains as many as possible samples in one class and is as far as possible from the other one. THSVM has a fast learning speed since it solves two small sized support vector machine (SVM)-type quadratic programming problems (QPPs). However, it only simply considers the prior class-based structural information in the optimization problems. In this paper, a structural information-based THSVM (STHSVM) classifier for binary classification is presented. This proposed STHSVM focuses on the cluster-based structural information of the corresponding class in each optimization problem, which is vital for designing a good classifier in different real-world problems. In addition, it also leads to a fast learning speed since this STHSVM solves a series of smaller-sized QPPs compared with THSVM. Experimental results demonstrate that STHSVM is superior in generalization performance to other classifiers.

Most of machine learning problems assume, that we have at our disposal objects originating from two or more classes. By learning from a representative training set a classifier is able to estimate proper decision boundaries. However, in many real-life problems obtaining objects from some of the classes is difficult, or even impossible. In such cases, we are dealing with one- class classification, or learning in the absence of counterexamples. Such recognition systems must display a high robustness to new, unseen objects that may belong to an unknown class. That is why ensemble learning has become an attractive perspective in this field. In our work, we propose a novel one-class ensemble classifier, based on wagging. A weighted version of boosting is used, and the output weights for each object are used directly in the process of training Weighted One-Class Support Vector Machines. This introduces a diversity into the pool of one-class classifiers and extends the competence of formed ensemble. Experimental analysis, carried out on a number of benchmarks and backed-up with statistical analysis proves that the proposed method can outperform state-of-the-art ensembles dedicated to one-class classification.

Maximum likelihood classifier (MLC) and support vector machines (SVM) are two commonly used approaches in machine learning. MLC is based on Bayesian theory in estimating parameters of a probabilistic model, whilst SVM is an optimization based nonparametric method in this context. Recently, it is found that SVM in some cases is equivalent to MLC in probabilistically modeling the learning process. In this paper, MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making. In total four groups of data are used for evaluations, covering sonar, vehicle, breast cancer, and DNA sequences. The data samples are characterized in terms of Gaussian/non-Gaussian distributed and balanced/unbalanced samples which are then further used for performance assessment in comparing the SVM and the combined SVM-MLC classifier. Interesting results are reported to indicate how the combined classifier may work under various conditions. Accepted on May 11, 2015

One of the most important tasks in pattern, machine learning, and data mining is classification problem. Introducing a general classifier is a challenge for pattern recognition communities, which enables one to learn each problem׳s dataset. Many classifiers have been proposed to learn any problem thus far. However, many of them have their own positive and negative aspects. So they are good only for specific problems. But there is no strong solution to recognize which classifier is better or good for a specific problem. Fortunately, ensemble learning provides a good way to have a near-optimal classifying system for any problem. One of the most challenging problems in classifier ensemble is introducing a suitable ensemble of base classifiers. Every ensemble needs diversity. It means that if a group of classifiers is to be a successful ensemble, they must be diverse enough to cover their errors. Therefore, during ensemble creation, a mechanism is needed to ensure that the ensemble classifiers are diverse. Sometimes this mechanism can select/remove a subset of base classifiers with respect to maintaining the diversity of the ensemble. This paper proposes a novel method, named the Classifier Selection Based on Clustering (CSBS), for ensemble creation. To insure diversity in ensemble classifiers, this method uses the clustering of classifiers technique. Bagging is used to produce base classifiers. During ensemble creation, every type of base classifier is the same as a decision tree classier or a multilayer perceptron classifier. After producing a number of base classifiers, CSBC partitions them by using a clustering algorithm. Then CSBC produces a final ensemble by selecting one classifier from each cluster. Weighted majority vote method is used as an aggregator function. In this paper we investigate the influence of cluster number on the performance of the CSBC method; we also probe how we can select a good approximate value for cluster number in any dataset. We base our study on a large number of real datasets of UCI repository to reach a definite result.

One of the main challenges in pedestrian classification is partial occlusion. This study presents a new method for pedestrian classification with partial occlusion handling. The proposed method involves a set of part-based classifiers trained on histogram of oriented gradients features derived from non-occluded pedestrian data set. The score of each part classifier is then employed to weight features used to train a second stage full-body classifier. The full-body classifier based on local weighted linear kernel support vector machine is trained using both non-occluded and artificially generated partial occlusion pedestrian dataset. The new kernel allows to significantly focus on the non-occluded parts and reduce the impact of the occluded ones. Experimental results on real-world dataset, with both partially occluded and non-occluded data, show high performance of the proposed method compared with other state-of-the-art methods.

A novel framework for music genre classification, namely the joint sparse low-rank representation (JSLRR) is proposed in order to: 1) smooth the noise in the test samples, and 2) identify the subspaces that the test samples lie onto. An efficient algorithm is proposed for obtaining the JSLRR and a novel classifier is developed, which is referred to as the JSLRR-based classifier. Special cases of the JSLRR-based classifier are the joint sparse representation-based classifier and the low-rank representation-based one. The performance of the three aforementioned classifiers is compared against that of the sparse representation-based classifier, the nearest subspace classifier, the support vector machines, and the nearest neighbor classifier for music genre classification on six manually annotated benchmark datasets. The best classification results reported here are comparable with or slightly superior than those obtained by the state-of-the-art music genre classification methods.

This paper examines the construction issues of classifier ensembles for bankruptcy prediction.The first issue focuses on the classification techniques, which are based on MLP, SVM, and DT.The second issue is the combination method, which is based on bagging and boosting.The third issue is based on examining different numbers of combined classifiers.We show that DT ensembles composed of 80-100 classifiers using the boosting method perform best. The aim of bankruptcy prediction in the areas of data mining and machine learning is to develop an effective model which can provide the higher prediction accuracy. In the prior literature, various classification techniques have been developed and studied, in/with which classifier ensembles by combining multiple classifiers approach have shown their outperformance over many single classifiers. However, in terms of constructing classifier ensembles, there are three critical issues which can affect their performance. The first one is the classification technique actually used/adopted, and the other two are the combination method to combine multiple classifiers and the number of classifiers to be combined, respectively. Since there are limited, relevant studies examining these aforementioned disuses, this paper conducts a comprehensive study of comparing classifier ensembles by three widely used classification techniques including multilayer perceptron (MLP) neural networks, support vector machines (SVM), and decision trees (DT) based on two well-known combination methods including bagging and boosting and different numbers of combined classifiers. Our experimental results by three public datasets show that DT ensembles composed of 80-100 classifiers using the boosting method perform best. The Wilcoxon signed ranked test also demonstrates that DT ensembles by boosting perform significantly different from the other classifier ensembles. Moreover, a further study over a real-world case by a Taiwan bankruptcy dataset was conducted, which also demonstrates the superiority of DT ensembles by boosting over the others.

In this paper, we propose a new design methodology of granular fuzzy classifiers based on a concept of information granularity and information granules. The classifier uses the mechanism of information granulation with the aid of which the entire input space is split into a collection of subspaces. When designing the proposed fuzzy classifier, these information granules are constructed in a way they are made reflective of the geometry of patterns belonging to individual classes. Although the elements involved in the generated information granules (clusters) seem to be homogeneous with respect to the distribution of patterns in the input (feature) space, they still could exhibit a significant level of heterogeneity when it comes to the class distribution within the individual clusters. To build an efficient classifier, we improve the class homogeneity of the originally constructed information granules (by adjusting the prototypes of the clusters) and use a weighting scheme as an aggregation mechanism.

A new classifier designed based on linear regression classification (LRC) classifier and simple-fast representation-based classifier (SFR), named double linear regression classification (DLRC) classifier, is proposed for image recognition in this paper. As we all know, the traditional LRC classifier only uses the distance between test image vectors and predicted image vectors of the class subspace for classification. And the SFR classifier uses the test image vectors and the nearest image vectors of the class subspace to classify the test sample. However, the DLRC classifier computes out the predicted image vectors of each class subspace and uses all the predicted vectors to construct a novel robust global space. Then, the DLRC utilizes the novel global space to get the novel predicted vectors of each class for classification. A mass number of experiments on AR face database, JAFFE face database, Yale face database, Extended YaleB face database, and PIE face database are used to evaluate the performance o...

In view of the higher mistaken-detection rate problem of the human face detection in complex conditions, we put forward an improved algorithm. This article Proposes one kind of the method Which unifies the level of difference between the threshold and the feature value of the weak classifier with the weak classifier's overall error rate. Compared to the method which only based on the overall classification error rate to update the weights, this method can achieve higher detection rate while reduces the mistaken-detection rate. This article redefines the training error which is caused When we train the weak classifier, and Proposes MCE-AdaBoost algorithm. The new definition of training error will pay more attention to the error Which erroneously estimates the face Sample as non-face sample; this much more conforms to face detection of this special target detection issue. The experimental results show that MCE-AdaBoost algorithm can effectively improve the detection Performance of the final classifier.

This work proposes an effective synergy of the Intervals@? Number k-nearest neighbor (INknn) classifier, that is a granular extension of the conventional knn classifier in the metric lattice of Intervals@? Numbers (INs), with the gravitational search algorithm (GSA) for stochastic search and optimization. Hence, the gsaINknn classifier emerges whose effectiveness is demonstrated here on 12 benchmark classification datasets. The experimental results show that the gsaINknn classifier compares favorably with alternative classifiers from the literature. The far-reaching potential of the gsaINknn classifier in computing with words is also delineated.

In this paper, a novel classifier based on two-phase test sample sparse representation (TPTSSR) classifier and coarse k nearest neighbor (C-kNN) classifier, called novel classification rule of two-phase test sample sparse representation (NCR-TPTSSR) classifier, is proposed for image recognition. Being similar to TPTSSR classifier and C-kNN classifier, NCR-TPTSSR classifier also uses the two phases to classify the test sample. However, the classification rule of NCR-TPTSSR classifier is different to the decision rule of TPTSSR classifier and C-kNN classifier. A large number of experiments on FERET face database, AR face database, JAFFE face database and PolyU FKP database are used to evaluate the proposed algorithm. The experimental results demonstrate that the proposed method achieves better recognition rate than TPTSSR classifier, C-kNN classifier, nearest feature center (NFC) classifier, nearest feature line (NFL) classifier, nearest neighbor (NN) and so on.

Summary This paper analyses the botnet traffic using Ensemble of classifier algorithm to find out bot evidence. We used ISCX dataset for training and testing purpose. We extracted the features of both training and testing datasets. After extracting the features of this dataset, we bifurcated these features into two classes, normal traffic and botnet traffic and provide labelling. Thereafter using modern data mining tool, we have applied ensemble of classifier algorithm. Our experimental results show that the performance for finding bot evidence using ensemble of classifiers is better than single classifier. Ensemble based classifiers perform better than single classifier by either combining powers of multiple algorithms or introducing diversification to the same classifier by varying input in bot analysis. Our results are showing that by using voting method of ensemble based classifier accuracy is increased up to 96.41% from 93.37%.

Automated electrocardiogram (ECG) beat classification is an important component of heart monitoring systems used for raising an emergency call during sudden cardiac disorder of patients. An automatic ECG beat classifier is proposed to exploit the bandwidth features that are exclusively extracted from analytic intrinsic mode functions (IMFs). The proposed methodology employs artificial bee colony (ABC) algorithm-based least-squares support vector machines (LSSVM) classifier to classify ECG beat types using radial basis function kernel. Simulation results illustrate that proposed classifier gives the best results with second IMF. This novelty lies in its unique combination of ABC with LSSVM classifier that efficiently exploits bandwidth features for automatic classification of ECG beats.

A new learning method for classification problems that is suitable for integrated circuit implementation is presented. The method, which outperforms current approaches in many data sets, is based on a structural description of the learning set represented by a planar graph. The final classification function is composed of a hierarchical mixture of local experts, which yields a large margin classifier for the whole learning set. Since it is based only on distance calculations, on-chip learning can also be executed. The method is also appropriate for online and incremental learning, since model parameters are obtained directly from the data set, without need of user interaction for learning.

Support vector machines, which maximize the margin from patterns to the separation hyperplane subject to correct classification, have received remarkable success in machine learning. Margin error bounds based on Hilbert spaces have been introduced in the literature to justify the strategy of maximizing the margin in SVM. Recently, there has been much interest in developing Banach space methods for machine learning. Large margin classification in Banach spaces is a focus of such attempts. In this letter we establish a margin error bound for the SVM on reproducing kernel Banach spaces, thus supplying statistical justification for large-margin classification in Banach spaces.

This article presents a new SVM classifier for the prediction of the extended early-stage (ES) protein structures. The classifier is based on physicochemical features and position-specific scoring matrix (PSSM). Experiments have shown that prediction results for specific classes are significantly better than those already obtained.

This paper presents a novel methodology based on machine learning for assessing the power system transient stability (TS) in real-time using synchronized phasor measurements as input in order to predict the power system TS status related to the single machine equivalent (SIME) results. Based on the probabilistic models of input parameters, such as load variation and the occurrence of contingencies, Monte Carlo-type simulation is performed off-line to iteratively evaluate the system TS responses using the SIME methodology. Afterwards, the database obtained from the off-line simulations is employed for structuring and training an intelligent SIME's TS margin classifier based on support vector machines (SVM) to be used for real-time transient stability assessment (TSA). Besides, the SVM is optimally tuned by using the swarm variant of the mean-variance mapping optimization MVMOS. Several tests are then performed on the New England benchmark power system. Results demonstrate the feasibility and effectiveness that could be achieved in estimating the transient stability status related to the SIME's TS margins, which will also be of great value for defining suitable emergency control actions in real time.

Support Vector Machine (SVM) is a popular machine learning methods in the field of data analysis. The Random Projections (RP) method can solve the problem of dimensionality reduction of high-dimensional data quickly and effectively to reduce the computational cost of the related optimization problem and is widely used in the SVM method. However, the large-scale SVM has the problem of reduced classification accuracy after dimension reduction by random projection feature. Therefore, we propose a linear kernel support vector machine based on dual-random projection (drp-LSVM) for large-scale classification problem, which combines the duality recovery theory. In this paper, we analyze the geometric properties of drp-LSVM, and prove that while dividing the geometric advantage of support vector machine based on random projection (rp-LSVM), the division of hyperplane is closer to that obtained by all data training Original classifier. The experimental results show that the drp-LSVM can reduce the classification error and improve the training precision, and the performance evaluation is closer to the classifier trained with the original data while inheriting the advantages of rp-LSVM.

Support Vector Machine (SVM) is one of the most popular machine learning algorithms. An energy-efficient SVM classifier is proposed in this paper, where approximate computing is utilized to reduce energy consumption and silicon area. A hardware architecture with reconfigurable kernels and overflow-resilient limiter is presented. For different applications, different kernels can be chosen and configured to achieve the optimum energy efficiency while achieving the performance requirement. For an epileptic seizure detection application, on average, 15% energy and 14% area savings are achieved with the proposed approximate SVM classifier compared to a fully-accurate SVM implementation with almost no accuracy degradation.

Nowadays, surface defect detection systems for steel strip have replaced traditional artificial inspection systems, and automatic defect detection systems offer good performance when the sample set is large and the model is stable. However, the trained model does work well when a new production line is initiated with different equipment, processes, or detection devices. These variables make just tiny changes to the real-world model but have a significant impact on the classification result. To overcome these problems, we propose an evolutionary classifier with a Bayes kernel (BYEC) that can be adjusted with a small sample set to better adapt the model for a new production line. First, abundant features were introduced to cover detailed information about the defects. Second, we constructed a series of support vector machines (SVMs) with a random subspace of the features. Then, a Bayes classifier was trained as an evolutionary kernel fused with the results from the sub-SVM to form an integrated classifier. Finally, we proposed a method to adjust the Bayes evolutionary kernel with a small sample set. We compared the performance of this method to various algorithms; experimental results demonstrate that the proposed method can be adjusted with a small sample set to fit the changed model. Experimental evaluations were conducted to demonstrate the robustness, low requirement for samples, and adaptiveness of the proposed method.

Classifier systems with an efficient learning process are capable of producing a compact set of accurate rules, and therefore have the potential to play the role as a knowledge collector for other systems. In this paper, the accuracy-based Learning Classifier System (XCS) is guided to build the membership function and the rule set commonly owned by a fuzzy logic controller (FLC). Initialized with empty knowledge, an agent responds to real-valued inputs and learns to compose a knowledge using XCS. Results obtained from extensive experiments show that after a number of learning cycles, classifier systems can actually compose a set of rules similar to the mapping mechanism designed by humans for FLC's operation.

We propose to jointly learn a Discriminative Bayesian dictionary along a linear classifier using coupled Beta-Bernoulli Processes. Our representation model uses separate base measures for the dictionary and the classifier, but associates them to the class-specific training data using the same Bernoulli distributions. The Bernoulli distributions control the frequency with which the factors (e.g. dictionary atoms) are used in data representations, and they are inferred while accounting for the class labels in our approach. To further encourage discrimination in the dictionary, our model uses separate (sets of) Bernoulli distributions to represent data from different classes. Our approach adaptively learns the association between the dictionary atoms and the class labels while tailoring the classifier to this relation with a joint inference over the dictionary and the classifier. Once a test sample is represented over the dictionary, its representation is accurately labelled by the classifier due to the strong coupling between the dictionary and the classifier. We derive the Gibbs Sampling equations for our joint representation model and test our approach for face, object, scene and action recognition to establish its effectiveness.

The paper proposes a hybrid classifier model for Dynamic Stability Prediction in Power System (DSPPS). The Hybrid Classifier Model (HCM) is included multiple parallel classifiers. Data for dynamic stable state in power system are very large, complex, and non-linear. These lead to making difficult for a single classifier to learn boundaries of classes. While previous works mainly concentrate on a single classifier, this paper focuses on parallel classifiers. The paper proposes using energy of samples for data partition into clusters. Each single classifier only learns a margin of each cluster. Thus, the learning is easier and classification accuracy is higher. To build HCM, feature selection, and data partition are implemented. The HCM is built by Generalize Regression Neural Network. The study is tested on IEEE 39-bus power system network.

Yield estimation is becoming a challenging task for circuits that are replicated in millions of instances on a large design (High Replication Circuits, HRC) such as SRAMs and flip flops. This is because a rare event in a circuit cell may have a large impact on the system yield. To achieve high yield in HRC, the failure probability of the individual cell is requested to be very small. Thus the number of Monte Carlo simulations needed to detect a rare event is very large and no longer practical. The statistical blockade has been proposed to decrease the number of Monte Carlo simulations needed using classification of tail points and simulating these points only. The Support Vector Machine (SVM) was used in the classification of tail points. Kernel functions for SVM classifier, linear or radial, were chosen according to the data complexity. In this paper, Random Forest (RF) classifier is used as a general purpose classifier irrespective of the complexity of the data. It is shown that RF classifier provides the same accuracy or improves it without having to know the relationship between the input parameters.

In this paper the two-level intellectual classifier based on the SVM algorithm has been offered. This classifier works as the group of the SVM classifiers at the first level and as the final SVM classifier on the base of the modified PSO algorithm at the second level. Herewith, we use the original experimental dataset to develop the different SVM classifiers at the first level, and the new dataset, containing only supports vectors of the SVM classifiers of the first level, to develop the final SVM classifier on the base of the modified PSO algorithm at the second level. The results of experimental studies confirm the efficiency of the offered two-level intellectual classifier.

The paper deals with realizations of transfer learning for classification, i. e. the adaptation of a classifier model to a changed data distribution. This change could be a data drift or a more complex transformation. We propose to model those data changes by manifolds describing continuous transformations of the data. This description can be seen as a generalization of function based transfer models considered so far. The manifold description of the transfer function allows either to adjust the classifier model to the changed data distribution or a back-transformation of those data to the original data space. To get the approach feasible, the manifold is approximated by the affine part of the Taylor expansion of the manifold structure. Moreover, the affine approximation shows mathematical correspondences to tangent metric leaning, which was developed for handling of data with drifts in classification methods. The paper provides the mathematical background for manifold based transfer data learning. Further, the approach is exemplarily applied for the generalized learning vector quantization classifier. This classifier is a prominent method which frequently achieves a high performance and a robust behavior while the classifier complexity is low compared to more sophisticated approaches like deep learning architectures or support vector machines. Moreover, the good interpretability of learning vector quantization classifiers also contributes to an intuitive practical understanding of transfer learning.

This paper designs an algorithm for the moving object recognition based on support vector machine (SVM) in order to identify and classify the moving objects accurately. In view of the advantages of support vector machine in small sample, nonlinear, and high dimensional pattern recognition, a classifier is constructed based on support vector machine (SVM) is constructed. A feature vector is presented to train and classify support vector machines, which is composed of shape features and used to classify the samples. Furthermore, the support vector machine and binary decision tree are combined to form the multi class classifier. The object feature vector is used as the input of SVM, and the classifier is used to classify the detected moving objects. Finally, the experimental results show that the proposed algorithm can identify and classify different objects in video images accurately.

In this paper, a new type of 0-order multi-model classifier, called Autonomous Learning Multiple-Model (ALMMo-0), is proposed. The proposed classifier is non-iterative, feedforward and entirely data-driven. It automatically extracts the data clouds from the data per class and forms 0-order AnYa type fuzzy rule-based (FRB) sub-classifier for each class. The classification of new data is done using the “winner takes all” strategy according to the scores of confidence generated objectively based on the mutual distribution and ensemble properties of the data by the sub-classifiers. Numerical examples based on benchmark datasets demonstrate the high performance and computation-efficiency of the proposed classifier.

Pedestrian detection is an important topic in many applications, such as intelligent transportation systems (ITSs) or surveillance. For the purpose of applications used around the clock, the work for detecting pedestrian based on thermal sensors has attracted significant attention. To achieve this, this paper proposes a LBP (local binary pattern) encoded multi-level classifier for detecting pedestrians in thermal images. The proposed method consists of training and detection stages. The training stage firstly encodes the texture information of all training samples in all possible blocks by using LBP. For a particular block, the patches of all positive and negative samples with the same LBP code are collected to train a SVM (support vector machine) classifier for discrimination. HOGs (Histogram of Oriented Gradients) are used for the description of the patch appearance. By considering a set of learned SVMs for a particular block as a weak classifier, a set of discriminative weak classifiers is selected to form a final classifier (strong classifier) by using Adaboost. The experimental results on the constructed dataset demonstrate that the proposed method outperforms those in the literature.

AdaBoost is a classic ensemble learning algorithm with good classifier performance. In the past, it mainly used weak classifier as base classifier, such as KNN. They are simple and easy to train, but the essence of the weak classifier, it is impossible to get very high classification accuracy. In order to improve the correct rate, this paper introduces the AdaBoost ensemble classifier based on convolutional neural network, namely adaBoost-CNN, referred to as ACNN. ACNN design a new training method, it not only gives the weight of base classifier according to the error rate of base classifier in pre-training phase, but also dynamically adjusts this weight and learning coefficient of training sample according to the error rate of each class in ensemble training phase. Finally, through experiments on some public datasets, it was proved that ACNN not only can effectively reduce the classification error rate, but also can solve the problem of class recognition rate imbalanced caused by similar categories or training samples quantity deviation.

This article is devoted to improving previously developed texture classifier that performs on noisy images. The basic principle of this classifier is to join several simple local parameters using some fuzzy logic system (support vector machine or neural network). It is shown that aggregating procedure applied on the classifier's input can result in significant improvement of its efficiency.

This paper presents an exploratory machine learning attack based on deep learning to infer the functionality of an arbitrary classifier by polling it as a black box, and using returned labels to build a functionally equivalent machine. Typically, it is costly and time consuming to build a classifier, because this requires collecting training data (e.g., through crowdsourcing), selecting a suitable machine learning algorithm (through extensive tests and using domain-specific knowledge), and optimizing the underlying hyperparameters (applying a good understanding of the classifier's structure). In addition, all this information is typically proprietary and should be protected. With the proposed black-box attack approach, an adversary can use deep learning to reliably infer the necessary information by using labels previously obtained from the classifier under attack, and build a functionally equivalent machine learning classifier without knowing the type, structure or underlying parameters of the original classifier. Results for a text classification application demonstrate that deep learning can infer Naive Bayes and SVM classifiers with high accuracy and steal their functionalities. This new attack paradigm with deep learning introduces additional security challenges for online machine learning algorithms and raises the need for novel mitigation strategies to counteract the high fidelity inference capability of deep learning.

Compressed sensing is a simple and efficient technique that has a number of applications in signal processing and machine learning. In machine learning it provides answers to questions such as: "under what conditions is the sparse representation of data efficient?", "when is learning a large margin classifier directly on the compressed domain possible?", and "why does a large margin classifier learn more effectively if the data is sparse?". This work tackles the problem of feature representation from the context of sparsity and affine rank minimization by leveraging compressed sensing from the learning perspective in order to provide answers to the aforementioned questions. We show, for a full-rank signal, the high dimensional sparse representation of data is efficient because from the classifiers viewpoint such a representation is in fact a low dimensional problem. We provide practical bounds on the linear classifier to investigate the relationship between the SVM classifier in the high dimensional and compressed domains and show for the high dimensional sparse signals, when the bounds are tight, directly learning in the compressed domain is possible.

Silva, J., Bação, F., & Caetano, M. (2017). Specific land cover class mapping by semi-supervised weighted support vector machines. Remote Sensing, 9(2), [181]. https://doi.org/10.3390/rs9020181 In many remote sensing projects on land cover mapping, the interest is often in a sub-set of classes presented in the study area. Conventional multi-class classification may lead to a considerable training effort and to the underestimation of the classes of interest. On the other hand, one-class classifiers require much less training, but may overestimate the real extension of the class of interest. This paper illustrates the combined use of cost-sensitive and semi-supervised learning to overcome these difficulties. This method utilises a manually-collected set of pixels of the class of interest and a random sample of pixels, keeping the training effort low. Each data point is then weighted according to its distance to its near positive data point to inform the learning algorithm. The proposed approach was compared with a conventional multi-class classifier, a one-class classifier, and a semi-supervised classifier in the discrimination of high-mangrove in Saloum estuary, Senegal, from Landsat imagery. The derived classification accuracies were high: 93.90% for the multi-class supervised classifier, 90.75% for the semi-supervised classifier, 88.75% for the one-class classifier, and 93.75% for the proposed method. The results show that accuracy achieved with the proposed method is statistically non-inferior to that achieved with standard binary classification, requiring however much less training effort. publishersversion published

Modern machine tools with high speed machining capabilities could place rotating shafts, gears, and bearings under extreme thermal, static, and impact stresses, potentially increasing their failure rates. In this research, a gearbox damage detection strategy based on discrete wavelet transform (DWT), wavelet packet transform (WPT), support vector machine (SVM), and artificial neural networks (ANN) is presented. Three case studies are conducted to compare the classification performance of SVM kernel functions and ANN. First, a fault detection analysis based on DWT and WPT is carried out to extract the damage information from the gearbox’s raw vibration signal. In this step, wavelet coefficients obtained from DWT are characterized using statistical calculations. Energy characteristics of the gearbox signal are acquired using WPT and their statistical characteristics are also computed. These three sets of information extracted from wavelet transforms are utilized as the input to SVM and ANN classifiers. Secondly, the improved distance evaluation technique (IDE) is implemented to select the sensitive input features for SVM and ANN. The penalty parameter C and kernel parameter γ in SVM are also optimized using the grid-search method. Finally, the optimized features and parameters are input into SVM and ANN algorithms to detect gearbox damage. The result shows that gearbox damage detection using energy characteristics extracted from WPT (Case 2) or their statistical values as input features (Case 3) to the learning algorithms produces higher classification accuracies than using statistical values of the DWT coefficients as inputs (Case 1). In addition, RBF-SVM has the best classification performance in Case 2 and 3 while Linear-SVM has the best classification accuracy rate in Case 1 in damage detection average.

Stability is an important aspect of a classification procedure because unstable predictions can potentially reduce users' trust in a classification system and also harm the reproducibility of scientific conclusions. The major goal of our work is to introduce a novel concept of classification instability, i.e., decision boundary instability (DBI), and incorporate it with the generalization error (GE) as a standard for selecting the most accurate and stable classifier. Specifically, we implement a two-stage algorithm: (i) initially select a subset of classifiers whose estimated GEs are not significantly different from the minimal estimated GE among all the candidate classifiers; (ii) the optimal classifier is chosen as the one achieving the minimal DBI among the subset selected in stage (i). This general selection principle applies to both linear and nonlinear classifiers. Large-margin classifiers are used as a prototypical example to illustrate the above idea. Our selection method is shown to be consistent in the sense that the optimal classifier simultaneously achieves the minimal GE and the minimal DBI. Various simulations and real examples further demonstrate the advantage of our method over several alternative approaches., 38 pages. To appear in Statistica Sinica

How to build a classifier on datasets that are distributed across different sites under privacy constrains has attracted much attention during the past few years. In this paper, a new method for constructing classifier ensemble for privacy-preserving distributed data mining is proposed. Different from existing methods, the proposed method can obtain, without any auxiliary assumption and releasing the original data of stakeholders, the optimal weights for classifier combination. Experiments show that the ensemble based on our approach achieved high performance.

In this work, a novel classifier which has the ability of making binary classifications by supervised learning is introduced. The proposed classifier generates a finite state machine which is derived from the dataset used for training. The states of this machine show the likelihood that the visiting samples belong to one of the classes concerned. Learning process is realized by recording the states mostly visited by the samples of a class. And the classification is done by examining the states visited by the sample being tested are mostly visited by the samples of which class during the training. In the realized experiments, it has been seen that the proposed classifier makes as successful classifications as artificial neural networks and binary classification trees. The main advantages of the proposed classifier are that it requires fewer iterations than neural networks and does not require rules such as splitting criterion as in binary decision trees. Its another advantage is that learning can be performed while data is flowing in one direction, without the need for a back-propagation method such as used in neural networks.

In this paper, we propose a reduction method for the multiobjective multiclass support vector machine (MMSVM) which can maintain the discrimination ability and reduce the computational complexity of the original MMSVM. The proposed method finds some centroids of each class by a k-means clustering and obtains a classifier based on the centroids where the normal vectors of the corresponding separating hyperplanes are given by weighted sums of the centroids, while the geometric margins are exactly maximized between class pairs. Through some numerical experiments for benchmark problems, we observed that the proposed method can reduce the computational complexity without decreasing its generalization ability widely.

One of the popular tools in pattern recognition is a neuro-fuzzy system. Most of the neuro-fuzzy systems are based on a multi-layer perceptrons. In this paper, we incorporate learning vector quantization in a neuro-fuzzy system. The prototype update equation is based on the learning vector quantization while the gradient descent technique is used in the weight update equation. Since weights contain informative information, they are exploited to select a good feature set. There are 8 data sets used in the experiment, i.e., Iris Plants, Wisconsin Breast Cancer (WBC), Pima Indians Diabetes, Wine, Ionosphere, Colon Tumor, Diffuse Large B-Cell Lymphoma (DLBCL), and Glioma Tumor (GLI_85). The results show that our algorithm provides good classification rates on all data sets. It is able to select a good feature set with a small number of features. We compare our results indirectly with the existing algorithms as well. The comparison result shows that our algorithm performs better than those existing ones.

usc Refereed/Peer-reviewed A novel evolving semi-supervised classifier, namely Parsimonious Classifier+ (pClass+), is proposed in this paper. pClass+ enhances a recently developed classifier, namely pClass, for a semi-supervised learning scenario. As with its predecessor, pClass+ is capable of initiating its learning process from scratch with an empty rule base and adopts an open network structure, where fuzzy rules are evolved, pruned, and recalled automatically on demands. The novelty of pClass+ lies in an online active learning technique, which decreases operator’s annotation efforts and expedites its training process. pClass+ is also equipped with a new parameter identification strategy to cope with the class overlapping situation. The efficacy of pClass+ has been experimentally validated with numerous synthetic and real-world study cases, confirmed by thorough statistical tests and comparisons against state-of-the art classifiers, where pClass+ outperforms its counterparts in achieving the best trade-off between accuracy and complexity.

Aiming to solving the problem of slow training speed and learning efficiency existed in the deep auto-encoder network, this paper puts forward a new kind of modified deep auto-encoder network model based on extreme learning machine (ELM-MDAE). Through training the deep auto-encoder networks with the training method of extreme learning machine, the classification accuracy and training time of ELM-MDAE are compared with traditional deep auto-encoder network utilizing the rolling bearing fault vibration dataset released by Case Western Reserve University in United States. Experiments turn out to be that the average diagnostic accuracy rate could reach to 98.42%, and the average training time is 3.70 s with the method established in this paper. Therefore, ELM-MDAE possesses a better classification ability and fewer training time.