Search

In this paper, we present a framework for performing automatic analysis of Land Use Zones based on Location-Based Social Networks (LBSNs). We model city areas using a hierarchical structure of POIs extracted from foursquare. We encode such structures in kernel machines, e.g., Support Vector Machines, using a new Tree Kernel, i.e., the Hierarchical POI Kernel (HPK), which can take the importance of the individual POIs into account during the substructure matching. This way, HPK projects structures in the space of all their possible substructures such that each dimension corresponds to a semantic structural feature, weighted according to the discriminative power of POIs. We generated four different datasets for the following cities: Barcelona, Lisbon, Amsterdam and Milan, where we trained and tested our models. The results show that our approach largely outperforms previous work and standard baseline built on simple features, such as counts of different POIs. Finally, we apply a mining algorithm to extract the most relevant features (tree fragments) from the implicit TK space according to the weights the kernel machine assigned to them. Our approach can produce an explicit set of representative features that can be used to classify and characterize urban areas. [ABSTRACT FROM AUTHOR]

Clustering is a widely used machine learning technique for unlabelled data. One of the recently proposed techniques is the twin support vector clustering (TWSVC) algorithm. The idea of TWSVC is to generate hyperplanes for each cluster. TWSVC utilizes the hinge loss function to penalize the misclassification. However, the hinge loss relies on shortest distance between different clusters, and is unstable for noise-corrupted datasets, and for re-sampling. In this paper, we propose a novel Sparse Pinball loss Twin Support Vector Clustering (SPTSVC). The proposed SPTSVC involves the $\epsilon$ -insensitive pinball loss function to formulate a sparse solution. Pinball loss function provides noise-insensitivity and re-sampling stability. The $\epsilon$ -insensitive zone provides sparsity to the model and improves testing time. Numerical experiments on synthetic as well as real world benchmark datasets are performed to show the efficacy of the proposed model. An analysis on the sparsity of various clustering algorithms is presented in this work. In order to show the feasibility and applicability of the proposed SPTSVC on biomedical data, experiments have been performed on epilepsy and breast cancer datasets. [ABSTRACT FROM AUTHOR]

Hyperspectral imagery (HSI) has shown promising results in real-world applications. However, the technological evolution of optical sensors poses two main challenges in HSI classification: 1) the spectral band is usually redundant and noisy and 2) HSI with millions of pixels has become increasingly common in real-world applications. Motivated by the recent success of hybrid huberized support vector machines (HHSVMs), which inherit the benefits of both lasso and ridge regression, this paper first investigates the advantages of HHSVM for HSI applications. Unfortunately, the existing HHSVM solvers suffer from prohibitive computational costs on large-scale data sets. To solve this problem, this paper proposes simple and effective stochastic HHSVM algorithms for HSI classification. In the stochastic settings, we show that with a probability of at least $1-\varrho $ , our algorithms find an $\epsilon $ -accurate solution using $\tilde {O}({1}/{\lambda _{2}\epsilon })$ iterations. Since the convergence rate of our algorithms does not depend on the size of the training set, our algorithms are suitable for handling large-scale problems. We demonstrate the superiority of our algorithms by conducting experiments on large-scale binary and multiclass classification problems, comparing to the state-of-the-art HHSVM solvers. Finally, we apply our algorithms to real HSI classification and achieve promising results. [ABSTRACT FROM AUTHOR]

Multi-label classification is an important research topic in machine learning, for which exploiting label dependencies is an effective modeling principle. Recently, probabilistic models have shown great potential in discovering dependencies among labels. In this paper, motivated by the recent success of multi-view learning to improve the generalization performance, we propose a novel multi-view probabilistic model named latent conditional Bernoulli mixture (LCBM) for multi-label classification. LCBM is a generative model taking features from different views as inputs, and conditional on the latent subspace shared by the views a Bernoulli mixture model is adopted to build label dependencies. Inside each component of the mixture, the labels have a weak correlation which facilitates computational convenience. The mean field variational inference framework is used to carry out approximate posterior inference in the probabilistic model, where we propose a Gaussian mixture variational autoencoder (GMVAE) for effective posterior approximation. We further develop a scalable stochastic training algorithm for efficiently optimizing the model parameters and variational parameters, and derive an efficient prediction procedure based on greedy search. Experimental results on multiple benchmark datasets show that our approach outperforms other state-of-the-art methods under various metrics. [ABSTRACT FROM AUTHOR]

As the training dataset size and the model size of machine learning increase rapidly, more computing resources are consumed to speedup the training process. However, the scalability and performance reproducibility of parallel machine learning training, which mainly uses stochastic optimization algorithms, are limited. In this paper, we demonstrate that the sample difference in the dataset plays a prominent role in the scalability of parallel machine learning algorithms. We propose to use statistical properties of dataset to measure sample differences. These properties include the variance of sample features, sample sparsity, sample diversity, and similarity in sampling sequences. We choose four types of parallel training algorithms as our research objects: (1) the asynchronous parallel SGD algorithm (Hogwild! algorithm), (2) the parallel model average SGD algorithm (minibatch SGD algorithm), (3) the decentralization optimization algorithm, and (4) the dual coordinate optimization (DADM algorithm). Our results show that the statistical properties of training datasets determine the scalability upper bound of these parallel training algorithms. [ABSTRACT FROM AUTHOR]

Multi-view learning (MVL) is an active direction in machine learning that aims at exploiting the consensus and complementarity information among multiple distinct feature sets to boost the generalization performance of the counterpart algorithm. So far, two classical SVM-based MVL methods are SVM-2K and multi-view twin support vector machine (MvTSVM). They are designed only for two-view classification and cannot tackle the general multi-view classification problem. They also cannot effectively leverage the complementarity information among different feature views. In this paper, we propose two novel multi-view support vector machines with the consensus and complementarity information for MVL that not only can deal with the two-view classification problem but also the general multi-view classification problem by jointly learning multiple different views in a non-pairwise way. The disagreement among different views is regarded as a constraint or a regularization term in the objective function which plays an important role in exploring the consensus information. Combination weights for the reconstruction of each view in regularization terms are learned to explore complementarity information among different views. Finally, an efficient iteration algorithm with the classical convex quadratic programming is developed for optimization. Experimental results validate the effectiveness of our proposed methods. [ABSTRACT FROM AUTHOR]

This paper proposes a methodology for predicting online rotor angle stability in power system operation under significant contribution from wind generation. First, a novel algorithm is developed to extract a stability index (SI) that quantifies the margin of rotor angle stability of power systems reflecting the dynamics of wind power. An approach is proposed that takes advantage of the machine learning technique and the newly defined SI. In case of a contingency, the developed algorithm is employed in parallel to find SIs for all possible instability modes. The SIs are formed as a vector and then applied to a classifier algorithm for rotor angle stability prediction. Compared to other features used in state-of-the-art methods, SI vectors are highly recognizable and thus can lead to a more accurate and reliable prediction. The proposed approach is validated on two IEEE test systems with various wind power penetration levels and compared to existing methods, followed by a discussion of results. [ABSTRACT FROM AUTHOR]

Despite the several speedup methods proposed in the literature, the computational complexity of High Efficiency Video Coding (HEVC) video encoding is still a problem. This paper proposes a fast coding unit (CU) partition decision for use in HEVC encoders based on support vector machine (SVM)-trained offline. The SVM classifiers, features, and training procedures are described in detail, and a justification for the use of SVMs is provided. The trained classifiers are incorporated into a modified reference encoder in the form of a fast CU partition decision algorithm, which decides if the exhaustive search for the best partition is continued or terminated prematurely. Using the proposed method, an average complexity reduction of 48% is achieved with a 0.48% Bjontegaard-Delta bitrate (BD-BR) loss using the random access coding configuration, 44% reduction with a 0.62% BD-BR loss for the Low Delay B, and a 41% reduction with a 0.6% BD-BR loss for the Low Delay P configuration. We also tested our approach under constant bitrate conditions, achieving a 47% reduction in encoding time with a 1.11% loss in the BD-BR. In addition, a decision threshold adaptation is also proposed to allow adjusting the rate-distortion/complexity trade-off of our solution. With this approach, the computational complexity reduction can be varied from 34.9% (with a 0.13% loss in the BD-BR) up to 52.4% (with a 1.11% BD-BR loss) using the random access configuration. Compared with the state-of-the-art solutions, our decision scheme outperforms the related works in terms of combined rate distortion and complexity. [ABSTRACT FROM AUTHOR]

Target tracking is an important field of investigation in wireless sensor networks. When multiple targets are closely spaced, their measurement points are mixed together due to the insufficient accuracy of the sensor. This may bring some difficulties in determining the positions of observation points required by subsequent algorithms and applications. Most of the known tracking algorithms are derived from the Kalman filter, extended Kalman filter, and particles filter. In this paper, a novel measurement data classification algorithm based on support vector machine (SVM) is provided. SVM and Kalman filter are combined to obtain the updated classification line at each sampling period, and the sampling points would be classified by the updated classification line to calculate the coordinates of the corresponding observation points, which are then used to estimate the precise positions of two targets. A series of simulations and experiments are carried out to validate the presented algorithm on classifying and tracking two targets moving closely. Simulation results for a maneuvering targets classification example illustrate the feasibility of the new algorithm, as well as experimental and quantitative results from the practical data validate the effectiveness and stability of our proposal in contrast with existing methods. [ABSTRACT FROM AUTHOR]

This paper investigates the application of support vector machine to the modeling of high-speed interconnects with largely varying and/or highly uncertain design parameters. The proposed method relies on a robust and well-established mathematical framework, yielding accurate surrogates of complex dynamical systems. An identification procedure based on the observation of a small set of system responses allows generating compact parametric relations, which can be used for design optimization and/or stochastic analysis. The feasibility and strength of the method are demonstrated based on a benchmark function and on the statistical assessment of a realistic printed circuit board interconnect, highlighting the main features and benefits of this technique over state-of-the-art solutions. Emphasis is given to the effects of the initial sample size and of input noise on the model estimation. [ABSTRACT FROM AUTHOR]

Semisupervised scheme has emerged as a popular strategy in the machine learning community due to the expensiveness of getting enough labeled data. In this paper, a semisupervised incremental support vector machine (SE-INC-SVM) algorithm based on neighborhood kernel estimation is proposed. First, kernel regression is constructed to estimate the unlabeled data from the labeled neighbors and its estimation accuracy is discussed from the analogy with tradition RBF neural network. The incremental scheme is derived to improve the learning efficiency and reduce the computing time. Simulations for manual data set and industrial benchmark-penicillin fermentation process demonstrate the effectiveness of the proposed SE-INC-SVM method. [ABSTRACT FROM AUTHOR]

The function of a protein is determined by its structure, which creates a need for efficient methods of protein structure determination to advance scientific and medical research. Because current experimental structure determination methods carry a high price tag, computational predictions are highly desirable. Given a protein sequence, computational methods produce numerous 3D structures known as decoys. Selection of the best quality decoys is both challenging and essential as the end users can handle only a few ones. Therefore, scoring functions are central to decoy selection. They combine measurable features into a single number indicator of decoy quality. Unfortunately, current scoring functions do not consistently select the best decoys. Machine learning techniques offer great potential to improve decoy scoring. This paper presents two machine-learning based scoring functions to predict the quality of proteins structures, i.e., the similarity between the predicted structure and the experimental one without knowing the latter. We use different metrics to compare these scoring functions against three state-of-the-art scores. This is a first attempt at comparing different scoring functions using the same non-redundant dataset for training and testing and the same features. The results show that adding informative features may be more significant than the method used. [ABSTRACT FROM AUTHOR]

In this paper, we analyze the spectrum occupancy in cognitive radio networks (CRNs) using different machine learning techniques. Both supervised techniques [naive Bayesian classifier (NBC), decision trees (DT), support vector machine (SVM), linear regression (LR)] and unsupervised algorithms [hidden Markov model (HMM)] are studied to find the best technique with the highest classification accuracy (CA). A detailed comparison of the supervised and unsupervised algorithms in terms of the computational time and the CA is performed. The classified occupancy status is further utilized to evaluate the blocking probability of secondary user for future time slots, which can be used by system designers to define spectrum-allocation and spectrum-sharing policies. Numerical results show that SVM is the best algorithm among all the supervised and unsupervised classifiers. Based on this, we proposed a new SVM algorithm by combining it with a firefly algorithm (FFA), which is shown to outperform all the other algorithms. [ABSTRACT FROM PUBLISHER]

Machine learning (ML) algorithms play a vital role in the brain age estimation frameworks. The impact of regression algorithms on prediction accuracy in the brain age estimation frameworks have not been comprehensively evaluated. Here, we sought to assess the efficiency of different regression algorithms on brain age estimation. To this end, we built a brain age estimation framework based on a large set of cognitively healthy (CH) individuals ($N = 788$) as a training set followed by different regression algorithms (22 different algorithms in total). We then quantified each regression-algorithm on independent test sets composed of 88 CH individuals, 70 mild cognitive impairment patients as well as 30 Alzheimer’s disease patients. The prediction accuracy in the independent test set (i.e., CH set) varied in regression algorithms mean absolute error (MAE) from 4.63 to 7.14 yrs, $R^2$ from 0.76 to 0.88. The highest and lowest prediction accuracies were achieved by Quadratic Support Vector Regression algorithm (MAE $= 4.63$ yrs, $R^2 = 0.88, 95\%$ CI $= [-1.26, 1.42]$) and Binary Decision Tree algorithm (MAE $= 7.14$ yrs, $R^2 = 0.76, 95\%$ CI $= [-1.50, 2.62]$), respectively. Our experimental results demonstrate that the prediction accuracy in brain age frameworks is affected by regression algorithms, indicating that advanced machine learning algorithms can lead to more accurate brain age predictions in clinical settings. [ABSTRACT FROM AUTHOR]

This paper presents a kernel-based feature selection method for the classification of hyperspectral images. The proposed method aims at selecting a subset of the original features that are both 1) relevant (discriminant) for the considered classification problem, i.e., preserve the functional relationship between input and output variables, and 2) invariant (stable) across different domains, i.e., minimize the data-set shift between the source and the target domains. Domains can be associated with hyperspectral images collected either on different geographical areas or on the same area at different times. We propose a novel measure of data-set shift for evaluating the domain stability, which computes the distance of the conditional distributions between the source and target domains in a reproducing kernel Hilbert space. Such a measure is defined on the basis of the kernel embeddings of the conditional distributions resulting in a nonparametric approach that does not require estimating the distribution of the classes. The adopted search strategy is based on a multiobjective optimization algorithm, which optimizes the two terms of the criterion function for the estimation of the Pareto-optimal solutions. This results in an effective approach of performing feature selection in a transfer learning setting. The experimental results obtained on two hyperspectral images show the effectiveness of the proposed method in selecting features with high generalization capabilities. [ABSTRACT FROM AUTHOR]

Rare class problems are common in real-world applications across a wide range of domains. Standard classification algorithms are known to perform poorly in these cases, since they focus on overall classification accuracy. In addition, we have seen a significant increase of data in recent years, resulting in many large scale rare class problems. In this paper, we focus on nonlinear kernel based classification methods expressed as a regularized loss minimization problem. We address the challenges associated with both rare class problems and large scale learning, by 1) optimizing area under curve of the receiver of operator characteristic in the training process, instead of classification accuracy and 2) using a rare class kernel representation to achieve an efficient time and space algorithm. We call the algorithm RankRC. We provide justifications for the rare class representation and experimentally illustrate the effectiveness of RankRC in test performance, computational complexity, and model robustness. [ABSTRACT FROM PUBLISHER]

Facial age classification is an approach to classify face images into one of several predefined age groups. One of the difficulties in applying learning techniques to the age classification problem is the large amount of labeled training data required. Acquiring such training data is very costly in terms of age progress, privacy, human time, and effort. Although unlabeled face images can be obtained easily, it would be expensive to manually label them on a large scale and getting the ground truth. The frugal selection of the unlabeled data for labeling to quickly reach high classification performance with minimal labeling efforts is a challenging problem. In this paper, we present an active learning approach based on an online incremental bilateral two-dimension linear discriminant analysis (IB2DLDA) which initially learns from a small pool of labeled data and then iteratively selects the most informative samples from the unlabeled set to increasingly improve the classifier. Specifically, we propose a novel data selection criterion called the furthest nearest-neighbor (FNN) that generalizes the margin-based uncertainty to the multiclass case and which is easy to compute, so that the proposed active learning algorithm can handle a large number of classes and large data sizes efficiently. Empirical experiments on FG-NET and Morph databases together with a large unlabeled data set for age categorization problems show that the proposed approach can achieve results comparable or even outperform a conventionally trained active classifier that requires much more labeling effort. Our IB2DLDA-FNN algorithm can achieve similar results much faster than random selection and with fewer samples for age categorization. It also can achieve comparable results with active SVM but is much faster than active SVM in terms of training because kernel methods are not needed. The results on the face recognition database and palmprint/palm vein database showed that our approach can handle problems with large number of classes. Our contributions in this paper are twofold. First, we proposed the IB2DLDA-FNN, the FNN being our novel idea, as a generic on-line or active learning paradigm. Second, we showed that it can be another viable tool for active learning of facial age range classification. [ABSTRACT FROM AUTHOR]

A currency crisis is an economic event where a country's fixed exchange rate is under pressure by speculators. In some cases, currency crises are followed by strong recessions (e.g., recent Asian and Argentinean crises), but in other cases they are not. This paper seeks to determine what are the most significant factors in explaining the consequences of currency crises on the economy. This paper collects data on 25 variables for 64 currency crises between 1970 and 1999. This research uses a novel algorithm with support vector machines (SVM) for selecting significant variables. This algorithm works well with datasets characterized by nonlinearity and low variable-observation ratio. Variables of banking size and fragility, international trade, and devaluation were the most significant. Variables of banking supervision, economic development, and IMF intervention were found less significant. The variable selection results of the algorithm were compared with all-best subsets variable selection. The results of our algorithm are more consistent with the economic literature than the results from all-best subsets. [ABSTRACT FROM AUTHOR]

Standard Electromechanical Wave Imaging isochrone generation relies on manual selection of zero-crossing (ZC) locations on incremental strain curves for a number of pixels in the segmented myocardium for each echocardiographic view and patient. When considering large populations, this becomes a time-consuming process, that can be limited by inter-observer variability and operator bias. In this study, we developed and optimized an automated ZC selection algorithm, towards a faster more robust isochrone generation approach. The algorithm either relies on heuristic-based baselines or machine learning classifiers. Manually generated isochrones, previously validated against 3D intracardiac mapping, were considered as ground truth during training and performance evaluation steps. The machine learning models applied herein for the first time were: i) logistic regression; ii) support vector machine (SVM); and iii) Random Forest. The SVM and Random Forest classifiers successfully identified accessory pathways in Wolff-Parkinson-White patients, characterized sinus rhythm in humans, and localized the pacing electrode location in left ventricular paced canines on the resulting isochrones. Nevertheless, the best performing classifier was proven to be Random Forest with a precision rising from 89.5% to 97%, obtained with the voting approach that sets a probability threshold upon ZC candidate selection. Furthermore, the predictivity was not dependent on the type of testing dataset it was applied to, contrary to SVM that exhibited a 5% drop in precision on the canine testing dataset. Finally, these findings indicate that a machine learning approach can reduce user variability and considerably decrease the durations required for isochrone generation, while preserving accurate activation patterns. [ABSTRACT FROM AUTHOR]

This paper proposes a method for interactive surface recognition and surface categorization by a humanoid robot using a vibrotactile sensory modality. The robot was equipped with an artificial fingernail that had a built-in three-axis accelerometer. The robot interacted with 20 different surfaces by performing five different exploratory scratching behaviors on them. Surface-recognition models were learned by coupling frequency-domain analysis of the vibrations detected by the accelerometer with machine learning algorithms, such as support vector machine (SVM) and k-nearest neighbors (k -NN). The results show that by applying several different scratching behaviors on a test surface, the robot can recognize surfaces better than with any single behavior alone. The robot was also able to estimate a measure of similarity between any two surfaces, which was used to construct a grounded hierarchical surface categorization. [ABSTRACT FROM AUTHOR]

We present a closed-loop unsupervised clustering method for motion vectors extracted from highly dynamic video scenes. Motion vectors are assigned to nonconvex homogeneous clusters characterizing direction, size and shape of regions with multiple independent activities. The proposed method is based on support vector clustering. Cluster labels are propagated over time via incremental learning. The proposed method uses a kernel function that maps the input motion vectors into a high-dimensional space to produce nonconvex clusters. We improve the mapping effectiveness by quantifying feature similarities via a blend of position and orientation affinities. We use the Quasiconformal Kernel Transformation to boost the discrimination of outliers. The temporal propagation of the clusters’ identities is achieved via incremental learning based on the concept of feature obsolescence to deal with appearing and disappearing features. Moreover, we design an online clustering performance prediction algorithm used as a feedback that refines the cluster model at each frame in an unsupervised manner. We evaluate the proposed method on synthetic data sets and real-world crowded videos and show that our solution outperforms state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Different biomedical computing methods for cancer-specific gene recognition have been developed in recent years. Currently, building an open-box machine learning system to discover explainable knowledge from gene expression data is a difficult research problem due to a large number of genes, a small number of samples, and noise. Fuzzy systems can be used to deal with data ambiguity and noise issues and extract meaningful knowledge from gene data. In this article, we create a new deep fuzzy neural network to handle the uncertainty in gene data to generate useful knowledge for specific disease diagnosis. A new hybrid algorithm is designed to preprocess data and select informative genes for accurate cancer detection. Various experiments using six different cancer datasets indicate that the new method has better and more reliable performance than the other conventional classification methods with different gene selection methods. [ABSTRACT FROM AUTHOR]

The $k$ -vectors algorithm for learning regression functions proposed here is akin to the well-known $k$ -means algorithm. Both algorithms partition the feature space, but unlike the $k$ -means algorithm, the $k$ -vectors algorithm aims to reconstruct the response rather than the feature. The partitioning rule of the algorithm is based on maximizing the correlation (inner product) of the feature vector with a set of $k$ vectors, and generates polyhedral cells, similar to the ones generated by the nearest-neighbor rule of the $k$ -means algorithm. Similarly to $k$ -means, the learning algorithm alternates between two types of steps. In the first type of steps, $k$ labels are determined via a centroid-type rule (in the response space), which uses a surrogate hinge-type loss function to the mean squared error loss function. In the second type of steps, the $k$ vectors which determine the partition are updated according to a multiclass classification rule, in the spirit of support vector machines. It is proved that both steps of the algorithm only require solving convex optimization problems, and that the algorithm is empirically consistent - as the length of the training sequence increases to infinity, fixed-points of the empirical version of the algorithm tend to fixed points of the population version of the algorithm. Learnability of the predictor class posit by the algorithm is also established. [ABSTRACT FROM AUTHOR]

Machine learning techniques for computer vision applications like object recognition, scene classification, etc., require a large number of training samples for satisfactory performance. Especially when classification is to be performed over many categories, providing enough training samples for each category is infeasible. This paper describes new ideas in multiclass active learning to deal with the training bottleneck, making it easier to train large multiclass image classification systems. First, we propose a new interaction modality for training which requires only yes-no type binary feedback instead of a precise category label. The modality is especially powerful in the presence of hundreds of categories. For the proposed modality, we develop a Value-of-Information (VOI) algorithm that chooses informative queries while also considering user annotation cost. Second, we propose an active selection measure that works with many categories and is extremely fast to compute. This measure is employed to perform a fast seed search before computing VOI, resulting in an algorithm that scales linearly with dataset size. Third, we use locality sensitive hashing to provide a very fast approximation to active learning, which gives sublinear time scaling, allowing application to very large datasets. The approximation provides up to two orders of magnitude speedups with little loss in accuracy. Thorough empirical evaluation of classification accuracy, noise sensitivity, imbalanced data, and computational performance on a diverse set of image datasets demonstrates the strengths of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the “generalized” single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM. [ABSTRACT FROM AUTHOR]

Definition acquisition is a necessary step in building an artificial cognitive assistant that helps military personnel to gain fast and precise understanding of the various terms and procedures defined in applicable legal documents. We approach the task of identifying definitional sentences from operations law documents by formalizing this task as a sentence-classification task and solving it by using machine-learning methods. This paper reports on a series of empirical experiments in that we evaluate and compare the performance of learning algorithms in terms of label-prediction accuracy. Using supervised techniques results in an F1 score of 95.93% and a 96.72% recall rate. However, for real-world applications, it would be too costly and unrealistic to ask personnel involved in military operations to label substantial amounts of data in order to build a new classifier for different types or genres of text data. Therefore, we propose and implement a semisupervised (SS) solution that trades off prediction accuracy to label efficiency. Our SS approach achieves a 90.47% F1 score and 93.44% recall rate by using only eight sentences labeled by a human expert. [ABSTRACT FROM PUBLISHER]

The support vector machine (SVM) has provided higher performance than traditional learning machines and has been widely applied in real-world classification problems and nonlinear function estimation problems. Unfortunately, the training process of the SVM is sensitive to the outliers or noises in the training set. In this paper, a common misunderstanding of Gaussian-function-based kernel fuzzy clustering is corrected, and a kernel fuzzy c-means clustering-based fuzzy SVM algorithm (KFCM-FSVM) is developed to deal with the classification problems with outliers or noises. In the KFCM-FSVM algorithm, we first use the FCM clustering to cluster each of two classes from the training set in the high-dimensional feature space. The farthest pair of clusters, where one cluster comes from the positive class and the other from the negative class, is then searched and forms one new training set with membership degrees. Finally, we adopt FSVM to induce the final classification results on this new training set. The computational complexity of the KFCM-FSVM algorithm is analyzed. A set of experiments is conducted on six benchmarking datasets and four artificial datasets for testing the generalization performance of the KFCM-FSVM algorithm. The results indicate that the KFCM-FSVM algorithm is robust for classification problems with outliers or noises. [ABSTRACT FROM AUTHOR]

In this paper, we consider a new periodic activation function for the multivalued neuron (MVN). The MVN is a neuron with complex-valued weights and inputs/output, which are located on the unit circle. Although the MVN outperforms many other neurons and MVN-based neural networks have shown their high potential, the MVN still has a limited capability of learning highly nonlinear functions. A periodic activation function, which is introduced in this paper, makes it possible to learn nonlinearly separable problems and non-threshold multiple-valued functions using a single multivalued neuron. We call this neuron a multivalued neuron with a periodic activation function (MVN-P). The MVN-Ps functionality is much higher than that of the regular MVN. The MVN-P is more efficient in solving various classification problems. A learning algorithm based on the error-correction rule for the MVN-P is also presented. It is shown that a single MVN-P can easily learn and solve those benchmark classification problems that were considered unsolvable using a single neuron. It is also shown that a universal binary neuron, which can learn nonlinearly separable Boolean functions, and a regular MVN are particular cases of the MVN-P. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs) is a popular machine learning technique, which works effectively with balanced datasets. However, when it comes to imbalanced datasets, SVMs produce suboptimal classification models. On the other hand, the SVM algorithm is sensitive to outliers and noise present in the datasets. Therefore, although the existing class imbalance learning (CIL) methods can make SVMs less sensitive to class imbalance, they can still suffer from the problem of outliers and noise. Fuzzy SVMs (FSVMs) is a variant of the SVM algorithm, which has been proposed to handle the problem of outliers and noise. In FSVMs, training examples are assigned different fuzzy-membership values based on their importance, and these membership values are incorporated into the SVM learning algorithm to make it less sensitive to outliers and noise. However, like the normal SVM algorithm, FSVMs can also suffer from the problem of class imbalance. In this paper, we present a method to improve FSVMs for CIL (called FSVM-CIL), which can be used to handle the class imbalance problem in the presence of outliers and noise. We thoroughly evaluated the proposed FSVM-CIL method on ten real-world imbalanced datasets and compared its performance with five existing CIL methods, which are available for normal SVM training. Based on the overall results, we can conclude that the proposed FSVM-CIL method is a very effective method for CIL, especially in the presence of outliers and noise in datasets. [ABSTRACT FROM AUTHOR]

Machine learning techniques have been used extensively to build models for the analysis and retrieval of multimedia data. The explosion of multimedia data on the Web poses a great challenge to such techniques not simply because of the sheer data volume, but also because of the heterogeneity of the data. With data from a wide variety of domains, models trained from one domain do not generalize well to other domains, while at the same time it is prohibitively expensive to build new models for each and every domain due to the high cost for labeling training examples. In this paper, we tackle the heterogeneity challenge in large-scale multimedia data using cross-domain model adaptation for better performance and reduced human cost. Specifically, we investigate the problem of adapting supervised classifiers trained from one or more source domains to a new classifier for a target domain that has only limited labeled examples. The foundation of our work is a general framework for function-level classifier adaptation based on the regularized loss minimization principle, which adapts a classifier by directly modifying its decision function. Under this framework, one can derive concrete adaptation algorithms by plugging in any loss and regularization functions, among which we elaborate on adaptive support vector machines (a-SVM). We further extend this framework for multiclassifier adaptation, namely adapting multiple existing classifiers into a classifier for the target domain, in a way that the contributions of these existing classifiers are automatically determined. We evaluate the proposed approaches in cross-domain semantic concept detection based on TRECVID corpora. The results show that our approaches outperform existing (adaptation and nonadaptation) methods in terms of accuracy and/or efficiency, and adaptation from multiple classifiers offers further benefits. [ABSTRACT FROM PUBLISHER]

Support vector machine (SVM), as one of the most popular classifiers, aims to find a hyperplane that can separate two classes of data with maximal margin. SVM classifiers are focused on achieving more separation between classes than exploiting the structures in the training data within classes. However, the structural information, as an implicit prior knowledge, has recently been found to be vital for designing a good classifier in different real-world problems. Accordingly, using as much prior structural information in data as possible to help improve the generalization ability of a classifier has yielded a class of effective structural large margin classifiers, such as the structured large margin machine (SLMM) and the Laplacian support vector machine (LapSVM). In this paper, we unify these classifiers into a common framework from the concept of “structural granularity” and the formulation for optimization problems. We exploit the quadratic programming (QP) and second-order cone programming (SOCP) methods, and derive a novel large margin classifier, we call the new classifier the structural regularized support vector machine (SRSVM). Unlike both SLMM at the cross of the cluster granularity and SOCP and LapSVM at the cross of the point granularity and QP, SRSVM is located at the cross of the cluster granularity and QP and thus follows the same optimization formulation as LapSVM to overcome large computational complexity and non-sparse solution in SLMM. In addition, it integrates the compactness within classes with the separability between classes simultaneously. Furthermore, it is possible to derive generalization bounds for these algorithms by using eigenvalue analysis of the kernel matrices. Experimental results demonstrate that SRSVM is often superior in classification and generalization performances to the state-of-the-art algorithms in the framework, both with the same and different structural granularities. [ABSTRACT FROM AUTHOR]

Chronic pain is a common long-term condition that affects a person's physical and emotional functioning. Currently, the integrated biopsychosocial approach is the mainstay treatment for people with chronic pain. Self-reporting (the use of questionnaires) is one of the most common methods to evaluate treatment outcome. The questionnaires can consist of more than 300 questions, which is tedious for people to complete at home. This paper presents a machine learning approach to analyze self-reporting data collected from the integrated biopsychosocial treatment, in order to identify an optimal set of features for supporting self-management. In addition, a classification model is proposed to differentiate the treatment stages. Four different feature selection methods were applied to rank the questions. In addition, four supervised learning classifiers were used to investigate the relationships between the numbers of questions and classification performance. There were no significant differences between the feature ranking methods for each classifier in overall classification accuracy or AUC ( p > 0.05); however, there were significant differences between the classifiers for each ranking method (p < 0.001). The results showed the multilayer perceptron classifier had the best classification performance on an optimized subset of questions, which consisted of ten questions. Its overall classification accuracy and AUC were 100% and 1, respectively. [ABSTRACT FROM AUTHOR]

Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs. [ABSTRACT FROM AUTHOR]

A machine learning technique is applied to the design and optimization of reflectarray antennas to considerably accelerate computing time without compromising accuracy. In particular, support vector machines (SVMs), automatic learning structures that are able to deal with regression problems, are employed to obtain surrogate models of the reflectarray element to substitute the full-wave analysis tool for the characterization of the unit cell in the design and optimization algorithms. The analysis, design, and optimization of a very large reflectarray antenna for Direct Broadcast Satellite applications are accelerated up to three orders of magnitude. This is here demonstrated with three examples: one showing the design of a reflectarray and two for the crosspolar optimization, one with one coverage for each linear polarization (Europe and the Middle East) and another with a Middle East coverage working in dual-linear polarization. The accuracy of the proposed approach is validated by means of a comparison of the final designs with full-wave simulations based on local periodicity obtaining good agreement. The crosspolar dicrimination and crosspolar isolation are greatly improved using the SVMs while considerably reducing computing time. [ABSTRACT FROM AUTHOR]

Kernel methods have been shown to be effective for many machine learning tasks such as classification and regression. In particular, support vector machines with the Gaussian kernel have proved to be powerful classification tools. The standard way to apply kernel methods is to use the kernel trick, where the inner product of the vectors in the feature space is computed via the kernel function. Using the kernel trick for SVMs, however, leads to training that is quadratic in the number of input vectors and classification that is linear with the number of support vectors. We introduce a new kernel, the CRO (Concomitant Rank Order) kernel that approximates the Gaussian kernel on the unit sphere. We also introduce a randomized feature map, called the CRO feature map that produces sparse, high-dimensional feature vectors whose inner product asymptotically equals the CRO kernel. Using the Discrete Cosine Transform for computing the CRO feature map ensures that the cost of computing feature vectors is low, allowing us to compute the feature map explicitly. Combining the CRO feature map with linear SVM we introduce the CROification algorithm which gives us the efficiency of a sparse high-dimensional linear SVM with the accuracy of the Gaussian kernel SVM. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) INTRA coding improves compression efficiency by adopting advanced coding technologies, such as multi-level quad-tree block partitioning and up to 35-mode INTRA prediction. However, it significantly increases the coding complexity, memory access, and power consumption, which goes against its widely applications, especially for ultra-high definition and/or mobile video applications. To tackle this problem, we propose effective data driven coding unit (CU) size decision approaches for HEVC INTRA coding, which consists of two stages of support vector machine-based fast INTRA CU size decision schemes at four CU decision layers. At the first stage classification, a three output classifier with offline learning is developed to early terminate the CU size decision or early skip checking the current CU depth. As for the samples that neither early skipped nor early terminated, the second stage of binary classification, which learns online from previous coded frames, is proposed to further refine the CU size decision. Representative features for the CU size decision are explored at different decision layers and stages of classifications. Finally, the optimal parameters derived from the training data are achieved to reasonably allocate complexity among different CU layers at given total rate-distortion degradation constraint. Extensive experiments show that the proposed overall algorithm can achieve 27.95%–80.53% and 52.48% on average complexity reduction for the CU size decision as compared with the original HM16.7 model. Meanwhile, the average Bjonteggard delta peak-signal-to-noise ratio degradation is only −0.08 dB, which is negligible. The overall performance of the proposed algorithm outperforms the state-of-the-art benchmark schemes. [ABSTRACT FROM AUTHOR]

The objective of this work is recognition and spatiotemporal localization of two-person interactions in video. Our approach is person-centric. As a first stage we track all upper bodies and heads in a video using a tracking-by-detection approach that combines detections with KLT tracking and clique partitioning, together with occlusion detection, to yield robust person tracks. We develop local descriptors of activity based on the head orientation (estimated using a set of pose-specific classifiers) and the local spatiotemporal region around them, together with global descriptors that encode the relative positions of people as a function of interaction type. Learning and inference on the model uses a structured output SVM which combines the local and global descriptors in a principled manner. Inference using the model yields information about which pairs of people are interacting, their interaction class, and their head orientation (which is also treated as a variable, enabling mistakes in the classifier to be corrected using global context). We show that inference can be carried out with polynomial complexity in the number of people, and describe an efficient algorithm for this. The method is evaluated on a new dataset comprising 300 video clips acquired from 23 different TV shows and on the benchmark UT--Interaction dataset. [ABSTRACT FROM PUBLISHER]

Recently, there has been much interest around multitask learning (MTL) problem with the constraints that tasks should share a common sparsity profile. Such a problem can be addressed through a regularization framework where the regularizer induces a joint-sparsity pattern between task decision functions. We follow this principled framework and focus on \ellp-\ellq (with 0\leq p\leq 1 and 1\leq q\leq 2) mixed norms as sparsity-inducing penalties. Our motivation for addressing such a larger class of penalty is to adapt the penalty to a problem at hand leading thus to better performances and better sparsity pattern. For solving the problem in the general multiple kernel case, we first derive a variational formulation of the \ell1-\ellq penalty which helps us in proposing an alternate optimization algorithm. Although very simple, the latter algorithm provably converges to the global minimum of the \ell1-\ellq penalized problem. For the linear case, we extend existing works considering accelerated proximal gradient to this penalty. Our contribution in this context is to provide an efficient scheme for computing the \ell1-\ellq proximal operator. Then, for the more general case, when 0

Multiview, broadband, acoustic classification of individual fish was investigated using a recently developed laboratory scattering system. Scattering data from nine different species of saltwater fish were collected. Using custom software, these data were processed and filtered to yield a data set of 36 individuals, and between 200 and 500 echoes per individual. These data were sampled uniformly randomly in fish orientation. Feature-, decision-, and collaborative-fusion algorithms were then developed and tested using support vector machines (SVMs) as the underlying classifiers. Decision fusion was implemented by cascading two levels of support vectors machines. Collaborative fusion was implemented by using SVM outputs to estimate confidence levels and performing weighted averaging of probabilities computed from each view with feedback from other views. Collaborative fusion performed as well or better than the others, and did so without requiring assumptions about view geometry. In addition to a comparison between classification algorithms and feature transformations, two data collection geometries were explored, including random observation geometries. In all cases, combining multiple, broadband views yielded significant reductions in classification error (>50%) over single-view methods, for uniformly random fish orientation. [ABSTRACT FROM PUBLISHER]

One-class and cost-sensitive support vector machines (SVMs) are state-of-the-art machine learning methods for estimating density level sets and solving weighted classification problems, respectively. However, the solutions of these SVMs do not necessarily produce set estimates that are nested as the parameters controlling the density level or cost-asymmetry are continuously varied. Such nesting not only reflects the true sets being estimated, but is also desirable for applications requiring the simultaneous estimation of multiple sets, including clustering, anomaly detection, and ranking. We propose new quadratic programs whose solutions give rise to nested versions of one-class and cost-sensitive SVMs. Furthermore, like conventional SVMs, the solution paths in our construction are piecewise linear in the control parameters, although here the number of breakpoints is directly controlled by the user. We also describe decomposition algorithms to solve the quadratic programs. These methods are compared to conventional (non-nested) SVMs on synthetic and benchmark data sets, and are shown to exhibit more stable rankings and decreased sensitivity to parameter settings. [ABSTRACT FROM AUTHOR]

We present a method for explaining predictions for individual instances. The presented approach is general and can be used with all classification models that output probabilities. It is based on the decomposition of a model's predictions on individual contributions of each attribute. Our method works for the so-called black box models such as support vector machines, neural networks, and nearest neighbor algorithms, as well as for ensemble methods such as boosting arid random forests. We demonstrate that the generated explanations closely follow the learned models and present a visualization technique that shows the utility of our approach and enables the comparison of different prediction methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a nonconvex online Support Vector Machine (SVM) algorithm (LASVM-NC) based on the Ramp Loss, which has the strong ability of suppressing the influence of outliers. Then, again in the online learning setting, we propose an outlier filtering mechanism (LASVM-I) based on approximating nonconvex behavior in convex optimization. These two algorithms are built upon another novel SVM algorithm (LASVM-G) that is capable of generating accurate intermediate models in its iterative steps by leveraging the duality gap. We present experimental results that demonstrate the merit of our frameworks in achieving significant robustness to outliers in noisy data classification where mislabeled training instances are in abundance. Experimental evaluation shows that the proposed approaches yield a more scalable online SVM algorithm with sparser models and less computational running time, both in the training and recognition phases, without sacrificing generalization performance. We also point out the relation between nonconvex optimization and min-margin active learning. [ABSTRACT FROM AUTHOR]