Search

Semi-supervised learning (SSL) is a typical learning paradigms training a model from both labeled and unlabeled data. The traditional SSL models usually assume unlabeled data are relevant to the labeled data, i.e., following the same distributions of the targeted labeled data. In this paper, we address a different, yet formidable scenario in semi-supervised classification, where the unlabeled data may contain irrelevant data to the labeled data. To tackle this problem, we develop a maximum margin model, named tri-class support vector machine (3C-SVM), to utilize the available training data, while seeking a hyperplane for separating the targeted data well. Our 3C-SVM exhibits several characteristics and advantages. First, it does not need any prior knowledge and explicit assumption on the data relatedness. On the contrary, it can relieve the effect of irrelevant unlabeled data based on the logistic principle and maximum entropy principle. That is, 3C-SVM approaches an ideal classifier. This classifier relies heavily on labeled data and is confident on the relevant data lying far away from the decision hyperplane, while maximally ignoring the irrelevant data, which are hardly distinguished. Second, theoretical analysis is provided to prove that in what condition, the irrelevant data can help to seek the hyperplane. Third, 3C-SVM is a generalized model that unifies several popular maximum margin models, including standard SVMs, Semi-supervised SVMs (S 3 VMs), and SVMs learned from the universum ( U -SVMs) as its special cases. More importantly, we deploy a concave–convex produce to solve the proposed 3C-SVM, transforming the original mixed integer programming, to a semi-definite programming relaxation, and finally to a sequence of quadratic programming subproblems, which yields the same worst case time complexity as that of S 3 VMs. Finally, we demonstrate the effectiveness and efficiency of our proposed 3C-SVM through systematical experimental comparisons. [ABSTRACT FROM AUTHOR]

Abstract: This paper provides a sparse learning algorithm for Support Vector Classification (SVC), called Sparse Support Vector Classification (SSVC), which leads to sparse solutions by automatically setting the irrelevant parameters exactly to zero. SSVC adopts the L 0-norm regularization term and is trained by an iteratively reweighted learning algorithm. We show that the proposed novel approach contains a hierarchical-Bayes interpretation. Moreover, this model can build up close connections with some other sparse models. More specifically, one variation of the proposed method is equivalent to the zero-norm classifier proposed in (); it is also an extended and more flexible framework in parallel with the Sparse Probit Classifier proposed by . Theoretical justifications and experimental evaluations on two synthetic datasets and seven benchmark datasets show that SSVC offers competitive performance to SVC but needs significantly fewer Support Vectors. [ABSTRACT FROM AUTHOR]

Abstract: Kernel methods have been widely used in pattern recognition. Many kernel classifiers such as Support Vector Machines (SVM) assume that data can be separated by a hyperplane in the kernel-induced feature space. These methods do not consider the data distribution and are difficult to output the probabilities or confidences for classification. This paper proposes a novel Kernel-based Maximum A Posteriori (KMAP) classification method, which makes a Gaussian distribution assumption instead of a linear separable assumption in the feature space. Robust methods are further proposed to estimate the probability densities, and the kernel trick is utilized to calculate our model. The model is theoretically and empirically important in the sense that: (1) it presents a more generalized classification model than other kernel-based algorithms, e.g., Kernel Fisher Discriminant Analysis (KFDA); (2) it can output probability or confidence for classification, therefore providing potential for reasoning under uncertainty; and (3) multi-way classification is as straightforward as binary classification in this model, because only probability calculation is involved and no one-against-one or one-against-others voting is needed. Moreover, we conduct an extensive experimental comparison with state-of-the-art classification methods, such as SVM and KFDA, on both eight UCI benchmark data sets and three face data sets. The results demonstrate that KMAP achieves very promising performance against other models. [Copyright &y& Elsevier]

Abstract: Time series prediction, especially financial time series prediction, is a challenging task in machine learning. In this issue, the data are usually non-stationary and volatile in nature. Because of its good generalization power, the support vector regression (SVR) has been widely applied in this application. The standard SVR employs a fixed -tube to tolerate noise and adopts the -norm ( or 2) to model the functional complexity of the whole data set. One problem of the standard SVR is that it considers data in a global fashion only. Therefore it may lack the flexibility to capture the local trend of data; this is a critical aspect of volatile data, especially financial time series data. Aiming to attack this issue, we propose the localized support vector regression (LSVR) model. This novel model is demonstrated to provide a systematic and automatic scheme to adapt the margin locally and flexibly; while the margin in the standard SVR is fixed globally. Therefore, the LSVR can tolerate noise adaptively. The proposed LSVR is promising in the sense that it not only captures the local information in data, but more importantly, it establishes connection with several models. More specifically: (1) it can be regarded as the regression extension of a recently proposed promising classification model, the Maxi-Min Margin Machine; (2) it incorporates the standard SVR as a special case under certain mild assumptions. We provide both theoretical justifications and empirical evaluations for this novel model. The experimental results on synthetic data and real financial data demonstrate its advantages over the standard SVR. [Copyright &y& Elsevier]

Abstract Random forests algorithms have been widely used in many classification and regression applications. However, the theory of random forests lags far behind their applications. In this paper, we propose a novel random forests classification algorithm based on cooperative game theory. The Banzhaf power index is employed to evaluate the power of each feature by traversing possible feature coalitions. Hence, we call the proposed algorithm Banzhaf random forests (BRFs). Unlike the previously used information gain ratio, which only measures the power of each feature for classification and pays less attention to the intrinsic structure of the feature variables, the Banzhaf power index can measure the importance of each feature by computing the dependency among the group of features. More importantly, we have proved the consistency of BRFs, which narrows the gap between the theory and applications of random forests. Extensive experiments on several UCI benchmark data sets and three real world applications show that BRFs perform significantly better than existing consistent random forests on classification accuracy, and better than or at least comparable with Breiman’s random forests, support vector machines (SVMs) and k-nearest neighbors (KNNs) classifiers. Highlights • A novel random classification forests algorithm, called Banzhaf random forests (BRFs), is proposed. • The Banzhaf power index is employed to evaluate the power of each feature by traversing possible feature coalitions. • The consistency of BRFs is proved. [ABSTRACT FROM AUTHOR]

• Artificial immune network (AIN) for optimizing machine learning algorithms. • Convergence analysis of the proposed optimization algorithm is presented. • Successful experimentation on many benchmark machines learning datasets. • Achieved classification performance boost of 2% – 9%. Artificial Immune Networks (AIN) is a population-based evolutionary algorithm that is inspired by theoretical immunology. It applies ideas and metaphors from the biological immune system to solve multi-disciplinary problems. This paper presents a novel application of the AIN for optimizing shallow machine learning (ML) classification algorithms. AIN accomplishes this task by searching the best hyper-parameter set for a specific classification algorithm (also termed model selection), which minimizes training error and enhances the generalization capability of the algorithm. We present a convergence analysis of the proposed algorithm and employ it in conjunction with selected, well-known ML classifiers, namely, an extreme learning machine (ELM), a support vector machine (SVM) and an echo state network (ESN). The performance is evaluated in terms of classification accuracy and learning time, using a range of benchmark datasets, and compared against grid search as well as evolutionary strategy (ES)-based optimization techniques. An empirical study with different datasets demonstrates improved classification accuracy of SVM, from 2% to 5%, for ESN from 3% to 6%, whereas in the case of ELM from 3% to 9%. Comparative simulation results demonstrate the potential of AIN as an alternative optimizer for shallow ML algorithms. [ABSTRACT FROM AUTHOR]

Exchange rate prediction is an important yet challenging problem in financial time series analysis. Although the historical exchange rates can provide valuable information, other factors will also affect the prediction significantly. These factors could be numerical or non-numerical ones, which are related to politics, economics, military, or even market psychology. Previous automatic exchange rate prediction merely considers numerical data (or simply the historical rates) for predicting the next day value. In this paper, we show how to utilize and combine many related factors, both numerical and non- numerical factors, for exchange rate prediction. With an example on forecasting exchange rate between US dollar and Japanese yen, we investigate how to exploit the information from non-numerical factors. We then engage a novel integrated approach which successfully combines information obtained from both numerical and non-numerical factors. We show how to quantify the non-numerical fundamental information, provide details steps on how to construct single predictors on different kinds of information separately, and finally describe how to integrate these separate predictors. Experimental results showed that our method can achieve the Directional Symmetry (DS) accuracy of 86.96%, which is much higher than only exploiting numerical information. [ABSTRACT FROM AUTHOR]