Search

In this paper, we propose a new filtering framework called discriminatively guided image filtering (DGF), for hyperspectral image (HSI) classification. DGF integrates a discriminative classifier and a generative classifier by the guided filtering (GF), considering the complementary strength of these two types of classification paradigms. To demonstrate the effectiveness of the proposed framework, the combination of support vector machine (SVM) and linear discriminative analysis (LDA), which serve as a discriminative classifier and a generative classifier respectively, is investigated in this paper. Specifically, the original HSI is projected into the low-dimensional space induced by LDA to serve as guidance images for filtering the intermediate classification results induced by SVM. Experiment results show the superior performance of the proposed DGF compared with that of the principal component analysis (PCA)-based GF.

In this paper, we introduce taxonomies for similarity and dissimilarity measures, respectively, based on their mathematical properties. Further, we propose a definition for rank equivalence of (dis)similarities regarding given data for prototype based methods. Starting with this definition we provide a measure to judge the degree of equivalence, which can be used to compare respective measures as well as to consider the influence of data preprocessing regarding a single (dis)similarity measure. In the last part of the paper an adaptive mixture approach of (dis)similarity measures for improved classification learning is presented.

Interval-valued data arises in situations where it is needed to manage either the uncertainty related to measurements, or the variability inherent to a group rather than an individual. This paper proposes a fuzzy c-means clustering algorithm based on suitable adaptive City-Block and Hausdorff distances with the purpose to cluster interval-valued data. This fuzzy c-means clustering algorithm optimizes explicitly an objective function by alternating three steps aiming to compute the fuzzy cluster representatives, the fuzzy partition, as well as relevance weights for the interval-valued variables. Indeed, most often conventional fuzzy c-means clustering algorithms consider that all variables are equally important for the clustering task. However, in real situations, some variables may be more or less important or even irrelevant for clustering. Due to the use of adaptive City-Block and Hausdorff distances, the paper proposes a fuzzy c-means clustering algorithm that tackles this problem with a step where a relevance weight is automatically learned for each interval-valued variable. Additionally, various tools for the fuzzy partition and fuzzy cluster interpretation of interval-valued data provided by this algorithm is also presented. Experiments with suitable synthetic and real interval-valued datasets demonstrate the robustness and the usefulness of this fuzzy c-means clustering algorithm and the merit of the fuzzy partition and fuzzy cluster interpretation tools.

Over the past years, digital image forensics has become a hot topic in multimedia security field. Among various image forensics technologies, image resampling detection is a standard detection tool. Furthermore, examining parameters of geometric transformations such as scaling factors or rotation angles is very useful for exploring the overall manipulation history of an image. In this paper, we propose a learning-to-rank approach for automatically estimating the scaling factor based on the normalized energy density features and moment features. Specifically, the difference of these features of the ordered image pairs are used for training, then the scaling factor of a new input image can be evaluated from its corresponding rank values. Our proposed method can not only effectively eliminate the long-known ambiguity between upscaling and downscaling in the analysis of resampling but also accurately estimate the factors of downscaling and weak scaling, i.e., the scaling factors near 1. Empirical experiments on extensive images with different scaling factors demonstrate the superiority of our proposed method when compared with the state-of-the-art methods. HighlightsThis paper proposes a learning-to-rank framework of image scaling factor estimation.Difference of two features of the ordered image pairs are used to train rank values.Scaling factor of a revised image can be evaluated according to the learned rank value.

This paper proposes a graph feature selection method for dementia diagnosis, by adding the information inherent in the observations into a sparse multi-task learning framework. Specifically, this paper first defines two relations (i.e., the feature-feature relation and the sample-sample relation, respectively) based on the prior knowledge in the data. The feature-feature selection enforces the similarity relationship between features to be preserved in the coefficient matrix while the sample-sample relation is designed to preserve the relation between samples invariant in the predicted space. Then we embed these two kinds of relations into a multi-task learning framework (i.e., a least square loss function plus an l2-norm regularization term) to conduct feature selection. Furthermore, we feed the reduced data into Support Vector Machine (SVM) for conducting the identification of Alzheimer's Disease (AD). Finally, the experimental results on a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset showed the effectiveness of the proposed method in terms of classification accuracy, by comparing with the state-of-the-art methods, including k Nearest Neighbor (kNN), ridge regression, SVM, and so on. Refereed/Peer-reviewed

The paper presents a persons' verification method based on the dynamic features of a given signature. In the proposed approach features space is connected with the set of similarity measures which can be used in the signature verification process. Features and linked similarity coefficients create a new composed signature features. Composed features are then reduced in the Hotelling reduction process, where the most appropriate features together with the most distinctive similarity measures are determined for each person. Finally, it leads to reduction of a composed signature features space. Compared to other approaches proposed solution automatically selects the best discriminatory features and similarity measures.The obtained verification results confirm that the proposed method, compared to other verification techniques, is very efficient. Proposed approach has been checked in different experiments, where various classifiers have been taken into consideration - perceptron and Bayesian network based classifiers as well as k-NN, random trees, random forests and others.In this paper we have included the simulation results for the two available datasets of dynamic signatures: SVC2004 and MCYT databases.

Constructive and destructive parsimonious extreme learning machines (CP-ELM and DP-ELM) were recently proposed to sparsify ELM. In comparison with CP-ELM, DP-ELM owns the advantage in the number of hidden nodes, but it loses the edge with respect to the training time. Hence, in this paper an equivalent measure is proposed to accelerate DP-ELM (ADP-ELM). As a result, ADP-ELM not only keeps the same hidden nodes as DP-ELM but also needs less training time than CP-ELM, which is especially important for the training time sensitive scenarios. The similar idea is extended to regularized ELM (RELM), yielding ADP-RELM. ADP-RELM accelerates the training process of DP-RELM further, and it works better than CP-RELM in terms of the number of hidden nodes and the training time. In addition, the computational complexity of the proposed accelerating scheme is analyzed in theory. From reported results on ten benchmark data sets, the effectiveness and usefulness of the proposed accelerating scheme in this paper is confirmed experimentally.

Object tracking is a fundamental problem in computer vision. Although much progress has been made, object tracking is still a challenging problem as it entails learning an effective model to account for appearance change caused by intrinsic and extrinsic factors. To improve the reliability and effectiveness, this paper presents an approach that explores the combination of graph-based ranking and multiple feature representations for tracking. We construct multiple graph matrices with various types of visual features, and integrate the multiple graphs into a regularization framework to learn a ranking vector. In particular, the approach has exploited temporal consistency by adding a regularization term to constrain the difference between two weight vectors at adjacent frames. An effective iterative optimization scheme is also proposed in this paper. Experimental results on a variety of challenging video sequences show that the proposed algorithm performs favorably against the state-of-the-art visual tracking methods.

This paper explores the performance of Ensembles of Extreme Learning Machine classifiers for hyperspectral image classification and segmentation in a semisupervised and spatially regularized process. The approach assumes that we have available only a small training set of labeled samples, which we enrich with a set of guessed labelings on selected samples from the vast pool of unlabeled image pixels. Selection and label guessing is conditioned to an unsupervised classification of the image pixel spectra, and to the spatial proximity to the labeled samples in the image domain. Unlabeled pixels falling in the spatial neighborhood of a labeled training sample, and belonging to the same unsupervised class, acquire its label. Unsupervised classification can be performed by any clustering technique, in this paper we have resorted to the classical K-means. The classifier built from the enriched training dataset is applied to the entire hyperspectral image. Finally, we perform a spatial regularization of the classification label image, maximizing a rather general prior smoothness criterion, by the selection of the most frequent class in each pixel neighborhood. This paper reports experiments with homogeneous ensembles of ELM, rELM, and OP-ELM classifiers, including a sensitivity analysis over the ensemble size and the number of hidden nodes. Computational experiments on four well known benchmarking hyperspectral images give state-of-the-art results.

In actual industrial processes, the data used for modeling usually contains some outliers, which may lead to a corrupted approximation function. In this paper, we propose a new Robust Extreme Learning Machine method based on a Bayesian framework (RBELM). The main idea of RBELM is to replace the Gaussian distribution with a heavy-tailed distribution as the probability density function of the model output and give weights to the likelihood function based on the error of the model output, which makes the model more robust to outliers. Two different heavy-tailed distributions are used in this paper. One is the Laplace distribution, and the corresponding model is denoted as RBELM-l. In RBELM-l, the posterior distribution is replaced by an approximate surrogate function, and then the parameters learning problem can be solved by means of Maximum Likelihood-type II (ML-II). The other is the Student's t-distribution, and the corresponding model is denoted as RBELM-t. In order to solve RBELM-t, the Student's t-distribution is written as a scale-mixture of infinitely many Normal distributions and a Gamma distribution. Then, the variational inference method is used to obtain an approximate solution for the parameters. Also, we propose two methods for giving weights to the likelihood function based on the concepts of robust statistic. Finally, a robust error measure is proposed for evaluating the performance of models with outliers in the testing data. Results of numerical comparison based on one synthesis data set, four real benchmark regression problems, and two actual industrial modeling problems show the usefulness of the proposed RBELM-l and RBELM-t.

Development of an efficient real time intrusion detection system (IDS) has been proposed in the paper by integrating Q-learning algorithm and rough set theory (RST). The objective of the work is to achieve maximum classification accuracy while detecting intrusions by classifying NSL-KDD network traffic data either 'normal' or 'anomaly'. Since RST processes discrete data only, by applying cut operation attributes in training data are discretized. Using indiscernibility concept of RST, reduced attribute sets, called reducts are obtained and among the reducts a single reduct is chosen which provides highest classification accuracy. However, for the test data the same reduct would not provide highest classification accuracy due to change of discretized attribute values. Therefore, to overcome the problem discretization and feature selection processes are dealt in a comprehensive and systematic way in the paper using machine learning approach. The Q-learning algorithm has been modified to learn optimum cut value for different attributes so that corresponding reduct produces maximum classification accuracy while classifying network traffic data. Since, not all attributes but reduct only take part to detect intrusions, the proposed algorithm is faster than Q-learning and reduces complexity of the IDS. Classification accuracy with 98% success rate has been obtained using real time data, which demonstrates superior performance compared to other classifiers.

The massive amount of available data potentially used to discover patterns in machine learning is a challenge for kernel based algorithms with respect to runtime and storage capacities. Local approaches might help to relieve these issues. From a statistical point of view local approaches allow additionally to deal with different structures in the data in different ways. This paper analyses properties of localized kernel based, non-parametric statistical machine learning methods, in particular of support vector machines (SVMs) and methods close to them. We will show there that locally learnt kernel methods are universally consistent. Furthermore, we give an upper bound for the maxbias in order to show statistical robustness of the proposed method.

The output weights computing of extreme learning machine (ELM) encounters two problems, the computational and outlier robustness problems. The computational problem occurs when the hidden layer output matrix is a not full column rank matrix or an ill-conditioned matrix because of randomly generated input weights and biases. An existing solution to this problem is Singular Value Decomposition (SVD) method. However, the training speed is still affected by the large complexity of SVD when computing the Moore-Penrose (MP) pseudo inverse. The outlier robustness problem may occur when the training data set contaminated with outliers then the accuracy rate of ELM is extremely affected. This paper proposes the Extended Complete Orthogonal Decomposition (ECOD) method to solve the computational problem in ELM weights computing via ECODLS algorithm. And the paper also proposes the other three algorithms, i.e. the iteratively reweighted least squares (IRWLS-ELM), ELM based on the multivariate least-trimmed squares (MLTS-ELM), and ELM based on the one-step reweighted MLTS (RMLTS-ELM) to solve the outlier robustness problem. However, they also encounter the computational problem. Therefore, the ECOD via ECODLS algorithm is also used successfully in the three proposed algorithms. The experiments of regression problems were conducted on both toy and real-world data sets. The outlier types are one-sided and two-sided outliers. Each experiment was randomly contaminated with outliers, of one type only, with 10%, 20%, 30%, 40%, and 50% of the total training data size. Meta-metrics evaluation was used to measure the outlier robustness of the proposed algorithms compared to the existing algorithms, i.e. the minimax probability machine regression (MPMR) and the ordinary ELM. The experimental results showed that ECOD can effectively replace SVD. The ECOD is robust to the not full column rank or the ill-conditional problem. The speed of the ELM training using ECOD is also faster than the ordinary training algorithm. Moreover, the meta-metrics measure showed that the proposed algorithms are less affected by the increasing number of outliers than the existing algorithms.

In this paper, a novel semi-supervised learning approach is proposed. It assumes that, for the ith sample x"i, the samples from x"i's sparse neighborhood have the same label with x"i and the label of x"i can be linearly reconstructed by the labels of those samples from x"i's sparse neighborhood. Our algorithm firstly selects the sparse neighborhood for each sample, and then in that sparse neighborhood finds the sparse coefficients to represent the local geometry structure, finally seeks a label propagation way. Different from many existing methods, we construct the adapting graph, simultaneously, give the weight of each edge. What's more, we highlight the role of those samples in that sparse neighborhood, meanwhile, eliminate the role of those samples out of that sparse neighborhood. The experimental results on face recognition and document classification demonstrate the effectiveness and efficiency of our proposed approach in this paper.

Whenever evolutionary algorithms are used to solve certain classes of problems such as those that present a huge search space, the incorporation of problem-specific knowledge is required to achieve adequate levels of performance. In this paper, we propose a multi-objective optimization-based procedure that includes such a domain-specific knowledge to cope with a difficult problem, the protein structure prediction (PSP). This problem is considered to be an open problem as there is no recognized “best” procedure to find solutions. It presents a vast search space and the analysis of each protein conformation requires significant amount of computing time. In our procedure, we provide a reduction of the search space by using the dependent rotamer library and include new heuristics to improve a multi-objective approach to PSP based on the PAES algorithm. As it is shown in the paper, by using benchmark proteins from the CASP8 set, this hybrid PSP procedure provides competitive results when it is compared with some of the better proposals appeared up to now.

In this paper, a general kernelization framework for learning algorithms is proposed via a two-stage procedure, i.e., transforming data by kernel principal component analysis (KPCA), and then directly performing the learning algorithm with the transformed data. It is worth noting that although a very few learning algorithms were also kernelized by this procedure before, why and under what condition this procedure is feasible have not been further studied. In this paper, we explicitly present this kernelization framework, and give a rigorous justification to reveal that under some mild conditions, the kernelization under this framework is equivalent to traditional kernel method. We show that these mild conditions are usually satisfied in most of learning algorithms. Therefore, most of learning algorithms can be kernelized under this framework without having to reformulate it into inner product form, which is a common yet vital step in traditional kernel methods. Enlightened by this framework, we also propose a novel kernel method based on the low-rank KPCA, which could be used to remove the noise in the feature space, speed up the kernel algorithm and improve the numerical stability for the kernel algorithm. Experiments are presented to verify the validity and effectiveness of the proposed methods.

Olive fruit fly trap measurements are used as one of the indicators for olive grove infestation, and therefore, as a consultation tool on spraying parameters. In this paper, machine learning techniques are used to predict the next olive fruit fly trap measurement, given input knowledge of previous trap measurements as well as an attribute that acts as a correlation model between the temperature and the development of a pest’s population, known as the Degree Day model. This is the first time the Degree Day model is utilized as input in classification algorithms for the prediction of olive fruit fly trap measurements. Various classification algorithms are employed and applied to different environmental settings, in extensive comparative experiments, in order to detect the impact of the latter on olive fruit fly population prediction.

Clustering generic data, i.e., data not specific to a particular field, is a challenging problem due to their diverse complex structures in the original feature space. Traditional approaches address this problem by complementing clustering with feature learning methods, which either capture the intrinsic structure of the data or represent the data such that clusters are better revealed. In this paper, we propose an approach referred to as Extreme Learning Machine for Joint Embedding and Clustering (ELM-JEC), which incorporates desirable properties of both types of feature learning methods at the same time, specifically by (1) preserving the manifold structure of the data in the original space; (2) maximizing the class separability of the data in the embedded space. Since either type of method has improved clustering performance in some cases, our motivation is to integrate the two desirable properties to further improve the accuracy and robustness of clustering. Additional notable features of ELM-JEC are that it provides nonlinear feature mappings and achieves feature learning and clustering in the same formulation. The proposed approach can be implemented using alternating optimization, and its clustering performance compares favorably with several state-of-the-art methods on the real-world benchmark datasets. MOE (Min. of Education, S’pore)

Most proposed metaheuristics for feature selection and model parameter optimization are based on a two-termed L o s s + P e n a l t y function. Their main drawback is the need of a manual set of the parameter that balances between the loss and the penalty term. In this paper, a novel methodology referred as the GA-PARSIMONY and specifically designed to overcome this issue is evaluated in detail in thirteen public databases with five regression techniques. It is a GA-based meta-heuristic that splits the classic two-termed minimization functions by making two consecutive ranks of individuals. The first rank is based solely on the generalization error, while the second (named ReRank) is based on the complexity of the models, giving a special weight to the complexity entailed by large number of inputs. For each database, models with lowest testing RMSE and without statistical difference among them were referred as winner models. Within this group, the number of features selected was below 50%, which proves an optimal balance between error minimization and parsimony. Particularly, the most complex algorithms (MLP and SVR) were mostly selected in the group of winner models, while using around40–45% of the available attributes. The most basic IBk, ridge regression (LIN) and M5P were only classified as winner models in the simpler databases, but using less number of features in those cases (up to a 20–25% of the initial inputs).

The real-world networking data may contain different types of attribute views and relational view. Hence, it is desirable to collectively use available attribute views and relational view in order to build effective learning models. We call this framework multi-attribute and relational learning. Collective classification is one of the popular approaches that can handle both attribute and relational information for network data. However, in collective classification only one type of attribute and relational view is involved and little attention is received for multi-attribute and relational learning. In this paper, we propose a new semi-supervised collective classification approach, called hypergraph regularized generative model (HRGM), for multi-attribute and relational learning. In the approach, a generative model based on the Probabilistic Latent Semantic Analysis (PLSA) method is developed to leverage attribute information, and a hypergraph regularizer is incorporated to effectively exploit higher-order relational information among the data samples. Experimental results on various data sets have demonstrated the effectiveness of the proposed HRGM, and revealed that our approach outperforms existing collective classification methods and multi-view classification methods in terms of accuracy.

A primary challenge of time series classification is how to extract powerful features from training samples. Two kinds of classification methods, global-based and local-based methods, have been studied widely in recent years. The global-based methods, like 1-Nearest Neighbor(1-NN), take the entire series as features, which have the drawback that they are not able to indicate the intrinsic characters of a class. The local-based methods overcome this weakness by employing discriminative time series subsequences as features, called shapelets. However, most local-based methods are computationally expensive because of the massive number of shapelet candidates. In this paper, we propose a novel shapelets extraction method which takes each time series as a high-dimensional data and then finds the discriminative dimensions corresponding to the positions of shapelets. More specifically, the discriminative dimensions are determined by combining Local Fisher Discriminant Analysis (LFDA) method and two sparse restrictions which can encourage the continuous characteristic of time series. Extensive experimental results show that the proposed method achieves significant improvement compared to the existing shapelet-based methods in terms of classification accuracy and running time on the commonly used time series datasets. In addition, comparing with the accepted time series classification methods, NNDTW and COTE, our method still gets better results.

Extreme Learning Machine (ELM) has recently increased popularity and has been successfully applied to a wide range of applications. Variants using regularization are now a common practice in the state of the art in ELM field. The most commonly used regularization is the l2 norm, which improves generalization but result in a dense network. Regularization based on the elastic net has also been proposed but mainly applied to regression and binary classification problems. In this paper, we propose a generalization of regularized ELM (R-ELM) for multiclass classification problems, termed GR-ELM. We achieve such generalization using the l2,1 and Frobenius norm. Traditional R-ELM is a particular case of our method when binary classification tasks are considered. We also propose an alternative algorithm for GR-ELM when training data is distributed, namely DGR-ELM. We use Alternating Direction Method of Multipliers (ADMM) for solving the resulting optimization problems. Message Passing Interface (MPI) in a Single Program, Multiple Data (SPMD) programming style is chosen for implementing DGR-ELM. Extensive experiments are conducted to evaluate the proposed method. Our experiments show that GR-ELM and DGR-ELM have similar training and testing accuracy when compared to R-ELM, although usually faster testing time is obtained with our method due to the compactness of the resulting network.

Multi-view clustering, which explores complementary information between multiple feature sets by consensus grouping, has benefited many data analytic applications. The majority of previous multi-view clustering studies usually assume that all feature sets appear in complete. In real-world applications, however, it is often the case that some views could suffer from the missing of examples, resulting in incomplete feature sets. The incompleteness of views makes it difficult to synthesize all feature sets and achieve a comprehensive description of data samples. In this paper, we develop a novel incomplete multi-view clustering method, which projects all incomplete multi-view data to a complete and unified representation in a common subspace. Different from existing researches which exploit shallow learning to identify the common subspace, a deep incomplete multi-view clustering (DIMC) incorporating with the constraint of intrinsic geometric structure is proposed here to couple incomplete multi-view samples. To bridge the gap between each view and the common representation, the multi-view deep coupled networks are trained to map high-level semantic features. Besides, to preserve the local invariance for each view, an affinity graph based regularizer is constructed to encode geometrical information. Therefore, a new objective function is developed and the optimization processes are presented. Extensive experiments on several real-world datasets demonstrate that the proposed DIMC is superior to the state-of-the-art incomplete multi-view clustering methods.

Partial label learning (PLL) is a new weakly supervised learning framework that addresses the classification problems, where the true label of each training sample is concealed in a set of candidate labels. To learn from such weakly supervised training data, the key is to disambiguate the ambiguous labeling information. Because it is difficult to address by only focusing on the manipulation in the label space, manifold structure among training data in the feature space has gradually been exploited simultaneously to facilitate the disambiguation process by researchers in recent years. However, the manifold structure is commonly analyzed under an assumption that the samples close to each other in the feature space will share identical labels in the label space, which may be not correct in many real-world problems. In this paper, geometric mean metric learning approach is employed to learn a distance metric for PLL problems such that can maintain the aforementioned assumption correct in as many situations as possible. It is significantly more challenging than the conventional setup of distance metric learning because it is difficult to precisely identify whether a pair of training samples belong to the same class. We propose an alternative approach in which each training sample and its neighbor with shared candidate label are taken as a similarity pair, and each training sample and its neighbor without shared candidate label are taken as a dissimilarity pair. Considering that two samples with shared candidate label do not necessarily come from the same class, a weight is placed on each similarity pair. The experimental results on twenty four controlled UCI data sets and six real-world PLL problems show the proposed distance metric learning approach can be used as a front end of both the PLL algorithms exploiting the manifold structure among training data and other existing distance-based PLL algorithms to significantly improve their performance.

Recently, considerable efforts have been made in visual search reranking towards refining initial text-based image search results. In this paper, we propose a hypergraph based semi-supervised ranking method called HyperSSR for image search reranking. According to the basic visual consistency principle that visually similar images should have similar ranking scores, we introduce the hypergraph to capture the intrinsic geometrical structure of the data distribution. To build a robust hypergraph, a novel hypergraph construction approach is developed to incorporate relevance and pseudo relevance degree information from labeled and unlabeled samples, respectively. Based on the premise that a ranking model should work better with the prior pairwise preferences, we jointly incorporate the hypergraph regularizer and the prior pairwise preferences information into a unified ranking learning framework. Experimental results on MSRA-MM 1.0 dataset suggest our proposed approach produces superior performances compared with several state-of-the-art methods.

Canonical correlation analysis (CCA) is a useful dimensionality reduction method and has been widely used in projecting multi-view data. However, CCA learns from training data consisting of only target classes, ignoring Universum data that belongs to none of the target classes but comes from the same domain as the target classes. Recently, incorporating Universum data in learning has been used to gain more prior knowledge about the application domain and has been shown to achieve favorable improvements. In this paper, we extend CCA with Universum learning for multi-view data and the proposed method is termed as Universum CCA (UCCA). Due to the fact that Universum data in each view does not belong to any target class, correlation between Universum data and target data should be minimized. Consequently, UCCA aims to find basis vectors in multiple views to ensure that correlations between projections of target data are mutually maximized but correlations between projections of Universum data and target data mutually minimized. UCCA can be expressed as a generalized eigenvalue problem and the extracted features express patterns more distinctly. The experimental results on several real-world datasets demonstrate its marked improvements over conventional methods.

The selection of hyper-parameters in support vector machines (SVM) is a key point in the training process of these models when applied to regression problems. Unfortunately, an exact method to obtain the optimal set of SVM hyper-parameters is unknown, and search algorithms are usually applied to obtain the best possible set of hyper-parameters. In general these search algorithms are implemented as grid searches, which are time consuming, so the computational cost of the SVM training process increases considerably. This paper presents a novel study of the effect of including reductions in the range of SVM hyper-parameters, in order to reduce the SVM training time, but with the minimum possible impact in its performance. The paper presents reduction in parameter C, by considering its relation with the rest of SVM hyper-parameters (@c and @e), through an approximation of the SVM model. On the other hand, we use some characteristics of the Gaussian kernel function and a previous result in the literature to obtain novel bounds for @c and @e hyper-parameters. The search space reductions proposed are evaluated in different regression problems from UCI and StatLib databases. All the experiments carried out applying the popular LIBSVM solver have shown that our approach reduces the SVM training time, maintaining the SVM performance similar to when the complete range in SVM parameters is considered.

The problem of residual variance estimation consists of estimating the best possible generalization error obtainable by any model based on a finite sample of data. Even though it is a natural generalization of linear correlation, residual variance estimation in its general form has attracted relatively little attention in machine learning. In this paper, we examine four different residual variance estimators and analyze their properties both theoretically and experimentally to understand better their applicability in machine learning problems. The theoretical treatment differs from previous work by being based on a general formulation of the problem covering also heteroscedastic noise in contrary to previous work, which concentrates on homoscedastic and additive noise. In the second part of the paper, we demonstrate practical applications in input and model structure selection. The experimental results show that using residual variance estimators in these tasks gives good results often with a reduced computational complexity, while the nearest neighbor estimators are simple and easy to implement.

This paper presents an ensemble neural network and interval neutrosophic sets approach to the problem of binary classification. A bagging technique is applied to an ensemble of pairs of neural networks created to predict degree of truth membership, indeterminacy membership, and false membership values in the interval neutrosophic sets. In our approach, the error and vagueness are quantified in the classification process as well. A number of aggregation techniques are proposed in this paper. We applied our techniques to the classical benchmark problems including ionosphere, pima-Indians diabetes, and liver-disorders from the UCI machine learning repository. Our approaches improve the classification performance as compared to the existing techniques which applied only to the truth membership values. Furthermore, the proposed ensemble techniques also provide better results than those obtained from only a single pair of neural networks.

In object detection, object proposal methods have been widely used to generate candidate regions which may contain objects. Object proposal based on superpixel merging is one kind of object proposal methods, and the merging strategies of superpixels have been extensively explored. However, the ranking of generated candidate proposals still remains to be further studied. In this paper, we formulate the ranking of object proposals as a learning to rank problem, and propose a novel object proposals ranking method based on ListNet. In the proposed method, Selective Search, which is one of the state-of-the-art object proposal methods based on superpixel merging, is adopted to generate the candidate proposals. During the superpixel merging process, five discriminative objectness features are extracted from superpixel sets and the corresponding bounding boxes. Then, to weight each feature, a linear neural network is learned based on ListNet. Consequently, objectness scores can be computed for final candidate proposals ranking. Extensive experiments demonstrate the effectiveness and robustness of the proposed method.

Classification of brain voxels into gray matter, white matter, and cerebrospinal fluid (CSF) using magnetic resonance imaging (MRI) is pivotal for quantitative brain analyses. In spite of its computational effectiveness, the most commonly used statistical classification models are less capable of handling the intensity non-uniformity (INU) and partial volume effect (PVE), and hence may produce less accurate results. In this paper, we propose a novel approach, namely the VMG-NDE algorithm, to improve brain voxel classification in MRI images by considering all effects simultaneously. There are four planks in this algorithm, including (1) using variational mixture of Gaussians (VMG) model to characterize the variation of voxel values caused by PVE, (2) training a cohort of local VMG models on small data volumes extracted from the image to reduce the impact of INU, (3) employing the niche differential evolution (NDE) to infer each local VMG model, aiming to avoid falling into local optima, and (4) constructing a probabilistic brain atlas for each study and using it to incorporate the anatomy prior into the classification process. After training local VMG models, we classify each brain voxel using a linear combination of the predictions generated by all those models. This algorithm has been evaluated against the variational expectation-maximization based and genetic algorithm based segmentation algorithms and the segmentation routines in the widely used statistical parametric mapping (SPM) package, expectation-maximization segmentation (EMS) package and FSL package on both synthetic and clinical T1-weighted brain MRI studies. Our results suggest that the proposed algorithm can differentiate major brain tissue types more effectively and produce improved brain voxel classification accuracy.

With the rapid development of collection techniques, it is easy to gather various data which come from different domains, such as images, videos, documents, and etc, how to group these heterogeneous data becomes a research issue. Traditional techniques handle these clustering tasks separately, that is one task for one domain, so that they ignore the interactions among domains. In this paper, we present a co-transfer clustering method to deal with these separate tasks together with the aid of co-occurrence data which contain some instances represented in different domains. The proposed method consists of two steps, one is to learn the subspace of different domains that uncovers the latent common topics and respects the intrinsic geometric structure, the next is to simultaneously cluster the instances in all domains via the symmetric nonnegative matrix factorization method. A series of experiments on real-world data sets have shown the performance of the proposed method is better than the state-of-the-art methods.

This paper addresses an efficient scheme for clustering time-series through a novel regression mixture strategy (RMM) that simultaneously utilizes the benefits of the Markov random field (MRF) and mean template. Each component of the proposed RMM is a mixture of Student’s-t and non-symmetric Rayleigh distributions. The Student’s-t distribution has been validated to be effective in reducing the influence of outliers. Another advantage is it considers the property of time-series in which spatially adjacent voxels have higher probabilities when belonging to the same cluster; thus, this study seeks to impose the spatially interdependent constraints in the form of a prior through a Bayesian theorem. By introducing the non-symmetric Rayleigh distribution, the proposed method has flexibility to fit various types of observed time-series. To optimize the model parameters, the proposed algorithm is formulated as a maximum a posteriori (MAP) estimation problem. An expectation maximization (EM)-type approach is eventually executed to obtain the update equations. Additionally, an efficient energy function is applied in the proposed approach such that the EM-MAP algorithm can be directly applied to calculate the objective function, which makes the proposed approach easier to implement. To illustrate the suitability of the proposed approach, we have extensively compared clustering accuracy, curve square error, and intraclass index with those of existing similarity methods on synthetic spatiotemporal datasets, synthetic fMRI time-series, and real life fMRI time-series.

Zero-Shot Learning tries to predict the novel class samples that do not have any labeled instances in the training stage. This is typically achieved by exploring intermediate side information to transfer knowledge from seen classes to unseen testing ones. Different approaches vary in the usage of the side information and embedding methods. However, most methods only concern the relationships among different modalities but ignore to preserve the consistency among different samples in the same modality. In this paper, we propose an approach called Regularized Cross-Modality Ranking (ReCMR) to capture the semantic information from heterogeneous sources by taking both intra-modal and inter-modal semantics into consideration. Specifically, we employ the hinge ranking loss to exploit the structures among different modalities and devise efficient regularizers to constrain the variation of the samples in the identical modality. Experimental results on the popular AwA and CUB datasets show that ReCMR significantly outperforms the state-of-the-art methods.

Recent research has demonstrated that the performance of sparse representation based methods for single image super-resolution (SISR) reconstruction relies strongly on the degree of accuracy of sparse coding coefficients, and accordingly several more accurate models have been developed to overcome it by exploiting the nonlocal patch redundancy within the observed image. However, the capability of those models may be limited as they fail to simultaneously consider the redundant information both within the same scale and across multiple scales. Thus, in this paper, an improved SISR reconstruction method is proposed, in which a compensative pair of regularization terms defined by l1-norm is first constructed by taking advantage of the multiscale self-similarity. Then the calculated sparse coefficients are further aligned to this pair of standards in order to suppress sparse coding noise, and consequently result in more faithful recoveries. Finally, based on conventional iterative shrinkage-thresholding algorithm, a local-to-global and coarse-to-fine mathematic implementation is established to solve the proposed model effectively. Extensive experiments on both synthetic and real images demonstrate that our proposed method leads to a promising SISR performance and surpasses the recently published counterparts in terms of both objective evaluation and visual perception.

For robust visual tracking, appearance modeling should be able to well separate the object from its backgrounds, while accurately adapt to its appearance variations. However, most of the existing tracking methods mainly focus on one of the two aspects; or design two different modules to combine them with the price of double computational cost. In this paper, by using pairwise metric learning, we present a novel appearance model for robust visual tracking. Specifically, visual tracking is viewed as a pairwise regression problem, and extreme learning machine (ELM) is utilized to construct the pairwise regression framework. In ELM-based pairwise training, two constraints are enforced: the target observations must have different regression outputs from those background ones; while the various target observations during tracking should have approximate regression outputs. Thus, the discriminative and generative capabilities are fully considered in a single object tracking model. Moreover, online sequential ELM (OS-ELM) is used to update the resulting appearance model, thereby leading to a more robust tracking process. Extensive experimental evaluations on challenging video sequences demonstrate the effectiveness and efficiency of the proposed tracker.

Despite the continuous development of features and mid-level representations, effectively and reliably measuring the similarity among images remains a challenging problem in image retrieval tasks. Once traditional measures consider only pairwise analysis, context-sensitive measures capable of exploiting the intrinsic manifold structure became indispensable for improving the retrieval performance. In this scenario, diffusion processes and rank-based methods are the most representative approaches. This paper proposes a novel hybrid method, named rank diffusion, which uses a diffusion process based on ranking information. The proposed method consists in a diffusion-based re-ranking approach, which propagates contextual information through a diffusion process defined in terms of top-ranked objects, reducing the computational complexity of the proposed algorithm. Extensive experiments considering a rigorous experimental protocol were conducted on six public image datasets and several different descriptors. Experimental results and comparison with state-of-the-art methods demonstrate that high effectiveness gains can be obtained, despite the low-complexity of the algorithm proposed.

In this paper, we study the problem of supervised Fully PolSAR (polarimetric synthetic aperture radar) image classification. We estimate a complex Wishart model distribution for each class using training data, and we use such models to design a new classification procedure based on a diffusion-reaction equation. The method relies on simultaneously filtering and classifying pixels within the image. The diffusion term smooths the patches within the image, and the reaction term tends to move the pixel values towards the closest (in the sense of stochastic distances) representative class. We present a detailed study of the method accuracy using both simulated and true data, and we provide optimum parameters for its use. We show that the proposed method outperforms the results obtained using maximum likelihood and usual stochastic distance classification methods.

In multi-label learning, each example is represented by a single instance and associated with multiple class labels. Existing multi-label learning algorithms mainly exploit label correlations globally, by assuming that the label correlations are shared by all the examples. Moreover, these multi-label learning algorithms exploit the positive label correlations among different class labels. In practical applications, however, different examples may share different label correlations, and the labels are not only positive correlated, but also mutually exclusive with each other. In this paper, we propose a simple and effective Bayesian model for multi-label classification by exploiting Local positive and negative Pairwise Label Correlations, named LPLC. In the training stage, the positive and negative label correlations of each ground truth label for all the training examples are discovered. In the test stage, the k nearest neighbors and their corresponding positive and negative pairwise label correlations for each test example are first identified, then we make prediction through maximizing the posterior probability, which is estimated on the label distribution, the local positive and negative pairwise label correlations embodied in the k nearest neighbors. A comparative study with the state-of-the-art approaches manifests a competitive performance of our proposed method.

Dimension reduction methods always catch many attentions, because it could effectively solve the curse of dimensionality problem. In this paper, we propose an unsupervised feature selection method which could efficiently select a subset of informative features from unlabeled data. We integrate the low-rank constraint, hypergraph theory, and the self-representation property of features in a unified framework to conduct unsupervised feature selection. Specifically, we represent each feature by other features to conduct unsupervised feature selection via the feature-level self-representation property. We then embed a low-rank constraint to consider the relations among features. Moreover, a hypergarph regularizer is utilized to consider both the high-order relations and the local structure of the data. This enables the proposed model to take into account both the global structure of the data (via the low-rank constraint) and the local structure of the data (via the hypgergraph regularizer). We use an 2, p-norm regularizer to satisfy the constraints. Therefore, the proposed model is more robust to the previous models due to achieving better feature selection model. Experimental results on benchmark datasets showed that the proposed method effectively selected the most informative features by removing the adverse effect of irrelevant/redundant features, compared to the state-of-the-art methods.

Over the past decades, features generated by different models have been designed to describe various aspects of object. To connect the complementary information and represent the data properly, effective heterogeneous feature fusion methods are required. Multiple kernel learning (MKL) methods are widely adopted to learn the feature weights and to fuse features on score-level. In this paper, we exploit score distribution to address the feature fusion problem and propose a novel method named score-distribution MKL (SD-MKL) for image classification. Different from existing MKL methods, SD-MKL uses weights which are learned from score curves as a constraint on the weights of kernels. It contains two stages in off-line part: (1) independent data is used to construct reference curves according to classes and feature type; (2) samples and corresponding score-distribution weights are put into multi-kernel support vector machine (MKSVM) to learn feature weights. Our experimental results demonstrate the effect of exploiting score-distribution information on two datasets, which significantly benefits the performance of image classification.

A dynamic single-element-based Tikhonov graph regularization NMF and CF integrating manifold regularization were proposed.The proposed methods alleviate the curse of dimensionality of dual GWNMF.The proposed methods evade the frequent manipulations of indicator matrices.The proposed methods show better recommendation on RMSE, MAE, recall, and precision. Matrix factorization (MF) is an increasingly important approach in the field of missing value prediction because recommender systems are rapidly becoming ubiquitous. MF-based collaborative filtering (CF) seeks to improve recommender performance by combining useritem matrix with MF. However, most MF-based approaches available at present could not obtain high prediction accuracy because of the sparse availability of useritem matrices in CF models. The present paper proposes a framework that involves two efficient MF, dynamic single-element-based CF-integrating manifold regularization (DSMMF) and dynamic single-element-based Tikhonov graph regularization non-negative MF (DSTNMF). The aim of this framework is to better use the intrinsic structure of useritem rating matrix and user/item content information, overcome the dimensionality curse and ill-posed problem of weighted graph NMF, and evade the frequent manipulations of indicator matrices that lack practicability. We validate the effectiveness of our proposed algorithms with respect to recommender performance by four indices on three datasets. We demonstrate that our proposed approaches lead to considerable improvement compared with several other state-of-the-art approaches.

View-based 3D object retrieval techniques have become increasingly important in various fields, and lots of ingenious studies have promoted the development of retrieval performance from different aspects. In this paper, we focus on the 2D projective views that represent the 3D objects and propose a boosting approach by evaluating the discriminative ability of each objects views. The dissimilarity between views semantic and discriminative ability is firstly investigated through classification performance. We then propose a simple, robust and effective measurement to study the views discriminative ability. By employing the proposed reverse distance metric, we utilize the discriminative information for many to many view set matching. The proposed algorithm is then employed with various features to boost the multi-model graph learning based retrieval method. We compare our approach with several state of the art methods on ETH-80 dataset and National Taiwan University 3D model dataset. The results demonstrate the effectiveness of our method and its excellent boosting performance.

This paper proposes a niching evolutionary algorithm with adaptive negative correlation learning, denoted as NEA_ANCL, for training the neural network ensemble. In the proposed NEA_ANCL, an adaptive negative correlation learning, in which the penalty coefficient is set to dynamically change during training, has been developed. The adaptation strategy is based on a novel population diversity measure with the purpose of appropriately controlling the trade-off between the diversity and accuracy in the ensemble. Further, a modified dynamical fitness sharing method is applied to preserve the diversity of population during training. The proposed NEA_ANCL has been evaluated on a number of benchmark problems and compared with related ensemble learning algorithms. The results show that our method can be used to design a satisfactory NN ensemble and outperform related works.

Eye movement data collection is very expensive and laborious. Moreover, there are usually missing values. Assuming that we are collecting eye movement data from a set of images viewed by different users, there is a possibility that we will not able to collect the data of every user from every imageone or more views may not be represented in the image. We assume that the relationships among the views can be learnt from the whole collection of views (or items). The task is then to reproduce the missing part of the incomplete items from the relationships derived from the complete items and the known part of these items. Using certain properties of tensor algebra, we showed that this problem can be formulated consistently as a regression type learning task. Furthermore, there is a maximum margin based optimisation framework in which this problem can be solved in a tractable way. This problem is similar to learning to predict where a person is looking in an image. Therefore, we proposed an algorithm called Tensor-based Multi-View Learning(TMVL) in this paper. Furthermore, we also proposed a technique for improving prediction by introducing a new feature set obtained from Kronecker decomposition of the image fused with users eye movement data. Using this new feature can improve prediction performance markedly. The proposed approach was proven to be more effective than two well-known saliency detection techniques.

In usual real-world clustering problems, the set of features extracted from the data has two problems which prevent the methods from accurate clustering. First, the features extracted from the samples provide poor information for clustering purpose. Second, the feature vector usually has a high-dimensional multi-source nature, which results in a complex cluster structure in the feature space. In this paper, we propose to use a combination of multi-task clustering and fuzzy co-clustering techniques, to overcome these two problems. In addition, the Bregman divergence is used as the concept of dissimilarity in the proposed algorithm, in order to create a general framework which enables us to use any kind of Bregman distance function, which is consistent with the data distribution and the structure of the clusters. The experimental results indicate that the proposed algorithm can overcome the two mentioned problems, and manages the complexity and weakness of the features, which results in appropriate clustering performances.

We proposed a robust unsupervised method to remove redundant and irrelevant features.Both the cluster centers and the sparse representation are predicted.The feature selection and clustering are performed simultaneously.An efficient iterative update algorithm based on ADMM is proposed.Superiority over seven existing methods is established for five data sets. Dimensionality reduction is a challenging task for high-dimensional data processing in machine learning and data mining. It can help to reduce computation time, save storage space and improve the performance of learning algorithms. As an effective dimension reduction technique, unsupervised feature selection aims at finding a subset of features to retain the most relevant information. In this paper, we propose a novel unsupervised feature selection method, called Robust Unsupervised Feature Selection via Matrix Factorization (RUFSM), in which robust discriminative feature selection and robust clustering are performed simultaneously under l2, 1-norm while the local manifold structures of data are preserved. The advantages of this work are three-fold. Firstly, both the latent orthogonal cluster centers and the sparse representation of the projected data points based on matrix factorization are predicted for selecting robust discriminative features. Secondly, the feature selection and the clustering are performed simultaneously to guarantee an overall optimum. Thirdly, an efficient iterative update algorithm, which is based on Alternating Direction Method of Multipliers (ADMM), is used for RUFSM optimization. Compared with several state-of-the-art unsupervised feature selection methods, the proposed algorithm comes with better clustering performance for almost all datasets we have experimented with here.

As an issue that attracts increasing interests in both academia and industry, multiple-shot person re-identification has shown promising results but suffers from real-scenario complexities and feature-crafting heuristics. To tackle the problems of set-level data variation and sparseness during re-identification, this paper proposes a novel metric learning method, named Fair Set-Collaboration Metric Learning, motivated by utilizing the opportunities whilst overcoming the challenges from the set of multiple instances. This method optimizes a new set-collaboration dissimilarity measure, which introduces the fairness principle into the collaborative representation based set to sets distance, in the set based metric learning framework. Experiments on widely-used benchmark datasets have demonstrated the advantages of this method in terms of effectiveness and robustness.

Amortized variational inference, whereby the inferred latent variable posterior distributions are parameterized by means of neural network functions, has invigorated a new wave of innovation in the field of generative latent variable modeling, giving rise to the family of deep generative models (DGMs). Existing DGM formulations are based on the assumption of a symmetric Gaussian posterior over the model latent variables. This assumption, although mathematically convenient, can be well-expected to undermine the eventually obtained representation power, as it imposes apparent expressiveness limitations. Indeed, it has been recently shown that even some moderate increase in the latent variable posterior expressiveness, obtained by introducing an additional level of dependencies upon auxiliary (Gaussian) latent variables, can result in significant performance improvements in the context of semi-supervised learning tasks. Inspired from these advances, in this paper we examine whether a more potent increase in the expressiveness and representation power of modern DGMs can be achieved by completely relaxing their typical symmetric (Gaussian) latent variable posterior assumptions: Specifically, we consider DGMs with asymmetric posteriors, formulated as restricted multivariate skew-Normal (rMSN) distributions. We derive an efficient amortized variational inference algorithm for the proposed model, and exhibit its superiority over the current state-of-the-art in several semi-supervised learning benchmarks.

Developing a text detection method which is invariant to scripts in natural scene images is a challenging task due to different geometrical structures of various scripts. Besides, multi-oriented of text lines in natural scene images make the problem more challenging. This paper proposes to explore ring radius transform (RRT) for text detection in multi-oriented and multi-script environments. The method finds component regions based on convex hull to generate radius matrices using RRT. It is a fact that RRT provides low radius values for the pixels that are near to edges, constant radius values for the pixels that represent stroke width, and high radius values that represent holes created in background and convex hull because of the regular structures of text components. We apply k-means clustering on the radius matrices to group such spatially coherent regions into individual clusters. Then the proposed method studies the radius values of such cluster components that are close to the centroid and far from the centroid to detect text components. Furthermore, we have developed a Bangla dataset (named as ISI-UM dataset) and propose a semi-automatic system for generating its ground truth for text detection of arbitrary orientations, which can be used by the researchers for text detection and recognition in the future. The ground truth will be released to public. Experimental results on our ISI-UM data and other standard datasets, namely, ICDAR 2013 scene, SVT and MSRA data, show that the proposed method outperforms the existing methods in terms of multi-lingual and multi-oriented text detection ability.