Search

In this paper, we introduce taxonomies for similarity and dissimilarity measures, respectively, based on their mathematical properties. Further, we propose a definition for rank equivalence of (dis)similarities regarding given data for prototype based methods. Starting with this definition we provide a measure to judge the degree of equivalence, which can be used to compare respective measures as well as to consider the influence of data preprocessing regarding a single (dis)similarity measure. In the last part of the paper an adaptive mixture approach of (dis)similarity measures for improved classification learning is presented.

Interval-valued data arises in situations where it is needed to manage either the uncertainty related to measurements, or the variability inherent to a group rather than an individual. This paper proposes a fuzzy c-means clustering algorithm based on suitable adaptive City-Block and Hausdorff distances with the purpose to cluster interval-valued data. This fuzzy c-means clustering algorithm optimizes explicitly an objective function by alternating three steps aiming to compute the fuzzy cluster representatives, the fuzzy partition, as well as relevance weights for the interval-valued variables. Indeed, most often conventional fuzzy c-means clustering algorithms consider that all variables are equally important for the clustering task. However, in real situations, some variables may be more or less important or even irrelevant for clustering. Due to the use of adaptive City-Block and Hausdorff distances, the paper proposes a fuzzy c-means clustering algorithm that tackles this problem with a step where a relevance weight is automatically learned for each interval-valued variable. Additionally, various tools for the fuzzy partition and fuzzy cluster interpretation of interval-valued data provided by this algorithm is also presented. Experiments with suitable synthetic and real interval-valued datasets demonstrate the robustness and the usefulness of this fuzzy c-means clustering algorithm and the merit of the fuzzy partition and fuzzy cluster interpretation tools.

Over the past years, digital image forensics has become a hot topic in multimedia security field. Among various image forensics technologies, image resampling detection is a standard detection tool. Furthermore, examining parameters of geometric transformations such as scaling factors or rotation angles is very useful for exploring the overall manipulation history of an image. In this paper, we propose a learning-to-rank approach for automatically estimating the scaling factor based on the normalized energy density features and moment features. Specifically, the difference of these features of the ordered image pairs are used for training, then the scaling factor of a new input image can be evaluated from its corresponding rank values. Our proposed method can not only effectively eliminate the long-known ambiguity between upscaling and downscaling in the analysis of resampling but also accurately estimate the factors of downscaling and weak scaling, i.e., the scaling factors near 1. Empirical experiments on extensive images with different scaling factors demonstrate the superiority of our proposed method when compared with the state-of-the-art methods. HighlightsThis paper proposes a learning-to-rank framework of image scaling factor estimation.Difference of two features of the ordered image pairs are used to train rank values.Scaling factor of a revised image can be evaluated according to the learned rank value.

This paper proposes a graph feature selection method for dementia diagnosis, by adding the information inherent in the observations into a sparse multi-task learning framework. Specifically, this paper first defines two relations (i.e., the feature-feature relation and the sample-sample relation, respectively) based on the prior knowledge in the data. The feature-feature selection enforces the similarity relationship between features to be preserved in the coefficient matrix while the sample-sample relation is designed to preserve the relation between samples invariant in the predicted space. Then we embed these two kinds of relations into a multi-task learning framework (i.e., a least square loss function plus an l2-norm regularization term) to conduct feature selection. Furthermore, we feed the reduced data into Support Vector Machine (SVM) for conducting the identification of Alzheimer's Disease (AD). Finally, the experimental results on a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset showed the effectiveness of the proposed method in terms of classification accuracy, by comparing with the state-of-the-art methods, including k Nearest Neighbor (kNN), ridge regression, SVM, and so on. Refereed/Peer-reviewed

The paper presents a persons' verification method based on the dynamic features of a given signature. In the proposed approach features space is connected with the set of similarity measures which can be used in the signature verification process. Features and linked similarity coefficients create a new composed signature features. Composed features are then reduced in the Hotelling reduction process, where the most appropriate features together with the most distinctive similarity measures are determined for each person. Finally, it leads to reduction of a composed signature features space. Compared to other approaches proposed solution automatically selects the best discriminatory features and similarity measures.The obtained verification results confirm that the proposed method, compared to other verification techniques, is very efficient. Proposed approach has been checked in different experiments, where various classifiers have been taken into consideration - perceptron and Bayesian network based classifiers as well as k-NN, random trees, random forests and others.In this paper we have included the simulation results for the two available datasets of dynamic signatures: SVC2004 and MCYT databases.

The paper Ran and Hu (2014, Neurocomputing) examines identifiability and parameter redundancy in classes of models used in machine learning. This note discusses the results on global identifiability and also clarifies that the paper's results on parameter redundancy already exist in the paper Cole et al. (2010, Mathematical Biosciences).

Clustering generic data, i.e., data not specific to a particular field, is a challenging problem due to their diverse complex structures in the original feature space. Traditional approaches address this problem by complementing clustering with feature learning methods, which either capture the intrinsic structure of the data or represent the data such that clusters are better revealed. In this paper, we propose an approach referred to as Extreme Learning Machine for Joint Embedding and Clustering (ELM-JEC), which incorporates desirable properties of both types of feature learning methods at the same time, specifically by (1) preserving the manifold structure of the data in the original space; (2) maximizing the class separability of the data in the embedded space. Since either type of method has improved clustering performance in some cases, our motivation is to integrate the two desirable properties to further improve the accuracy and robustness of clustering. Additional notable features of ELM-JEC are that it provides nonlinear feature mappings and achieves feature learning and clustering in the same formulation. The proposed approach can be implemented using alternating optimization, and its clustering performance compares favorably with several state-of-the-art methods on the real-world benchmark datasets. MOE (Min. of Education, S’pore)

Most proposed metaheuristics for feature selection and model parameter optimization are based on a two-termed L o s s + P e n a l t y function. Their main drawback is the need of a manual set of the parameter that balances between the loss and the penalty term. In this paper, a novel methodology referred as the GA-PARSIMONY and specifically designed to overcome this issue is evaluated in detail in thirteen public databases with five regression techniques. It is a GA-based meta-heuristic that splits the classic two-termed minimization functions by making two consecutive ranks of individuals. The first rank is based solely on the generalization error, while the second (named ReRank) is based on the complexity of the models, giving a special weight to the complexity entailed by large number of inputs. For each database, models with lowest testing RMSE and without statistical difference among them were referred as winner models. Within this group, the number of features selected was below 50%, which proves an optimal balance between error minimization and parsimony. Particularly, the most complex algorithms (MLP and SVR) were mostly selected in the group of winner models, while using around40–45% of the available attributes. The most basic IBk, ridge regression (LIN) and M5P were only classified as winner models in the simpler databases, but using less number of features in those cases (up to a 20–25% of the initial inputs).

The real-world networking data may contain different types of attribute views and relational view. Hence, it is desirable to collectively use available attribute views and relational view in order to build effective learning models. We call this framework multi-attribute and relational learning. Collective classification is one of the popular approaches that can handle both attribute and relational information for network data. However, in collective classification only one type of attribute and relational view is involved and little attention is received for multi-attribute and relational learning. In this paper, we propose a new semi-supervised collective classification approach, called hypergraph regularized generative model (HRGM), for multi-attribute and relational learning. In the approach, a generative model based on the Probabilistic Latent Semantic Analysis (PLSA) method is developed to leverage attribute information, and a hypergraph regularizer is incorporated to effectively exploit higher-order relational information among the data samples. Experimental results on various data sets have demonstrated the effectiveness of the proposed HRGM, and revealed that our approach outperforms existing collective classification methods and multi-view classification methods in terms of accuracy.

A primary challenge of time series classification is how to extract powerful features from training samples. Two kinds of classification methods, global-based and local-based methods, have been studied widely in recent years. The global-based methods, like 1-Nearest Neighbor(1-NN), take the entire series as features, which have the drawback that they are not able to indicate the intrinsic characters of a class. The local-based methods overcome this weakness by employing discriminative time series subsequences as features, called shapelets. However, most local-based methods are computationally expensive because of the massive number of shapelet candidates. In this paper, we propose a novel shapelets extraction method which takes each time series as a high-dimensional data and then finds the discriminative dimensions corresponding to the positions of shapelets. More specifically, the discriminative dimensions are determined by combining Local Fisher Discriminant Analysis (LFDA) method and two sparse restrictions which can encourage the continuous characteristic of time series. Extensive experimental results show that the proposed method achieves significant improvement compared to the existing shapelet-based methods in terms of classification accuracy and running time on the commonly used time series datasets. In addition, comparing with the accepted time series classification methods, NNDTW and COTE, our method still gets better results.

Extreme Learning Machine (ELM) has recently increased popularity and has been successfully applied to a wide range of applications. Variants using regularization are now a common practice in the state of the art in ELM field. The most commonly used regularization is the l2 norm, which improves generalization but result in a dense network. Regularization based on the elastic net has also been proposed but mainly applied to regression and binary classification problems. In this paper, we propose a generalization of regularized ELM (R-ELM) for multiclass classification problems, termed GR-ELM. We achieve such generalization using the l2,1 and Frobenius norm. Traditional R-ELM is a particular case of our method when binary classification tasks are considered. We also propose an alternative algorithm for GR-ELM when training data is distributed, namely DGR-ELM. We use Alternating Direction Method of Multipliers (ADMM) for solving the resulting optimization problems. Message Passing Interface (MPI) in a Single Program, Multiple Data (SPMD) programming style is chosen for implementing DGR-ELM. Extensive experiments are conducted to evaluate the proposed method. Our experiments show that GR-ELM and DGR-ELM have similar training and testing accuracy when compared to R-ELM, although usually faster testing time is obtained with our method due to the compactness of the resulting network.

Multi-view clustering, which explores complementary information between multiple feature sets by consensus grouping, has benefited many data analytic applications. The majority of previous multi-view clustering studies usually assume that all feature sets appear in complete. In real-world applications, however, it is often the case that some views could suffer from the missing of examples, resulting in incomplete feature sets. The incompleteness of views makes it difficult to synthesize all feature sets and achieve a comprehensive description of data samples. In this paper, we develop a novel incomplete multi-view clustering method, which projects all incomplete multi-view data to a complete and unified representation in a common subspace. Different from existing researches which exploit shallow learning to identify the common subspace, a deep incomplete multi-view clustering (DIMC) incorporating with the constraint of intrinsic geometric structure is proposed here to couple incomplete multi-view samples. To bridge the gap between each view and the common representation, the multi-view deep coupled networks are trained to map high-level semantic features. Besides, to preserve the local invariance for each view, an affinity graph based regularizer is constructed to encode geometrical information. Therefore, a new objective function is developed and the optimization processes are presented. Extensive experiments on several real-world datasets demonstrate that the proposed DIMC is superior to the state-of-the-art incomplete multi-view clustering methods.

Partial label learning (PLL) is a new weakly supervised learning framework that addresses the classification problems, where the true label of each training sample is concealed in a set of candidate labels. To learn from such weakly supervised training data, the key is to disambiguate the ambiguous labeling information. Because it is difficult to address by only focusing on the manipulation in the label space, manifold structure among training data in the feature space has gradually been exploited simultaneously to facilitate the disambiguation process by researchers in recent years. However, the manifold structure is commonly analyzed under an assumption that the samples close to each other in the feature space will share identical labels in the label space, which may be not correct in many real-world problems. In this paper, geometric mean metric learning approach is employed to learn a distance metric for PLL problems such that can maintain the aforementioned assumption correct in as many situations as possible. It is significantly more challenging than the conventional setup of distance metric learning because it is difficult to precisely identify whether a pair of training samples belong to the same class. We propose an alternative approach in which each training sample and its neighbor with shared candidate label are taken as a similarity pair, and each training sample and its neighbor without shared candidate label are taken as a dissimilarity pair. Considering that two samples with shared candidate label do not necessarily come from the same class, a weight is placed on each similarity pair. The experimental results on twenty four controlled UCI data sets and six real-world PLL problems show the proposed distance metric learning approach can be used as a front end of both the PLL algorithms exploiting the manifold structure among training data and other existing distance-based PLL algorithms to significantly improve their performance.

Recently, considerable efforts have been made in visual search reranking towards refining initial text-based image search results. In this paper, we propose a hypergraph based semi-supervised ranking method called HyperSSR for image search reranking. According to the basic visual consistency principle that visually similar images should have similar ranking scores, we introduce the hypergraph to capture the intrinsic geometrical structure of the data distribution. To build a robust hypergraph, a novel hypergraph construction approach is developed to incorporate relevance and pseudo relevance degree information from labeled and unlabeled samples, respectively. Based on the premise that a ranking model should work better with the prior pairwise preferences, we jointly incorporate the hypergraph regularizer and the prior pairwise preferences information into a unified ranking learning framework. Experimental results on MSRA-MM 1.0 dataset suggest our proposed approach produces superior performances compared with several state-of-the-art methods.

Canonical correlation analysis (CCA) is a useful dimensionality reduction method and has been widely used in projecting multi-view data. However, CCA learns from training data consisting of only target classes, ignoring Universum data that belongs to none of the target classes but comes from the same domain as the target classes. Recently, incorporating Universum data in learning has been used to gain more prior knowledge about the application domain and has been shown to achieve favorable improvements. In this paper, we extend CCA with Universum learning for multi-view data and the proposed method is termed as Universum CCA (UCCA). Due to the fact that Universum data in each view does not belong to any target class, correlation between Universum data and target data should be minimized. Consequently, UCCA aims to find basis vectors in multiple views to ensure that correlations between projections of target data are mutually maximized but correlations between projections of Universum data and target data mutually minimized. UCCA can be expressed as a generalized eigenvalue problem and the extracted features express patterns more distinctly. The experimental results on several real-world datasets demonstrate its marked improvements over conventional methods.

In object detection, object proposal methods have been widely used to generate candidate regions which may contain objects. Object proposal based on superpixel merging is one kind of object proposal methods, and the merging strategies of superpixels have been extensively explored. However, the ranking of generated candidate proposals still remains to be further studied. In this paper, we formulate the ranking of object proposals as a learning to rank problem, and propose a novel object proposals ranking method based on ListNet. In the proposed method, Selective Search, which is one of the state-of-the-art object proposal methods based on superpixel merging, is adopted to generate the candidate proposals. During the superpixel merging process, five discriminative objectness features are extracted from superpixel sets and the corresponding bounding boxes. Then, to weight each feature, a linear neural network is learned based on ListNet. Consequently, objectness scores can be computed for final candidate proposals ranking. Extensive experiments demonstrate the effectiveness and robustness of the proposed method.

Classification of brain voxels into gray matter, white matter, and cerebrospinal fluid (CSF) using magnetic resonance imaging (MRI) is pivotal for quantitative brain analyses. In spite of its computational effectiveness, the most commonly used statistical classification models are less capable of handling the intensity non-uniformity (INU) and partial volume effect (PVE), and hence may produce less accurate results. In this paper, we propose a novel approach, namely the VMG-NDE algorithm, to improve brain voxel classification in MRI images by considering all effects simultaneously. There are four planks in this algorithm, including (1) using variational mixture of Gaussians (VMG) model to characterize the variation of voxel values caused by PVE, (2) training a cohort of local VMG models on small data volumes extracted from the image to reduce the impact of INU, (3) employing the niche differential evolution (NDE) to infer each local VMG model, aiming to avoid falling into local optima, and (4) constructing a probabilistic brain atlas for each study and using it to incorporate the anatomy prior into the classification process. After training local VMG models, we classify each brain voxel using a linear combination of the predictions generated by all those models. This algorithm has been evaluated against the variational expectation-maximization based and genetic algorithm based segmentation algorithms and the segmentation routines in the widely used statistical parametric mapping (SPM) package, expectation-maximization segmentation (EMS) package and FSL package on both synthetic and clinical T1-weighted brain MRI studies. Our results suggest that the proposed algorithm can differentiate major brain tissue types more effectively and produce improved brain voxel classification accuracy.

With the rapid development of collection techniques, it is easy to gather various data which come from different domains, such as images, videos, documents, and etc, how to group these heterogeneous data becomes a research issue. Traditional techniques handle these clustering tasks separately, that is one task for one domain, so that they ignore the interactions among domains. In this paper, we present a co-transfer clustering method to deal with these separate tasks together with the aid of co-occurrence data which contain some instances represented in different domains. The proposed method consists of two steps, one is to learn the subspace of different domains that uncovers the latent common topics and respects the intrinsic geometric structure, the next is to simultaneously cluster the instances in all domains via the symmetric nonnegative matrix factorization method. A series of experiments on real-world data sets have shown the performance of the proposed method is better than the state-of-the-art methods.

This paper addresses an efficient scheme for clustering time-series through a novel regression mixture strategy (RMM) that simultaneously utilizes the benefits of the Markov random field (MRF) and mean template. Each component of the proposed RMM is a mixture of Student’s-t and non-symmetric Rayleigh distributions. The Student’s-t distribution has been validated to be effective in reducing the influence of outliers. Another advantage is it considers the property of time-series in which spatially adjacent voxels have higher probabilities when belonging to the same cluster; thus, this study seeks to impose the spatially interdependent constraints in the form of a prior through a Bayesian theorem. By introducing the non-symmetric Rayleigh distribution, the proposed method has flexibility to fit various types of observed time-series. To optimize the model parameters, the proposed algorithm is formulated as a maximum a posteriori (MAP) estimation problem. An expectation maximization (EM)-type approach is eventually executed to obtain the update equations. Additionally, an efficient energy function is applied in the proposed approach such that the EM-MAP algorithm can be directly applied to calculate the objective function, which makes the proposed approach easier to implement. To illustrate the suitability of the proposed approach, we have extensively compared clustering accuracy, curve square error, and intraclass index with those of existing similarity methods on synthetic spatiotemporal datasets, synthetic fMRI time-series, and real life fMRI time-series.

Zero-Shot Learning tries to predict the novel class samples that do not have any labeled instances in the training stage. This is typically achieved by exploring intermediate side information to transfer knowledge from seen classes to unseen testing ones. Different approaches vary in the usage of the side information and embedding methods. However, most methods only concern the relationships among different modalities but ignore to preserve the consistency among different samples in the same modality. In this paper, we propose an approach called Regularized Cross-Modality Ranking (ReCMR) to capture the semantic information from heterogeneous sources by taking both intra-modal and inter-modal semantics into consideration. Specifically, we employ the hinge ranking loss to exploit the structures among different modalities and devise efficient regularizers to constrain the variation of the samples in the identical modality. Experimental results on the popular AwA and CUB datasets show that ReCMR significantly outperforms the state-of-the-art methods.

Recent research has demonstrated that the performance of sparse representation based methods for single image super-resolution (SISR) reconstruction relies strongly on the degree of accuracy of sparse coding coefficients, and accordingly several more accurate models have been developed to overcome it by exploiting the nonlocal patch redundancy within the observed image. However, the capability of those models may be limited as they fail to simultaneously consider the redundant information both within the same scale and across multiple scales. Thus, in this paper, an improved SISR reconstruction method is proposed, in which a compensative pair of regularization terms defined by l1-norm is first constructed by taking advantage of the multiscale self-similarity. Then the calculated sparse coefficients are further aligned to this pair of standards in order to suppress sparse coding noise, and consequently result in more faithful recoveries. Finally, based on conventional iterative shrinkage-thresholding algorithm, a local-to-global and coarse-to-fine mathematic implementation is established to solve the proposed model effectively. Extensive experiments on both synthetic and real images demonstrate that our proposed method leads to a promising SISR performance and surpasses the recently published counterparts in terms of both objective evaluation and visual perception.

For robust visual tracking, appearance modeling should be able to well separate the object from its backgrounds, while accurately adapt to its appearance variations. However, most of the existing tracking methods mainly focus on one of the two aspects; or design two different modules to combine them with the price of double computational cost. In this paper, by using pairwise metric learning, we present a novel appearance model for robust visual tracking. Specifically, visual tracking is viewed as a pairwise regression problem, and extreme learning machine (ELM) is utilized to construct the pairwise regression framework. In ELM-based pairwise training, two constraints are enforced: the target observations must have different regression outputs from those background ones; while the various target observations during tracking should have approximate regression outputs. Thus, the discriminative and generative capabilities are fully considered in a single object tracking model. Moreover, online sequential ELM (OS-ELM) is used to update the resulting appearance model, thereby leading to a more robust tracking process. Extensive experimental evaluations on challenging video sequences demonstrate the effectiveness and efficiency of the proposed tracker.

Despite the continuous development of features and mid-level representations, effectively and reliably measuring the similarity among images remains a challenging problem in image retrieval tasks. Once traditional measures consider only pairwise analysis, context-sensitive measures capable of exploiting the intrinsic manifold structure became indispensable for improving the retrieval performance. In this scenario, diffusion processes and rank-based methods are the most representative approaches. This paper proposes a novel hybrid method, named rank diffusion, which uses a diffusion process based on ranking information. The proposed method consists in a diffusion-based re-ranking approach, which propagates contextual information through a diffusion process defined in terms of top-ranked objects, reducing the computational complexity of the proposed algorithm. Extensive experiments considering a rigorous experimental protocol were conducted on six public image datasets and several different descriptors. Experimental results and comparison with state-of-the-art methods demonstrate that high effectiveness gains can be obtained, despite the low-complexity of the algorithm proposed.

In this paper, we study the problem of supervised Fully PolSAR (polarimetric synthetic aperture radar) image classification. We estimate a complex Wishart model distribution for each class using training data, and we use such models to design a new classification procedure based on a diffusion-reaction equation. The method relies on simultaneously filtering and classifying pixels within the image. The diffusion term smooths the patches within the image, and the reaction term tends to move the pixel values towards the closest (in the sense of stochastic distances) representative class. We present a detailed study of the method accuracy using both simulated and true data, and we provide optimum parameters for its use. We show that the proposed method outperforms the results obtained using maximum likelihood and usual stochastic distance classification methods.

In multi-label learning, each example is represented by a single instance and associated with multiple class labels. Existing multi-label learning algorithms mainly exploit label correlations globally, by assuming that the label correlations are shared by all the examples. Moreover, these multi-label learning algorithms exploit the positive label correlations among different class labels. In practical applications, however, different examples may share different label correlations, and the labels are not only positive correlated, but also mutually exclusive with each other. In this paper, we propose a simple and effective Bayesian model for multi-label classification by exploiting Local positive and negative Pairwise Label Correlations, named LPLC. In the training stage, the positive and negative label correlations of each ground truth label for all the training examples are discovered. In the test stage, the k nearest neighbors and their corresponding positive and negative pairwise label correlations for each test example are first identified, then we make prediction through maximizing the posterior probability, which is estimated on the label distribution, the local positive and negative pairwise label correlations embodied in the k nearest neighbors. A comparative study with the state-of-the-art approaches manifests a competitive performance of our proposed method.

Dimension reduction methods always catch many attentions, because it could effectively solve the curse of dimensionality problem. In this paper, we propose an unsupervised feature selection method which could efficiently select a subset of informative features from unlabeled data. We integrate the low-rank constraint, hypergraph theory, and the self-representation property of features in a unified framework to conduct unsupervised feature selection. Specifically, we represent each feature by other features to conduct unsupervised feature selection via the feature-level self-representation property. We then embed a low-rank constraint to consider the relations among features. Moreover, a hypergarph regularizer is utilized to consider both the high-order relations and the local structure of the data. This enables the proposed model to take into account both the global structure of the data (via the low-rank constraint) and the local structure of the data (via the hypgergraph regularizer). We use an 2, p-norm regularizer to satisfy the constraints. Therefore, the proposed model is more robust to the previous models due to achieving better feature selection model. Experimental results on benchmark datasets showed that the proposed method effectively selected the most informative features by removing the adverse effect of irrelevant/redundant features, compared to the state-of-the-art methods.

Over the past decades, features generated by different models have been designed to describe various aspects of object. To connect the complementary information and represent the data properly, effective heterogeneous feature fusion methods are required. Multiple kernel learning (MKL) methods are widely adopted to learn the feature weights and to fuse features on score-level. In this paper, we exploit score distribution to address the feature fusion problem and propose a novel method named score-distribution MKL (SD-MKL) for image classification. Different from existing MKL methods, SD-MKL uses weights which are learned from score curves as a constraint on the weights of kernels. It contains two stages in off-line part: (1) independent data is used to construct reference curves according to classes and feature type; (2) samples and corresponding score-distribution weights are put into multi-kernel support vector machine (MKSVM) to learn feature weights. Our experimental results demonstrate the effect of exploiting score-distribution information on two datasets, which significantly benefits the performance of image classification.

A dynamic single-element-based Tikhonov graph regularization NMF and CF integrating manifold regularization were proposed.The proposed methods alleviate the curse of dimensionality of dual GWNMF.The proposed methods evade the frequent manipulations of indicator matrices.The proposed methods show better recommendation on RMSE, MAE, recall, and precision. Matrix factorization (MF) is an increasingly important approach in the field of missing value prediction because recommender systems are rapidly becoming ubiquitous. MF-based collaborative filtering (CF) seeks to improve recommender performance by combining useritem matrix with MF. However, most MF-based approaches available at present could not obtain high prediction accuracy because of the sparse availability of useritem matrices in CF models. The present paper proposes a framework that involves two efficient MF, dynamic single-element-based CF-integrating manifold regularization (DSMMF) and dynamic single-element-based Tikhonov graph regularization non-negative MF (DSTNMF). The aim of this framework is to better use the intrinsic structure of useritem rating matrix and user/item content information, overcome the dimensionality curse and ill-posed problem of weighted graph NMF, and evade the frequent manipulations of indicator matrices that lack practicability. We validate the effectiveness of our proposed algorithms with respect to recommender performance by four indices on three datasets. We demonstrate that our proposed approaches lead to considerable improvement compared with several other state-of-the-art approaches.

View-based 3D object retrieval techniques have become increasingly important in various fields, and lots of ingenious studies have promoted the development of retrieval performance from different aspects. In this paper, we focus on the 2D projective views that represent the 3D objects and propose a boosting approach by evaluating the discriminative ability of each objects views. The dissimilarity between views semantic and discriminative ability is firstly investigated through classification performance. We then propose a simple, robust and effective measurement to study the views discriminative ability. By employing the proposed reverse distance metric, we utilize the discriminative information for many to many view set matching. The proposed algorithm is then employed with various features to boost the multi-model graph learning based retrieval method. We compare our approach with several state of the art methods on ETH-80 dataset and National Taiwan University 3D model dataset. The results demonstrate the effectiveness of our method and its excellent boosting performance.

This paper proposes a niching evolutionary algorithm with adaptive negative correlation learning, denoted as NEA_ANCL, for training the neural network ensemble. In the proposed NEA_ANCL, an adaptive negative correlation learning, in which the penalty coefficient is set to dynamically change during training, has been developed. The adaptation strategy is based on a novel population diversity measure with the purpose of appropriately controlling the trade-off between the diversity and accuracy in the ensemble. Further, a modified dynamical fitness sharing method is applied to preserve the diversity of population during training. The proposed NEA_ANCL has been evaluated on a number of benchmark problems and compared with related ensemble learning algorithms. The results show that our method can be used to design a satisfactory NN ensemble and outperform related works.

Eye movement data collection is very expensive and laborious. Moreover, there are usually missing values. Assuming that we are collecting eye movement data from a set of images viewed by different users, there is a possibility that we will not able to collect the data of every user from every imageone or more views may not be represented in the image. We assume that the relationships among the views can be learnt from the whole collection of views (or items). The task is then to reproduce the missing part of the incomplete items from the relationships derived from the complete items and the known part of these items. Using certain properties of tensor algebra, we showed that this problem can be formulated consistently as a regression type learning task. Furthermore, there is a maximum margin based optimisation framework in which this problem can be solved in a tractable way. This problem is similar to learning to predict where a person is looking in an image. Therefore, we proposed an algorithm called Tensor-based Multi-View Learning(TMVL) in this paper. Furthermore, we also proposed a technique for improving prediction by introducing a new feature set obtained from Kronecker decomposition of the image fused with users eye movement data. Using this new feature can improve prediction performance markedly. The proposed approach was proven to be more effective than two well-known saliency detection techniques.

In usual real-world clustering problems, the set of features extracted from the data has two problems which prevent the methods from accurate clustering. First, the features extracted from the samples provide poor information for clustering purpose. Second, the feature vector usually has a high-dimensional multi-source nature, which results in a complex cluster structure in the feature space. In this paper, we propose to use a combination of multi-task clustering and fuzzy co-clustering techniques, to overcome these two problems. In addition, the Bregman divergence is used as the concept of dissimilarity in the proposed algorithm, in order to create a general framework which enables us to use any kind of Bregman distance function, which is consistent with the data distribution and the structure of the clusters. The experimental results indicate that the proposed algorithm can overcome the two mentioned problems, and manages the complexity and weakness of the features, which results in appropriate clustering performances.

We proposed a robust unsupervised method to remove redundant and irrelevant features.Both the cluster centers and the sparse representation are predicted.The feature selection and clustering are performed simultaneously.An efficient iterative update algorithm based on ADMM is proposed.Superiority over seven existing methods is established for five data sets. Dimensionality reduction is a challenging task for high-dimensional data processing in machine learning and data mining. It can help to reduce computation time, save storage space and improve the performance of learning algorithms. As an effective dimension reduction technique, unsupervised feature selection aims at finding a subset of features to retain the most relevant information. In this paper, we propose a novel unsupervised feature selection method, called Robust Unsupervised Feature Selection via Matrix Factorization (RUFSM), in which robust discriminative feature selection and robust clustering are performed simultaneously under l2, 1-norm while the local manifold structures of data are preserved. The advantages of this work are three-fold. Firstly, both the latent orthogonal cluster centers and the sparse representation of the projected data points based on matrix factorization are predicted for selecting robust discriminative features. Secondly, the feature selection and the clustering are performed simultaneously to guarantee an overall optimum. Thirdly, an efficient iterative update algorithm, which is based on Alternating Direction Method of Multipliers (ADMM), is used for RUFSM optimization. Compared with several state-of-the-art unsupervised feature selection methods, the proposed algorithm comes with better clustering performance for almost all datasets we have experimented with here.

As an issue that attracts increasing interests in both academia and industry, multiple-shot person re-identification has shown promising results but suffers from real-scenario complexities and feature-crafting heuristics. To tackle the problems of set-level data variation and sparseness during re-identification, this paper proposes a novel metric learning method, named Fair Set-Collaboration Metric Learning, motivated by utilizing the opportunities whilst overcoming the challenges from the set of multiple instances. This method optimizes a new set-collaboration dissimilarity measure, which introduces the fairness principle into the collaborative representation based set to sets distance, in the set based metric learning framework. Experiments on widely-used benchmark datasets have demonstrated the advantages of this method in terms of effectiveness and robustness.

Amortized variational inference, whereby the inferred latent variable posterior distributions are parameterized by means of neural network functions, has invigorated a new wave of innovation in the field of generative latent variable modeling, giving rise to the family of deep generative models (DGMs). Existing DGM formulations are based on the assumption of a symmetric Gaussian posterior over the model latent variables. This assumption, although mathematically convenient, can be well-expected to undermine the eventually obtained representation power, as it imposes apparent expressiveness limitations. Indeed, it has been recently shown that even some moderate increase in the latent variable posterior expressiveness, obtained by introducing an additional level of dependencies upon auxiliary (Gaussian) latent variables, can result in significant performance improvements in the context of semi-supervised learning tasks. Inspired from these advances, in this paper we examine whether a more potent increase in the expressiveness and representation power of modern DGMs can be achieved by completely relaxing their typical symmetric (Gaussian) latent variable posterior assumptions: Specifically, we consider DGMs with asymmetric posteriors, formulated as restricted multivariate skew-Normal (rMSN) distributions. We derive an efficient amortized variational inference algorithm for the proposed model, and exhibit its superiority over the current state-of-the-art in several semi-supervised learning benchmarks.

Developing a text detection method which is invariant to scripts in natural scene images is a challenging task due to different geometrical structures of various scripts. Besides, multi-oriented of text lines in natural scene images make the problem more challenging. This paper proposes to explore ring radius transform (RRT) for text detection in multi-oriented and multi-script environments. The method finds component regions based on convex hull to generate radius matrices using RRT. It is a fact that RRT provides low radius values for the pixels that are near to edges, constant radius values for the pixels that represent stroke width, and high radius values that represent holes created in background and convex hull because of the regular structures of text components. We apply k-means clustering on the radius matrices to group such spatially coherent regions into individual clusters. Then the proposed method studies the radius values of such cluster components that are close to the centroid and far from the centroid to detect text components. Furthermore, we have developed a Bangla dataset (named as ISI-UM dataset) and propose a semi-automatic system for generating its ground truth for text detection of arbitrary orientations, which can be used by the researchers for text detection and recognition in the future. The ground truth will be released to public. Experimental results on our ISI-UM data and other standard datasets, namely, ICDAR 2013 scene, SVT and MSRA data, show that the proposed method outperforms the existing methods in terms of multi-lingual and multi-oriented text detection ability.

This paper proposes an enhanced fuzzy minmax neural (EFMN) network model with ant colony optimization based-rule-extractor for grouping of the data-patterns and decision making by rule-list. There are many methods to extract the rules which are having less accuracy and with less performance. The earlier methods have drawbacks like they have not maintained the rule accuracy in terms of consistency. The proposed method has improved the accuracy, consistency and performance against existing methods. The number of rules obtained from this system is less in count and having higher rank. One of the strength of this method is that the rules obtained from the system are comprehensible, as they are in rule list format. This rule list is useful in decision making problem.

In the field of data clustering, finding the cluster numbers automatically and generating reliable clusters for a given dataset are fundamental but challenging tasks. Recently, a clustering analysis algorithm for the automatic identification of cluster numbers was presented, and it achieves accurate clustering results for those datasets with complex structures. Unfortunately, this algorithm utilizes a hard partition approach in the process of integration and does not make full use of the membership information from each fuzzy c-means (FCM) clustering result. Thus, this scheme has to integrate many more FCM clustering results and also requires many iterations during the process of iterative graph partitioning. To address this problem, an automatic fuzzy clustering algorithm is proposed in this paper, combining the soft partition method with the membership information from each FCM clustering result. Finally, extensive experiments are performed, and under the premise of obtaining accurate clustering results simultaneously, the proposed algorithm can effectively decrease the number of FCM clustering results in the process of integration compared with the original algorithm. Furthermore, the number of iterations of the proposed scheme in the iterative graph partitioning process is half that of the original approach.

As a fundamental preprocessing of various multimedia applications, object proposal aims to detect the candidate windows possibly containing arbitrary objects in images with two typical strategies, window scoring and grouping. In this paper, we first analyze the feasibility of improving object proposal performance by integrating window scoring and grouping strategies. Then, we propose a novel object proposal method for RGB-D images, named elastic edge boxes. The initial bounding boxes of candidate object regions are efficiently generated by edge boxes, and further adjusted by grouping the super-pixels within elastic range to obtain more accurate candidate windows. To validate the proposed method, we construct the largest RGB-D image data set NJU1800 for object proposal with balanced object number distribution. The experimental results show that our method can effectively and efficiently generate the candidate windows of object regions and it outperforms the state-of-the-art methods considering both accuracy and efficiency.

Data reduction plays a very important role in the data mining field, but the existing methods have not been able to efficiently identify all major features which are hidden in the large datasets. On some occasions, they even cause the loss of the original key features. In this paper, a new efficient measure was developed to reduce a given dataset and to uncover the major features by multiplying the defined absolute density with the defined local density of any data. These two kinds of densities were estimated with a fast grid-based bisecting method. To test its performance on feature reduction and sample reduction, a group of feature-different synthetic datasets and 24 benchmark datasets were used as examples and the clustering accuracy, runtime and separability among clusters were used as measurements. The results strongly proved the proposed method could fast reduce a dataset and identify the most important key features. Additionally, it also can effectively determine the optimal number of clusters by suppressing the noisy data and enhancing the separation among clusters.

A novel model fitting approach via Structure Decision Graph (SDG) is proposed.SDG is constructed based on the weight score and minimum arrived distance.P-distance is developed to measure similarity via continuous consensus set.SDG is less disturbed by noises and outliers and easy to implement.Experiments show the superiority of SDG for multiple-structure segmentation. We propose a novel and effective robust model fitting approach based on the Structure Decision Graph (SDG) to segment multiple-structure data in the presence of outliers. The proposed approach is motivated by the observations that each structure can be characterized by one representative hypothesis, called as the Structure Prototype (SP), and the SPs have relatively large distances among them. In this paper, instead of analyzing each hypothesis individually, the residuals over all the hypotheses are used to explicitly construct an SDG, where a sorted weight score set and a minimum arrived distance set are respectively computed. Based on the SDG, the SPs corresponding to different structures can be easily determined. Compared with conventional robust model fitting approaches, one distinguishing characteristic of our approach is that the clustering procedure is not required. Therefore, the proposed approach is less disturbed by noises and outliers, and is relatively easy to implement. Experimental results on synthetic data and real-world image datasets demonstrate the superiority of the proposed approach over the state-of-the-art robust model fitting approaches for multiple-structure data segmentation.

In this paper, we propose multiple kernel learning (MKL) based on three discriminant features to learn an efficient P300 classifier to improve the accuracy of character recognition in a P300 speller BCI. Three discriminant features are specified in raw samples and two morphological features, which can complement the MKL of a P300 classification. A linear kernel is established for each discriminant feature. A kernel weight differentiates the linear kernel to both explore complementary information among the three discriminant features and weigh a contribution of each discriminant feature for the MKL. Here, the 1 norm regularization of the kernel weight ultimately enforces an optimal discriminant feature set of the MKL of a P300 classification.The performance of the proposed method is then evaluated according to the size of the three discriminant feature sets that are generated from dataset II of BCI competition III. Compared to an existing SVM-based classification method, the proposed method consistently obtains better or similar accuracy in terms of character recognition, with a different execution time for the variable size of the three discriminant feature sets. Furthermore, the kernel weight of the raw samples was found to consistently be more dominant than the kernel weight of the two morphological features on the variable size of the three discriminant feature sets. This finding means that the two morphological features supplement the lack of the raw samples for the MKL of a P300 classification. We ultimately could conclude that the proposed method is sufficiently robust to improve the accuracy of character recognition with a different time for the variable size of the three discriminant feature sets in a P300 speller BCI.

In the past decades, we have witnessed a surge of interests of learning distance metrics for various image processing tasks. However, facing with features from multiple views, most metric learning methods fail to integrate compatible and complementary information from multi-view features to train a common distance metric. Most information is thrown away by those single-view methods, which affects their performances severely. Therefore, how to fully exploit information from multiple views to construct an optimal distance metric is of vital importance but challenging. To address this issue, this paper constructs a multi-view metric learning method which utilizes KL-divergences to integrate features from multiple views. Minimizing KL-divergence between features from different views can lead to the consistency of multiple views, which enables MML to exploit information from multiple views. Various experiments on several benchmark multi-view datasets have verified the excellent performance of this novel method.

Policy making, strategic planning and management in general are complex decision making tasks, where the formulation of a quantitative mathematical model may be difficult or impossible due to lack of numerical data and dependence on imprecise verbal expressions. For such systems, knowledge representation graphs and cognitive maps are most familiar and often used for modelling complexity and aiding decision making. Fuzzy Cognitive Maps (FCM), as graph-based cognitive models, have been successfully used for knowledge representation and reasoning. In modelling complex systems usually a large number of concepts need to be considered. However, it is often difficult in real applications to find the appropriate number of concepts. Using only a few concepts is not enough to represent the modelled system with the required precision, and increasing the number of concepts increases the complexity of the model quadratically; it is burdensome to work with for the experts. The contribution of this paper is two-fold: (i) to propose a new concept reduction approach for FCM and (ii) to apply it on developing less complex FCM for management and decision making. The behaviour of reduced models is analysed through a number of scenarios with respect to the original complex system. The main idea of the reduction is a clustering based on fuzzy tolerance relations. The new approach is focused on reducing complexity in the modelling process, which provides a more transparent and easy to use model for policy makers. The applicability of the proposed method is demonstrated via literature examples and a solid waste management case study that initiated this research. The results clearly show the advantageous characteristics of the proposed concept reduction method for FCM and its aid in policy making.

Selecting the relevant factors in a particular domain is of utmost interest in the machine learning community. This paper concerns the feature selection process for twin support vector machine (TWSVM), a powerful classification method that constructs two nonparallel hyperplanes in order to define a classification rule. Besides the Euclidean norm, our proposal includes a second regularizer that aims at eliminating variables in both twin hyperplanes in a synchronized fashion. The baseline classifier is a twin SVM implementation based on second-order cone programming, which confers robustness to the approach and leads to potentially better predictive performance compared to the standard TWSVM formulation. The proposal is studied empirically and compared with well-known feature selection methods using microarray datasets, on which it succeeds at finding low-dimensional solutions with highest average performance among all the other methods studied in this work. HighlightsNovel feature selection approach for twin SVM based on second-order cone programming.Extension of the Group Lasso penalty for coordinated variable selection.A single optimization problem is used to solve both twin subproblems.Superior performance is achieved in experiments on high-dimensional datasets.

In this study, a new static analysis approach is proposed for enhanced Fuzzy Cognitive Maps (FCMs), which have non-singleton fuzzy numbers in casual relation strength representation. Cognitive Maps (CMs) are proposed as a type of directed graph that offers a means to model interrelationships or causalities among concepts, and have a clear way to visually represent them. They graphically describe a system in terms of concepts, and causal beliefs, and are powerful graphical tools to represent knowledge of the experts. Fuzzy cognitive maps, which are weighted cognitive maps, are proposed also as graphical modelling technique that follows a reasoning approach similar to processes of human reasoning and human decision-making. In FCMs, the casual relations and its strengths are assigned in a unit interval with a sign. The assigned casual strengths in conventional FCMs are singleton fuzzy (crisp) numbers, and only allow to interpret the effects linguistically but do not represent the uncertainty or ambiguity in causality. In this paper, a new analysis is presented for finding the indirect effects and total effects between the concepts of enhanced FCMs that are represented with non-singleton fuzzy numbers, especially for triangular or trapezoidal fuzzy numbers. Firstly, the mathematical approach about fuzzy numbers and the proposed analysis is presented, then secondly an experimental study on modelling ERP maintenance risks via FCM is presented. The results of the proposed causal effect analysis are discussed for this model and the outcomes are compared with a conventional FCM model where the casual strengths are singleton fuzzy numbers. The results of the experiment show the benefit of using triangular fuzzy numbers when a group of experts are involved in modelling. The uncertainty and varieties between the experts’ knowledge are easily captured and the casual effect between the concepts are successfully shown with the presented static analysis.

In this paper, an improved electrocardiogram (ECG) beats classification system is proposed, which is based on Fuzzy C-Means (FCM) clustering algorithm. The classification of ECG beats is necessary in order to diagnose the type of arrhythmia (e.g., Atrial Premature Contraction (APC), Premature Ventricular Contraction (PVC), Right Bundle Branch Block (RBBB) etc.) present in the ECG records. The efficiency of any classification model highly depends on the “most relevant” set of features used. The primary goal of this study is to classify different arrhythmic beats with reduced set of relevant-only ECG attributes. The attribute selection model is based on Mahalanobis-Taguchi System (MTS); a multi-dimensional pattern recognition tool, which can dynamically choose the important set of ECG features. The number of most relevant features can vary from person to person according to the type of arrhythmia present in the respective ECG signals. A traditional Euclidian Distance (ED) based FCM can detect the spherical clusters but it may lead to improper clustering in some cases. As a solution to this problem, Mahalanobis Distance (MD) is used in the proposed model in order to improve the distance measurement procedure. In our proposed system, MD based improved Fuzzy C-Means (FCM-M) clustering is used to classify the arrhythmic beats. Experimental results show that the performance of FCM-M is significantly better than the conventional FCM for arrhythmia classification. Another direction of our proposed research is to use the concept of initial cluster centroid in order to reduce the number of program iterations. In our experiments, the number of program iterations is reduced to an average of 53% when initial centroid is assigned to FCM-M with the same classification results.

Clustering ensemble has emerged as an important tool for data analysis, by which a more robust and accurate consensus clustering can be generated. On forming the ensembles, empirical studies have suggested that better ensembles can be obtained by simultaneously considering the quality of the ensembles and the diversity among ensemble members. However, little research efforts have been paid to incorporate prior background knowledge. In this paper, we first provide a theoretical analysis on the effect of the diversity and quality of the ensemble members. We then propose a unified framework to solve constraint-based clustering ensemble selection problem, where some instance level must-link and cannot-link constraints are given as prior knowledge or background information. We formalize this problem as a combinatorial optimization problem in terms of the consistency under the constraints, the diversity among ensemble members, and the overall quality of ensembles. Our proposed framework brings together two distinct yet interrelated themes from clustering: ensemble clustering and semi-supervised clustering. We study different techniques for searching high-quality solutions. Experiments on benchmark datasets demonstrate the effectiveness of our framework.

Motivation in the development of granular neural network (GNN) with the principle of granular computing (GrC) is to obtain possible degree of transparency in the network architecture and its operational steps, which are bottleneck for conventional neural network (NN). In addition, GNN provides improved performance and at the same time poses less computational burden compared to conventional NN and fuzzy NN (FNN). Topologically, the nodes of different layers of FNNs are fully connected whereas it is partial in case of GNNs. Further, the architectures of GNN is determined based on the extracted rules from the domain information. However, selection of relevant rules for GNN is a tedious task. To mitigate this, the present paper aims to develop a pattern classification model in the framework of unification of GNNs and tries to avoid the searching of optimum number of rules and the best combination of rules. The unified model thus works with a set of different rules-based GNNs and derives the final decision from the individual GNN. Each of these GNNs takes the class-supportive fuzzy granulated features of the input in order to preserve the feature-wise belonging information to different classes. These granulated features provide improved class discriminatory information for the classification of data sets with ill-defined and overlapping class boundaries. The proposed model thus explores mutually the advantages of class-supportive fuzzy granulation of features, informative rules-based GNNs and unification of GNNs. Superiority of the model to similar other methods are justified in terms of various performance measurement indexes using different benchmark data sets including remote sensing imagery.

Image enhancement is a very significant issue in image processing and analysis. In practice, many images (e.g.images captured from X-ray systems) are of low quality, such a slow-luminance and low-contrast, which must be enhanced before further processing. Fuzzy set theory is a useful tool for handling the ambiguity or uncertainty. Many researchers use the maximum Shannon entropy and fuzzy complement for image enhancement. But these methods are easy to be over-enhanced or under-enhanced or time-consuming. In this paper, a flexible method is proposed, which utilizes the maximum fuzzy Sure entropy, fuzzy c-partition and fuzzy complement (MSRM). Furthermore, a positive threshold value selection algorithm is developed to tune the enhancement performance of the proposed method. A variety of highly degraded images have been experimented by the proposed method. The comparisons of those experimental results show that the performance of our method overwhelms those of the existing ones.