Search

A small dataset often makes it difficult to build a reliable learning model, and thus some researchers have proposed virtual sample generation (VSG) methods to add artificial samples into small datasets to extend the data size. However, for some datasets the assumption of the distribution of data in the VSG methods may be vague, and when data only has a few attributes, such approaches may not work effectively. Other researchers thus proposed attribute extension methods to generate attributes to convert data into a higher dimensional space. Unfortunately, the resulting dataset may become a sparse dataset with many null or zero values in extended attributes, and then a large quantity of such attributes will reduce the representativeness of instances for the learning model. Therefore, based on fuzzy theories, this paper proposes a novel sample attribute extending (SEA) method to extend a suitable quantity of attributes to improve small dataset learning. In order to verify the validity of the SEA method, using SVR and BPNN, this paper adopts two real cases and two public datasets to conduct the learning of the predictive model, and uses the paired t-test to statistically examine the significance of improvement. The experimental results show that the proposed SEA method can effectively improve the learning accuracy of small datasets.

Neural Networks and Support Vector Machines (SVMs) are two of the most popular and efficient supervised classification models. However, in the context of large datasets many complexity issues arise due to high memory requirements and high computational cost. In the context of the application of Data Mining algorithms, data reduction techniques attempt to reduce the size of training datasets in terms of the number of instances by selecting some of the existing instances or by generating new training instances. The idea is to speed up the application of the data mining algorithm with minimum or no sacrifice in performance. Data reduction techniques have been extensively used in the context of k-Nearest Neighbor classification, a lazy classifier that works by directly using a training dataset rather than building a model. This paper explores the application of data reduction techniques as a preprocessing step before the training step of Neural Networks and SVMs. Furthermore, the paper proposes a new data reduction technique that is based on k-median clustering algorithm. Our experimental results illustrate that, in the case of SVMs, data reduction techniques can effectively reduce the dataset size incurring small performance degradation. In the case of Neural Networks, the performance loss is somewhat greater, for the same data reduction rate, but both SVM and Neural Network models outperform the k-NN approach that is typically used in Data Mining applications.

This paper investigates an adaptive neural network prescribed performance synchronization scheme for unknown complex dynamic networks with different dimensions. Based on predefined performance bounded and Lyapunov stability theory, adaptive neural networks controllers are designed to ensure that synchronization errors remain in a neighborhood of origin with the prescribed bounds. In addition, the paper analyses in detail that the synchronization behaviors between drive network selected as the three-dimension chaotic system and response network selected as the four-dimension hyperchaotic chaotic system. The numerical simulation results are presented to show the validity of the proposed approach.

An important feature characteristic of the data streams in many of today's big data applications is the intrinsic uncertainty, which could happen for both item occurrence and attribute value. While this has already posed great challenges for fundamental data mining tasks such as classification, things are made even more complicated by the fact that completely-labeled examples are usually unavailable in such settings, leaving researchers the only option to learn classifiers on partially-labeled examples on uncertain data streams. Furthermore, there will be concept drift on evolving data streams. To address these challenges, this paper therefore focuses on the study of learning from positive and unlabeled examples (PU) on uncertain data streams. To the best of our knowledge, this paper is the first work to address the uncertainty issue in both item occurrence (occurrence level) and attribute value (attribute level) for the problem of PU learning over streaming data. Firstly, we propose an algorithm to classify positive and unlabeled examples with both uncertainties (PUU). The algorithm extracts reliable positive and negative examples by clustering-based method, and then trains the classifier with Weighted Extreme Learning Machine (Weighted ELM). Secondly, we propose an algorithm of PU learning over uncertain data streams (PUUS). It adopts ensemble model and trains base classification by PUU. In order to detect concept drift, PUUS uses cluster set similarities between the current data block and history data block. We propose different update strategies for different concept drift to adapt to evolving uncertain data streams. Experimental results show that PUUS can effectively classify uncertain data streams with just positive and unlabeled data, while achieving in the meantime good performance in detecting and handling concept drifts.

An instance is often represented from different aspects (views or modalities), which leads to high-dimensional features and even multiple labels. In this paper, we focus on the feature selection problem in multi-label classification, for which a trivial solution is handling the labels dividedly. Obviously, such a scheme may not work well by leaving the label relationship out of consideration. Recently, several research works conduct feature selection directly under a multi-label framework by implicitly or explicitly modeling label relationship. However, these works assume that all labels share the same feature subset or subspace, which is not reasonable enough for some scenarios since different labels tend to convey different semantics. To address this problem, we develop a novel approach in this paper to select label-dependent features for multi-label classification. Specifically, we (1) formulate a convex model based on a more general and practical assumption that different labels convey different semantics with specific features; (2) design an alternating optimization algorithm based on Nesterov's method and L1-ball projection for efficiently finding the optimal solution, which can realize multi-label classification, feature selection, and label relationship estimation simultaneously. Finally, experiments on publicly available datasets show that the proposed algorithm achieves better performance than several related methods.

Personal recommendation technology is becoming a useful and popular solution to solve the problem of information overload with the popularity of big data cloud services. But most recommendation algorithms pay too much attention to the similarity to focus on the social trust between users. So this paper focus on the research of hybrid Recommendation algorithm for big data based on the optimization combining with the similarity and trust in sociology. In this paper, we introduced some user trust models including trust path model and loop trust model, and then we took these models into the calculation of mixed weighting. The experiment results show that the recommendation algorithm considering the trust models has the higher accuracy than the traditional recommendation algorithm, and we have a 2% increase in both MEA (Mean Absolute Error) and RMSE (Root Mean Square Error).

The Internet of Things (IoT) refers to an infrastructure that integrates things over standard wired/wireless networks and allows them to exchange information with each other. The IoT is a very complex heterogeneous network, enabling seamless integration of these things is a huge challenge. A publish/subscribe method of integration can be formulated to solve the problems of interconnecting billions of heterogeneous things. In our work, an IoT framework that uses an abstraction layer that decouples an application from the service calls and network interfaces is required to send and receive messages on a particular thing. This paper provides definitions and classifications for heterogeneous data/events/services according to the properties of the things in order to integrate them into a framework for description. Based on these definitions and classifications, heterogeneous data/events/services in the IoT were integrated via topic description through the Data Distribution Service (DDS) middleware standard for real-time publish/subscribe. This paper also concludes with general remarks and a discussion of future work.

This paper presents a method for determining how close a neural network (and its design) is to the systems it is intended to represent and model. The output of this method is a normalized numerical value (an index) which can be compared to other indices. It can also be used to compare one artificial neural network with another. The objective is to develop a numerical index of quality that would make it possible to compare a variety of methods and computational techniques. A brief revisit to the concept from which this paper originated is presented in section 2 (where authors summarize the Statistical Index of Quality), and the outline of the Neural Network Index of Quality is shown in subsection 3.2. Subsection 3.3 illustrates sample calculations and comparisons with other similar methods for neural network evaluation. Finally, section 4 addresses certain implications and outlines future research in pursuit of automated quality analysis of computational techniques. As proposed, the method is effective to compare different computational techniques adequately, with results close to ‘1’ for well-designed networks and results close to ‘0’ for neural networks with possible design flaws. The proposed method is designed for fixed-size, supervised artifical neural networks.

In this paper, a self-organizing map (SOM) neural network is used to visualize corrective actions of failure modes and effects analysis (FMEA). SOM is a popular unsupervised neural network model that aims to produce a low-dimensional map (typically a two-dimensional map) for visualizing high-dimensional data. With regards to FMEA, it is a popular methodology to identify potential failure modes for a product or a process, to assess the risk associated with those failure modes, also, to identify and carry out corrective actions to address the most serious concerns. Despite the popularity of FMEA in a wide range of industries, two well-known shortcomings are the complexity of the FMEA worksheet and its intricacy of use. To the best of our knowledge, the use of computation techniques for solving the aforementioned shortcomings is limited. The use of SOM in FMEA is new. In this paper, corrective actions in FMEA are described in their severity, occurrence and detect scores. SOM is then used as a visualization aid for FMEA users to see the relationship among corrective actions via a map. Color information from the SOM map is then included to the FMEA worksheet for better visualization. In addition, a Risk Priority Number Interval is used to allow corrective actions to be evaluated and ordered in groups. Such approach provides a quick and easily understandable framework to elucidate important information from a complex FMEA worksheet; therefore facilitating the decision-making tasks by FMEA users. The significance of this study is two-fold, viz., the use of SOM as an effective neural network learning paradigm to facilitate FMEA implementations, and the use of a computational visualization approach to tackle the two well-known shortcomings of FMEA.

The paper investigates application of several methods of feature selection to identification of the most important genes in autism disorder. The study is based on the expression microarray of genes. The applied methods analyze the importance of genes on the basis of different principles of selection. The most important step is to fuse the results of these selections into common set of genes, which are the best associated with autism. These genes may be treated as the biomarkers of this disorder and used in early prediction of autism. The paper proposes and compares three different methods of such fusion: purity of the clusterization space, application of genetic algorithm and random forest in the role of integrator. The numerical experiments are concerned with the identification of the most important biomarkers and their application in autism recognition. They show the applied fusion strategy of many independent selection methods leads to the significant improvement of the autism recognition rate.

The performance of many machine learning algorithms depends crucially on the hyperparameter settings, especially in Deep Learning. Manually tuning the hyperparameters is laborious and time consuming. To address this issue, Bayesian optimization (BO) methods and their extensions have been proposed to optimize the hyperparameters automatically. However, they still suffer from highly computational expense when applying to deep generative models (DGMs) due to their strategy of the black-box function optimization. This paper provides a new hyperparameter optimization procedure at the pre-training phase of the DGMs, where we avoid combining all layers as one black-box function by taking advantage of the layer-by-layer learning strategy. Following this procedure, we are able to optimize multiple hyperparameters in an adaptive way by using Gaussian process. In contrast to the traditional BO methods, which mainly focus on the supervised models, the pre-training procedure is unsupervised where there is no validation error can be used. To alleviate this problem, this paper proposes a new holdout loss, the free energy gap, which takes into account both factors of the model fitting and over-fitting. The empirical evaluations demonstrate that our method not only speeds up the process of hyperparameter optimization, but also improves the performances of DGMs significantly in both the supervised and unsupervised learning tasks.

To compact the architecture of extreme learning machine (ELM), two incremental learning algorithms are proposed in this paper. The previous incremental learning algorithms for ELM recruit hidden nodes randomly, which is equivalent to implementing a random selection from a candidate set of infinite size. Hence, it is impossible to recruit good hidden nodes, and thus it usually requires more hidden nodes than traditional neural networks to achieve matched performance. To improve the quality of the hidden nodes recruited, an incremental learning algorithm for ELM is presented based on Gram--Schmidt process (GSI-ELM), which recruits the best hidden node from a random subset of fixed size via defining an evaluating criterion at each learning step. However, the “nesting effect” exists in the GSI-ELM, that is to say, the hidden nodes once recruited by GSI-ELM can not be later discarded. To treat this “nesting problem”, the improved GSI-ELM (IGSI-ELM) is generated with an elimination mechanism. At each learning step IGSI-ELM eliminates the worst hidden node from the already-recruited group if it is not the newly-recruited one. Finally, to verify the efficacy and feasibility of the proposed algorithms, i.e. GSI-ELM and IGSI-ELM, in this paper, experiments on regression and classification benchmark data sets are investigated.

Multi-task learning employs a shared representation of knowledge for learning several instances of the same problem. Multi-step time series problem is one of the most challenging problems for machine learning methods. The performance of a prediction model face challenges for higher prediction horizons due to the accumulation of errors. Cooperative coevolution employs in a divide and conquer approach for training neural networks and has been very promising for single step ahead time series prediction. Recently, co-evolutionary multi-task learning has been proposed for dynamic time series prediction. In this paper, we adapt co-evolutionary multi-task learning for multi-step prediction where predictive recurrence is developed to feature knowledge from previous states for future prediction horizon. The goal of the paper is to present a network architecture with predictive recurrence which is capable of multi-step prediction through a form of multi-task learning. We employ cooperative neuro-evolution and an evolutionary algorithm as baselines for comparison. The results show that the proposed method provides the best generalization performance in most cases. Comparison of results with the literature has shown to be promising which motivates further application of the approach for related real-world problems.

The paper presents a subspace neuro-fuzzy system with ordering of data items.The system assigns fuzzy weights to attributes in each fuzzy rule.The ordering technique makes the system robust to outliers.The presented system can outperform a subspace neuro-fuzzy system for noisy data. Neuro-fuzzy systems are known for their ability to both approximate and generalize presented data. In real life data sets not always all attributes (dimensions) of data are relevant or have the same importance. Some of them may be noninformative or unnecessary. This is why subspace technique is applied. Unfortunately this technique is vulnerable to noise and outliers that are often present in real life data. The paper describes a subspace neuro-fuzzy system with data ordering technique. Data items are ordered and assigned with typicalities. Data items with low typicalities have lower influence on the elaborated fuzzy model. This technique makes fuzzy models more robust to noise and outliers. The paper is accompanied by numerical experiments on real life data sets.

The attention of this paper is mainly concentrated on the H ∞ state estimator's design problem for a kind of discrete-time memristive neural networks (MNNs) with multiple fading measurements. The phenomenon of multiple fading measurements is represented by a set of individual stochastic variables obeying a predetermined distribution on interval 0,1. Firstly, the augmented system comprised of MNNs and the dynamics of estimation errors are put forward to implement the performance analysis. Then, under the framework of the difference inclusion theory combined with the Lyapunov function method, several sufficient conditions are established to guarantee the exponential mean-square stability as well as H ∞ performance index. Furthermore, the desired estimator parameter is obtained in view of the solution of a convex optimization problem. In the end, an illustrative numerical example is exploited to check the reliability and usefulness of the design scheme in this paper.

Abnormal event detection in video surveillance is extremely important, especially for crowded scenes. In recent years, many algorithms have been proposed based on hand-crafted features. However, it still remains challenging to decide which kind of feature is suitable for a specific situation. In addition, it is hard and time-consuming to design an effective descriptor. In this paper, video events are automatically represented and modeled in unsupervised fashions. Specifically, appearance and motion features are simultaneously extracted using a PCANet from 3D gradients. In order to model event patterns, a deep Gaussian mixture model (GMM) is constructed with observed normal events. The deep GMM is a scalable deep generative model which stacks multiple GMM-layers on top of each other. As a result, the proposed method acquires competitive performance with relatively few parameters. In the testing phase, the likelihood is calculated to judge whether a video event is abnormal or not. In this paper, the proposed method is verified on two publicly available datasets and compared with state-of-the-art algorithms. Experimental results show that the deep model is effective for abnormal event detection in video surveillance.

Remote medical resources configuration and management involves complex combinatorial Multi-Objective Optimization problem, whose computational complexity is a typical NP problem. Based on the MOEA/D framework, this paper applies the two-way local search strategy and the new selection strategy based on domination amount and proposes the IMOEA/D framework, following which each individual produces two individuals in mutation. In this paper, by using a new selection strategy, the parent individual is compared with two mutated offspring individuals, and the more excellent one is selected for the next generation of evolution. The proposed algorithm IMOEA/D is compared with eMOEA, MOEA/D and NSGA-II, and experimental results show that for most test functions, IMOEA/D proposed is superior to the other three algorithms in terms of convergence rate and distribution.

The person re-identification is an active research branch of the computer vision and attracts many researchers study on it. However, because of the variance in viewpoints, illumination, pedestrians' pose and some of other factors, building a robust person re-identification algorithm in open-world is rather challenging. In this paper, we propose two algorithms, indiscriminative patches trim strategy and multi-instance multi-label learning method, to fix the person re-identification in closed short-term surveillance network. There are two main contributions of this paper: first, we define the discriminative region in a person's image. Correspondingly, an adapted canopy-kmeans algorithm is proposed to evaluate the discriminability of image patches which form the discriminative regions. Besides, two strategies, in local image and in global gallery dataset, are proposed to filter out distractions. Finally, the true discriminative patches are employed to compute the similarity between two images, and then used to re-identify pedestrians. Second, we first introduce the multi-instance multi-label learning methods into re-identification, and hence we propose a framework of solving person re-identification by applying MIMLL. We employ two MIMLL methods, the MIMLBOOST and the MIMLSVM, to detect attributes in each image, and those detected attributes contribute to recognize different pedestrians. Experiments on two benchmark datasets show the competitive performance of the two proposed algorithms.

In many data mining applications, dimensionality reduction is a primary technique to map high-dimensional data to a lower dimensional space. In order to preserve more local structure information, we propose a novel orthogonal least squares regression model for feature extraction in this paper. The main contributions of this paper are shown as follows: first, the new least squares regression method is constructed under the orthogonal constraint which can preserve more discriminant information in the subspace. Second, the optimization problem of classical least squares regression can be solved easily. However the proposed objective function is an unbalanced orthogonal procrustes problem, it is so difficult to obtain the solution that we present a novel iterative optimization algorithm to obtain the optimal solution. The last one, we also provide a proof of the convergence for our iterative algorithm. Additionally, experimental results show that we obtain a global optimal solution through our iterative algorithm even though the optimization problem is a non-convex problem. Both theoretical analysis and empirical studies demonstrate that our method can more effectively reduce the data dimensionality than conventional methods.

Fuzzy knowledge is prevalent in real life, and ontology is the main way of knowledge representation in semantic web. In this paper, two main kinds of common fuzzy knowledge are sorted out, and there is a great significance for ontology representation of common fuzzy knowledge in semantic web. In addition, searching knowledge in ontology is the most common operation of semantic web, but heterogeneous ontologies seriously affect accuracy of information retrieval, and ontology mapping is the key to solve the problem. Therefore, in this paper, firstly an ontology representation method of common fuzzy knowledge is presented. Common fuzzy knowledge is polytypic and includes most commonly used fuzzy knowledge in reality. So fuzzy sets and Cloud Model are used to reflect and represent these 2 types of common fuzzy knowledge in ontology. Then, based on the ontology representation method of common fuzzy knowledge mentioned above, a corresponding algorithm of ontology mapping is presented, which is based on similarity calculation of concepts and Support Vector Machine. The proposed methods extend the scope of ontology application and are significant for information retrieval of fuzzy knowledge in the semantic web. The experiments show that the proposed methods are practicable and effective.

In this paper, we tackle the problem of depth estimation from a single image, which is essential for understanding 3D scene structure and can promote the development of visual applications. Markov Random Field (MRF) related depth estimation methods have attracted extensive attentions due to their ability of building structural relationships on outputs. Almost all of them employed engineered low-level absolute and relative features for constructing MRF. However, the engineered features are not capable of producing accurate depths for various scenes. In this paper, we propose Convolutional Conditional Random Field Network (CCRFN) consisting of feature learning and depth tuning components. In feature learning component, we build two Convolutional Neural Network (CNN) architectures to learn absolute and relative features from raw images. In depth tuning component, the learned features are fed into Conditional Random Field (CRF) to generate the depths of all pixels in an image by optimizing a well-defined energy function. CCRFN has two advantages that (1) it does not need hand-crafted features and (2) it models the depths at individual points as well as the relationship between them in the deep model. Experiments on widely used Make3D dataset show that CCRFN outperforms state-of-the-art methods on three evaluation criteria and generates more accurate depth maps with sharper object boundaries.

Very recently, vehicle detection in satellite images has become an emerging research topic with various applications ranging from military to commercial systems. However, it retains as an open problem, mainly due to the complex variations in imaging conditions, object intra-class changes, as well as due to its low-resolution. Coming with the rapid advances in deep learning for feature representation, in this paper we investigate the possibility to exploit deep neural features towards robust vehicle detection. In addition, along with the rapid growth in the data volume, new classification methodology is also demanded to explicitly handle the intra-class variations. In this paper, we propose a vehicle detection framework, which combines Deep Convolutional Neural Network (DNN) based feature learning with Exemplar-SVMs (E-SVMS) based, robust instance classifier to achieve robust vehicle detection in satellite images. In particular, we adopt DNN to learn discriminative image features, which has a high learning capacity. In our practice, the leverage of DNN has achieve significant performance boost by comparing to a serial of handcraft designed features. In addition, we adopt E-SVMs based robust classifier to further improve the classification robustness, which can be considered as an instance-specific metric learning scheme. By conducting extensive experiments with comparisons to a serial of state-of-the-art and alternative works, we further show that the combination of both schemes can benefit from each other to jointly improve the detection accuracy and effectiveness.

In this paper, we discuss the problem of optimal filter design for a class of networked stochastic systems subject to state delay and missing measurements. Both the random perturbations and the missing measurements are addressed in the system model, where the random perturbations are characterized by the multiplicative noises and the addressed phenomena of the missing measurements are modeled by a series of mutually independent Bernoulli random variables with individual occurrence probability. In view of the innovative analysis approach and the recursive projection formula, we design an optimal filter for networked systems with multiplicative noises and missing measurements such that the filtering error is minimized in mean square error sense. The main advantage of the proposed result lies in its recursive form applicable for online computations. In addition, we can see that the filter parameter can be obtained by solving some recursive equations. Finally, we give a numerical simulation example to illustrate the effectiveness of the filtering method proposed in this paper. HighlightsWe design a new filtering algorithm for time-varying networked systems with time-delay and missing measurements.We propose a filtering compensation scheme by taking the missing probabilities into account.We show that the new filtering method is optimal in minimum mean square error sense.

Engineering Asset Management (EAM) is a recently attractive discipline and it aims to address valuable contributions of asset management to organization's success. As of today, there is no specific method to evaluate performance of EAM standards. This paper aims to fill this gap and rank performance of asset management automatically after conducting survey, instead of evaluating questionnaires, analyzing results and ranking performances with a tedious process. Hence, it is necessary to develop intelligent data classification to simplify the whole procedure. Among many supervised learning methods, support vector machine attracts much attention for binary classification problems and its extension, namely multiple support vector machines, is able to solve multiclass classification problems. It is crucial to find optimal parameters of support vector machines prior to their use for prediction of unknown testing data sets. In this paper, novel Bayesian inference on optimal parameters of support vector machines is proposed. Firstly, a state space model is constructed to find the relationship between parameters of support vector machines and guess cross-validation accuracy. Here, the guess cross-validation accuracy aims to prevent support vector machines from overfitting. Secondly, particle filter is introduced to iteratively find posterior probability density functions of the parameters of support vector machines. Then, optimal parameters of support vector machines can be found from the posterior probability density functions. Ultimately, survey data collected from industry are used to validate the effectiveness of the proposed Bayesian inference method. Comparisons with some randomly selected parameters are conducted to highlight the superiority of the proposed method. The results show that the proposed Bayesian inference method can result in both high training and testing accuracies.

Online social network presents a great opportunity to analyze user behavior and mine the implicit personality traits from the social network data. Considering the personality recognition as a multi-label classification problem, this paper proposes a new probabilistic topic model (PT-LDA model) to predict the personality traits within the framework of Five Factor Model. The proposed model extends the Latent Dirichlet Allocation (LDA) model to integrate the n-gram features into few latent topics and each topic is characterized by not only the multinomial distribution over words but also the Gaussian distributions over personality traits. This paper develops a Gibbs-EM algorithm to solve the proposed model iteratively based on Gibbs sampling and expectation maximization. Quantitative evaluation shows that PT-LDA is more accurate, efficient and robust than several baselines. Our experiment also shows that the proposed model can be used to extract the interpretable topics associated with each personality trait, which provides a new way to uncover user behaviors in online social network.

In today׳s society where audio–visual content is ubiquitous, violence detection in movies and Web videos has become a decisive functionality, e.g., for providing automated youth protection services. In this paper, we concentrate on two important aspects of video content analysis: Time efficiency and modeling of concepts (in this case, violence modeling). Traditional approaches to violent scene detection build on audio or visual features to model violence as a single concept in the feature space. Such modeling does not always provide a faithful representation of violence in terms of audio–visual features, as violence is not necessarily located compactly in the feature space. Consequently, in this paper, we target to close this gap. To this end, we present a solution which uses audio–visual features (MFCC-based audio and advanced motion features) and propose to model violence by means of multiple (sub)concepts. To cope with the heavy computations induced by the use of motion features, we perform a coarse-to-fine analysis, starting with a coarse-level analysis with time efficient audio features and pursuing with a fine-level analysis with advanced features when necessary. The results demonstrate the potential of the proposed approach on the standardized datasets of the latest editions of the MediaEval Affect in Multimedia: Violent Scenes Detection (VSD) task of 2014 and 2015.

In this paper, we propose a transfer subspace learning approach cross-dataset facial expression recognition. To our best knowledge, this problem has been seldom addressed in the literature. While many facial expression recognition methods have been proposed in recent years, most of them assume that face images in the training and testing sets are collected under the same conditions so that they are independently and identically distributed. In many real applications, this assumption does not hold as the testing data are usually collected online and are generally more uncontrollable than the training data. Hence, the testing samples are likely different from the training samples. In this paper, we define this problem as cross-dataset facial expression recognition as the training and testing data are considered to be collected from different datasets due to different acquisition conditions. To address this, we propose a transfer subspace learning approach to learn a feature subspace which transfers the knowledge gained from the source domain (training samples) to the target domain (testing samples) to improve the recognition performance. To better exploit more complementary information for multiple feature representations of face images, we develop a multi-view transfer subspace learning approach where multiple different yet related subspaces are learned to transfer information from the source domain to the target domain. Experimental results are presented to demonstrate the efficacy of these proposed methods for the cross-dataset facial expression recognition task.

Recent years have witnessed the fast development of multimedia platforms in China, such as Youku, LeTV and Weibo. Images and videos are usually uploaded with textual descriptions, such as titles and introductions of these media. These texts are the key to multimedia content understanding, and this paper is dedicated to multimedia understanding with visual content entailment via recognizing semantic entailment in these texts. In fact, the natural language processing community has been manifesting increasing interest in semantic entailment recognition in English texts. Yet, so far not much attention has been paid to semantic entailment recognition in Chinese texts. Therefore, this paper investigates on multimedia semantic entailment with Chinese texts. Recognizing semantic entailment in Chinese texts can be cast as a classification problem. In this paper, a classification model is constructed based on support vector machine to detect high-level semantic entailment relations in Chinese text pair, including entailment and non-entailment for the Binary-Class and forward entailment, reverse entailment, bidirectional entailment, contradiction and independence for the Multi-Class. We explore different semantic feature combinations based on three kinds of Chinese textual features, including Chinese surface textual, Chinese lexical semantic and Chinese syntactic features, and utilize them to feed our classification model. The experiment results show that the accuracy of our classification model for semantic entailment recognition with the feature combination using all the three kinds of Chinese textual features achieves a much better performance than the baseline in Multi-Class and slightly better results than the baseline in the Binary-Class.

In the traditional Support Vector Machine Plus (SVM+), the grouping method has great randomness and only takes into account part of the structural information of the dataset. In order to overcome these shortcomings, in this paper, we propose a novel framework termed as FCSVM+ to improve the performance of SVM+ by combining clustering technique and feature selection. The new framework strategy is expected to not only take fully into account the structural information of the training data, but also partition the training data into more meaningful groups. To prove the advantage of the framework, in particular, we adopt two simplest feature selection methods, i.e. F-score and Laplacian score methods, to select the features, then apply a recently proposed clustering technique to get a better partition of training data by the selected features, in which the number of clusters could be found automatically. Three major contributions of this paper can be concluded as: (1) improving the performance of the existing SVM+ classifier; (2) extracting the potential structural information of the training data by using more feature attributes instead of one; (3) replacing the truncation method in SVM+ with a clustering technique. The comprehensive experimental results on the UCI benchmark datasets illustrate the validity and advantage of our approach.

High resolution remote sensing image captured by the satellites or the aircraft is of great help for military and civilian applications. In recent years, with an increasing amount of high resolution remote sensing images, it becomes more and more urgent to find a way to retrieve them. In this case, a few methods based on the statistical information of the local features are proposed, which have achieved good performances. However, most of the methods do not take the topological structure of the features into account. In this paper, we propose a new method to represent these images, by taking the structural information into consideration. The main contributions of this paper include: (1) mapping the features into a manifold space by a Lipschitz smooth function to enhance the representation ability of the features; (2) training an anchor set with several regularization constrains to get the intrinsic manifold structure. In the experiments, the method is applied to two challenging remote sensing image datasets: UC Merced land use dataset and Sydney dataset. Compared to the state-of-the-art approaches, the proposed method can achieve a more robust and commendable performance.

This paper, proposes a novel artificial neural network, called self-adjusting feature map (SAM), and develop its unsupervised learning ability with self-adjusting mechanism. The trained network structure of representative connected neurons not only displays the spatial relation of the input data distribution but also quantizes the data well. The SAM can automatically isolate a set of connected neurons, in which, the used number of the sets may indicate the number of clusters. The idea of self-adjusting mechanism is based on combining of mathematical statistics and neurological advantages and retreat of waste. In the training process, for each representative neuron has are three phases, growth, adaptation and decline. The network of representative neurons, first create the necessary neurons according to the local density of the input data in the growth phase. In the adaption phase, it adjusts neighborhood neuron pair's connected/disconnected topology constantly according to the statistics of input feature data. Finally, the unnecessary neurons of the network are merged or remove in the decline phase. In this paper, we exploit the SAM to handle some peculiar cases that cannot be handled easily by classical unsupervised learning networks such as self-organizing map (SOM) network. The remarkable characteristics of the SAM can be seen on various real world cases in the experimental results.

In this paper, we focus on unsupervised feature selection. As we have known, the combination of several feature units into a whole feature vector is broadly adopted for effective object representation, which may inevitably includes some irrelevant/redundant feature units or feature dimensions. Most of the traditional feature selection models can only select the feature dimensions without concerning the intrinsic relationship among different feature units. By taking into consideration the group sparsity of feature dimensions and feature units based on an l 2 , 1 minimization, we propose a new unsupervised feature selection model, unsupervised deep sparse feature selection (UDSFS) in this paper. In comparison with the state-of-the-arts, our UDSFS model can not only select the most discriminative feature units but also assign proper weight to the useful feature dimensions concurrently; moreover, the efficiency and robustness of our UDSFS can be also improved without extracting the discarded irrelevant feature units. For model optimization, we introduce an efficient iterative algorithm to solve the non-smooth, convex model and obtain a global optimization with the convergence rate as O ( 1 / K 2 ) (K is the iteration number). For the experiments, a new medical endoscopic image dataset, Abnormal Endoscopic Image Detection dataset (AEID), is built for evaluation; we also test our model using two public UCI datasets. Various experiments and comparisons with other state-of-the-arts justified the effectiveness and efficiency of our UDSFS model.

In this paper we develop a set of parallel algorithms for image segmentation basing on the authors' former work, the self-organizing map (SOM) based vector quantization (VQ) approach, by extending the method from serial computation to parallel processing, in order to accelerate the computation process. The parallel segmentation scheme is composed of a group of parallel algorithms for implementing the whole segmentation process, including parallel classification of the image into edge and non-edge pattern vectors, parallel training of an SOM network, parallel segmentation of the image by using the trained SOM model with VQ method, and adaptive parallel estimation of the segment number of the image being processed. In the paper, all the parallel algorithms have been implemented on graphic processing units (GPU) and applied to segmenting the human brain MRI images. The experimental results obtained in the work show that, compared with the original serial method implemented on CPU, the proposed parallel approach can achieve a significant improvement on the computation efficiency with overall speedup ratios increasing from 28.81 to 89.12 as image sizes increasing from 128×128 to 1024×1024, while keeping the segmentation performance unchanged.

Data fitting and prediction are important in many domains. As computing power improves, artificial intelligence fitting methods such as the WASD (weights and structure-determination) neuronet become more operable. Although the WASD neuronet has been applied to several other issues, its application on fitting data needs to be explored and discussed more specifically. This paper is committed to introducing the WASD-neuronet model activated by Chebyshev polynomials of class 1 for data fitting and to exploring its capability of data prediction. The learning-checking method and the concept of global minimum point are introduced to improve the prediction performance and extend the application of the WASD-neuronet model. Applying such a model to Asian population prediction substantiates its excellent performance. With numerical experiments validating the predicting performance and a final prediction based on historical data, this paper presents a reasonable tendency of Asian population.

It is well known that the Leave-One-Out Cross-Validation (LOO-CV) is a highly reliable procedure in terms of model selection. Unfortunately, it is an extremely tedious method and has rarely been deployed in practical applications. In this paper, a highly efficient Leave-One-Out Cross-Validation (LOO-CV) formula has been developed and integrated with the popular Regularized Extreme Learning Machine (RELM). The main contribution of this paper is the proposed algorithm, termed as Efficient LOO-CV-based RELM (ELOO-RELM), that can effectively and efficiently update the LOO-CV error with every regularization parameter and automatically select the optimal model with limited user intervention. Rigorous analysis of computational complexity shows that the ELOO-RELM, including the tuning process, can achieve similar efficiency as the original RELM with pre-defined parameter, in which both scale linearly with the size of the training data. An early termination criterion is also introduced to further speed up the learning process. Experimentation studies on benchmark datasets show that the ELOO-RELM can achieve comparable generalization performance as the Support Vector Machines (SVM) with significantly higher learning efficiency. More importantly, comparing to the trial and error tuning procedure employed by the original RELM, the ELOO-RELM can provide more reliable results by the virtue of incorporating the LOO-CV procedure.

This paper proposes a self-representation method for kNN (k nearest neighbors) classification. Specifically, this paper first designs a self-reconstruction method to reconstruct each data point by all the data, and the derived reconstruction coefficient is then used for calculating the k value for each training sample. Furthermore, a decision tree is built with the resulting k values for each data point to output labels of the training samples. With the built decision tree, the proposed method classifies test samples. Finally, the experimental results on real datasets showed the proposed method outperformed the state-of-the-art methods.

This paper is concerned with longitudinal partially linear models (LPLM) with ultrahigh-dimensional covariates and predictors. As flexible extension of linear regression models by allowing nonparametric intercept function to capture the overall trend over time, the LPLM are expected to be highly potential statistical models for analyzing high-dimensional longitudinal data such as longitudinal genetic data and functional magnetic resonance image data. Feature screening and variable selection are indispensable for LPLM in the presence of ultrahigh-dimensional covariates such as genetic markers and all pixels in image data. This paper proposes a two-stage variable selection procedure that consists of a quick screening stage and a post-screening refining stage, for the ultrahigh dimensional longitudinal partially linear models. The proposed approach is based on the partial residual method for dealing with the nonparametric baseline function. We establish the sure screening property of the proposed screening procedure in the first stage. Simulation results demonstrate the validity of this two-stage method. We further demonstrate the proposed methodology by an empirical analysis of a real data set collected in a soybean plant longitudinal genetic study.

Image anti-forensics, which aims to remove or forge traces upon which image forensics is based, has made rapid progress recently. To rebuild the credibility of forensics, many countermeasures have been proposed for detecting different anti-forensics. However, most existing countermeasures just target only one type of anti-forensics and are difficult to extend to counter other anti-forensics. In this paper, a multi-purpose countermeasure using autoregressive (AR) model is proposed for detecting various anti-forensics. Experimental results demonstrate that the proposed countermeasure achieves satisfactory performance in detecting all of the five well-known anti-forensic methods discussed in this paper. Even compared to the state-of-art specific counter-measures, our proposed countermeasure achieves similar or better performance.

In practice industrial applications, the training dataset collected from production process usually contain some outliers. This is a difficult case for partial least squares (PLS) regression which is sensitive to outliers. In order to solve this problem, the paper presents a multivariate statistical process monitoring and diagnosis method based on modified partial robust M-regression (MPRM). MPRM can eliminate the effects of the outliers in the raw data and establish accurate model by down-weighting the outliers strategy. In industry application, the key performance indicator related prediction and fault diagnosis are important to ensure the product quality and enhance economic benefits. Although, the standard PLS is one kind of the most widely used method, it performs an oblique projection to the input space, and thus cannot well distinguish quality-related faults from quality-unrelated ones. To overcome this drawback, the paper proposes to use the singular value decomposition (SVD) on coefficient matrix to decompose input data into quality-related part and quality-unrelated part. Then, robust monitoring statistics and control limits are derived for process monitoring purposes. A numerical example illustrates that the proposed method shows superior process monitoring performance in comparison with standard PLS when the modeling dataset contains outliers.

Visual recognition of crowd dynamics has had a huge impact on several applications including surveillance, situation awareness, homeland security and intelligent environments. However, the state-of-the-art in crowd analysis has become diverse due to factors such as: (a) the underlying definition of a crowd, (b) the constituent elements of the crowd processing model, (c) its application, hence (d) the dataset and (e) the evaluation criteria available for benchmarking. Although such diversity is healthy, the techniques for crowd modelling thus developed have failed to establish credibility therefore becoming unreliable and of questionable validity across different research disciplines. This paper aims to give an account of such issues by deducing key statistical evidence from the existing literature and providing recommendations towards focusing on the general aspects of techniques rather than any specific algorithm. The advances and trends in crowd analysis are drawn in the context of crowd modelling studies published in leading journals and conferences over the past 7 years. Finally, this paper shall also provide a qualitative and quantitative comparison of some existing methods using various publicly available crowd datasets to substantiate some of the theoretical claims.

With the wide use of consumer electronics and the rapid development of online shopping, more and more ad videos are developed for IDTV and mobile users. However, a huge amount of time spending on the Internet advertising somehow brings users uncomfortable viewing experience rather than effectively generates high consumption of advertised products. Therefore, it is urgent to find a viewer-friendly and advertiser-beneficial solution to launch ads. This paper is the first attempt to improve the effectiveness of advertising through combining online shopping information with an ad video and directing viewers to proper online shopping places. The proposed ActiveAd framework includes four main components. Firstly, an ad video analysis component detects both syntactic and semantic elements from ad videos, e.g. FMPIs (Frame Marked with Production Information), visual concepts, and textual keywords within the ad videos. Our ad video analysis provides a comprehensive solution to extract meaningful elements from ad videos. Secondly, a visual linking by search component is proposed to collect websites which contain similar images as FMPIs. Features used for the visual search are weighted and fused in order to ensure the uniformity of search results. Thirdly, different kinds of tags and product categories extracted from collected websites are aggregated in order to identify the representative text of the product. Finally, query keywords are selected through calculating cosine similarity from two kinds of keywords, i.e. keywords identified from tag aggregation and keywords obtained through ad video analysis (visual concept detection and textual keyword detection). A query vector is generated by selected keywords and used to retrieve product online. In this paper, a powerful cross-media contextual search including visual search, tag aggregation and textual search is achieved with the help of ad video analysis. Experiments demonstrate that our proposed ActiveAd achieves product recommendation effectively and efficiently.

Auto-encoder-a tricky three-layered neural network, known as auto-association before, constructs the "building block" of deep learning, which has been demonstrated to achieve good performance in various domains. In this paper, we try to investigate the dimensionality reduction ability of auto-encoder, and see if it has some kind of good property that might accumulate when being stacked and thus contribute to the success of deep learning.Based on the above idea, this paper starts from auto-encoder and focuses on its ability to reduce the dimensionality, trying to understand the difference between auto-encoder and state-of-the-art dimensionality reduction methods. Experiments are conducted both on the synthesized data for an intuitive understanding of the method, mainly on two and three-dimensional spaces for better visualization, and on some real datasets, including MNIST and Olivetti face datasets. The results show that auto-encoder can indeed learn something different from other methods. Besides, we preliminarily investigate the influence of the number of hidden layer nodes on the performance of auto-encoder and its possible relation with the intrinsic dimensionality of input data.

Deep learning algorithms are a subset of the machine learning algorithms, which aim at discovering multiple levels of distributed representations. Recently, numerous deep learning algorithms have been proposed to solve traditional artificial intelligence problems. This work aims to review the state-of-the-art in deep learning algorithms in computer vision by highlighting the contributions and challenges from over 210 recent research papers. It first gives an overview of various deep learning approaches and their recent developments, and then briefly describes their applications in diverse vision tasks, such as image classification, object detection, image retrieval, semantic segmentation and human pose estimation. Finally, the paper summarizes the future trends and challenges in designing and training deep neural networks.

This paper proposes ELMOEA/D, a surrogate-assisted MOEA, for solving costly multi-objective problems in small evaluation budgets. The proposed approach encompasses a state-of-the-art MOEA based on decomposition and Differential Evolution (MOEA/D-DE) assisted by Extreme Learning Machines (ELMs). ELMOEA/D is tested in instances from three well-known benchmarks (ZDT, DTLZ and WFG) with 5-60 decision variables, 2 and 5 objectives. The ELMOEA/D's performance is also analyzed on a real problem (Airfoil Shape Optimization). The impact of some ELMs parameters and an automatic model selection mechanism is investigated. The results obtained by ELMOEA/D are compared with those of two state-of-the-art surrogate approaches (MOEA/D-RBF and ParEGO) and a non-surrogate-based MOEA (MOEA/D). The ELMOEA/D variants are among the best results for most benchmark instances and for the real problem. HighlightsThe problem of solving expensive MOPs with small evaluation budgets is treated.The paper investigates the use of ELM as a surrogate model in MOEAs.A procedure is used to automatically select ELM parameters.The scalability of surrogates concerning the total of decision variables and number of objectives is evaluated.Experimental results show the good performance of ELMOEA/D in different problems.

Large scale systems (LSS) have a large size with several control loops. It is always demanding that all the local controllers of the LSS work in an optimal situation to raise the efficiency of the system. Therefore the performance monitoring of the distributed systems is common to detect any roots of deficiency in the subsystems. This paper presents control performance assessment (CPA) for the Rhine-Meuse Delta water network. The Delta water system consists of many rivers and sea outlets with barriers and sluices. The water network is in the low-lying area surrounded by rivers and the North Sea, and because of its location, the system is at high risk of floods. For the monitoring of the system, two new data driven performance indexes based on wavelet-based probability density function (PDF) are designed for the system to assess the performance of the subsystems and also the whole system. Finally, a comparison with Model predictive control benchmark is made to show the capability of the new benchmark. Simulation results show the effectiveness of the suggested methods in detecting the floods. The Rhine-Meuse Delta water network is a large scale system.This paper presents control performance assessment for the Rhine-Meuse Delta water network.Two new data driven performance indexes are considered for monitoring the system.The performance indexes are designed based on wavelet-based probability density function.Simulation results show the ability of the suggested methods in detecting the floods.

This paper proposes an online tracking algorithm that employs a confidence combinatorial map model. Drifting is a problem that easily occurs in object tracking and most of the recent tracking algorithms have attempted to solve this problem. In this paper, we propose a confidence combinatorial map that describes the structure of the object, based on which the confidence combinatorial map model is developed. The model associates the relationship between the object in the current frame and that in the previous frame. On the strength of this relationship, more precisely classified samples can be selected and are employed in the model update stage, which directly influences the occurrence of the tracking drift. The proposed algorithm was estimated on several public video sequences and the performance was compared with several state-of-the-art algorithms. The experiments demonstrate that the proposed algorithm outperforms other comparative algorithms and gives a very good performance.

In this paper, we examine the problem of learning sparse representations of visual patterns in the context of artificial and biological vision systems. There are a myriad of strategies for sparse coding that often result in similar feature properties for the learned feature set. Typically this results in a bank of Gabor-like or edge filters that are sensitive to a range of distinct angular and radial frequencies. The theory and experimentation that is presented in this paper serves to provide a better understanding of a number of specific properties related to low-level feature learning. This includes close examination of the role of phase pairing in complex cells, the role of depth information and its relationship to variation of intensity and chroma, and deriving hybrid features that borrow from both analytic forms and statistical methods. Together, these specific examples provide context for more general discussion of effective strategies for feature learning. In particular, we make the case that imposing additional constraints on mechanisms for feature learning inspired by biological vision systems can be useful in guiding constrained optimization towards convergence, or specific desirable computational properties for representation of visual input in artificial vision systems.

The Bag-of-Visual-Words (BoVW) representation has been widely used to approach a number of different high-level computer vision tasks. The idea behind the BoVW representation is similar to the Bag-of-Words (BoW) used in Natural Language Processing (NLP) tasks: to extract features from the dataset, then build feature histograms that represent each instance. Although the approach is simple and effective facilitating its applicability to a wide range of problems, it inherits a well-known limitation from the traditional BoW; the disregarding of spatial information among extracted features (sequential information in text), which could be useful to capture discriminative visual-patterns. In this paper, we alleviate this limitation with the joint use of visual words and multi-directional sequences of visual words (visual n-grams). The contribution of this paper is twofold: (i) to build new simple-effective visual features inspired in the popular idea of n-gram representations in NLP and (ii) to propose the Multiple Kernel Learning (MKL) strategies to better exploit the joint use of visual words and visual n-grams in Image Classification (IC) tasks. For the former, we propose building a codebook of visual n-grams, and use them as attributes to represent images by means of the BoVW representation. For the second point, we consider the visual words and visual n-grams as different feature spaces, then we propose MKL strategies to better integrate the visual information. We evaluate our proposal in the image classification task using five different datasets: Histopathology, Birds, Butterflies, Scenes and a subset of 6 classes of CalTech-101. Experimental results show that the proposed strategies exploiting our visual n-grams, outperforms or is competitive with (i) the traditional BoVW, (ii) the BoVW using visual n-grams under traditional fusion schemes (e.g., ensemble based classifiers) and (iii) other approaches in the literature for IC that consider the spatial context.

The privacy-preserving data analysis has been gained significant interest across several research communities. The current researches mainly focus on privacy-preserving classification and regression. On the other hand, feature selection is also one of the key problems in data mining and machine learning. However, for privacy-preserving feature selection, the relevant papers are few. In this paper, a local learning-based feature weighting framework is introduced. Moreover, in order to preserve the data privacy during local learning-based feature selection, the objective perturbation and output perturbation strategies are used to produce local learning-based feature selection algorithms with privacy preservation. Meanwhile, we give deep analysis about their privacy preserving property based on the differential privacy model. Some experiments are conducted on benchmark data sets. The experimental results show that our algorithms can preserve the data privacy to some extent and the objective perturbation always obtains higher classification performance than output perturbation when the privacy preserving degree is constant.

Automated seizure detection using EEG has gained increasing attraction in recent years and appeared more and more helpful in both diagnosis and treatment. How to design an appropriate feature extraction method and how to select an efficient classifier are recognized to be crucial in the successful realization. This paper first proposes a new Mahalanobis-similarity-based feature extraction method on the basis of the Mahalanobis distance and discrete wavelet transformation (DWT). Then in order to further improve the performance, this paper designs a fusion feature (MS-SE-FF) in the feature-fusion level, where the Mahalanobis-similarity-based feature characterizing the similarity between signals and the sample-entropy-based feature characterizing the complexity of signals are combined together. Finally, an automated seizure detection method FF-ELM-SD has been built, which is integrated between the novel fusion feature MS-SE-FF and extreme learning machine (ELM). Experimental results demonstrate that the proposed method FF-ELM-SD does a good job in the epileptic seizure detection while preserving the efficiency and simplicity.