Search

At the moment, smell sensors for odor source localization in mobile robotics represent a topic of interest for researchers around the world. In particular, we introduce in this paper the idea of developing biologically inspired sniffing robots in combination with bioinspired techniques such as evolutionary computing. The aim is to approach the problem of creating an artificial nose that can be incorporated into a real working system, while considering the environmental model and odor behavior, the perception system, and algorithm for tracking the odor plume. Current algorithms try to emulate animal behavior in an attempt to replicate their capability to follow odors. Nevertheless, odor perception systems are still in their infancy and far from their biological counterpart. This paper presents a proposal in which a real-working artificial nose is tested as a perception system within a mobile robot. Genetic programming is used as the learning technique platform to develop odor source localization algorithms. Experiments in simulation and with an actual working robot are presented and the results compared with two algorithms. The quality of results demonstrates that genetic programming is able to recreate chemotaxis behavior by considering mathematical models for odor propagation and perception system.

In this paper, we examine the problem of learning sparse representations of visual patterns in the context of artificial and biological vision systems. There are a myriad of strategies for sparse coding that often result in similar feature properties for the learned feature set. Typically this results in a bank of Gabor-like or edge filters that are sensitive to a range of distinct angular and radial frequencies. The theory and experimentation that is presented in this paper serves to provide a better understanding of a number of specific properties related to low-level feature learning. This includes close examination of the role of phase pairing in complex cells, the role of depth information and its relationship to variation of intensity and chroma, and deriving hybrid features that borrow from both analytic forms and statistical methods. Together, these specific examples provide context for more general discussion of effective strategies for feature learning. In particular, we make the case that imposing additional constraints on mechanisms for feature learning inspired by biological vision systems can be useful in guiding constrained optimization towards convergence, or specific desirable computational properties for representation of visual input in artificial vision systems.

The Bag-of-Visual-Words (BoVW) representation has been widely used to approach a number of different high-level computer vision tasks. The idea behind the BoVW representation is similar to the Bag-of-Words (BoW) used in Natural Language Processing (NLP) tasks: to extract features from the dataset, then build feature histograms that represent each instance. Although the approach is simple and effective facilitating its applicability to a wide range of problems, it inherits a well-known limitation from the traditional BoW; the disregarding of spatial information among extracted features (sequential information in text), which could be useful to capture discriminative visual-patterns. In this paper, we alleviate this limitation with the joint use of visual words and multi-directional sequences of visual words (visual n-grams). The contribution of this paper is twofold: (i) to build new simple-effective visual features inspired in the popular idea of n-gram representations in NLP and (ii) to propose the Multiple Kernel Learning (MKL) strategies to better exploit the joint use of visual words and visual n-grams in Image Classification (IC) tasks. For the former, we propose building a codebook of visual n-grams, and use them as attributes to represent images by means of the BoVW representation. For the second point, we consider the visual words and visual n-grams as different feature spaces, then we propose MKL strategies to better integrate the visual information. We evaluate our proposal in the image classification task using five different datasets: Histopathology, Birds, Butterflies, Scenes and a subset of 6 classes of CalTech-101. Experimental results show that the proposed strategies exploiting our visual n-grams, outperforms or is competitive with (i) the traditional BoVW, (ii) the BoVW using visual n-grams under traditional fusion schemes (e.g., ensemble based classifiers) and (iii) other approaches in the literature for IC that consider the spatial context.

This paper addresses region aggregation issues of multi-agent networks (MANs) that consist of finite number of isolated sub-multi-agent systems (sub-MASs) with multiple equilibria (ME) in multi-dimensional coordinate systems (MDCSs). To investigate aggregation of this kind of MANs via switching strategies, switched systems with ME are used to describe such networks under the case that each agent has a unique equilibrium point and all the equilibria are different from each other. Based on the novel stability concepts of region stability (RS) and exponential region stability (ERS) introduced, this paper proposes several RS and ERS results for sub-MASs with ME via the maximum energy function method, and then presents several region aggregation results for such MANs under three suitable assumptions. Also, a numerical example illustrates the effectiveness and practicality of our new results.

The privacy-preserving data analysis has been gained significant interest across several research communities. The current researches mainly focus on privacy-preserving classification and regression. On the other hand, feature selection is also one of the key problems in data mining and machine learning. However, for privacy-preserving feature selection, the relevant papers are few. In this paper, a local learning-based feature weighting framework is introduced. Moreover, in order to preserve the data privacy during local learning-based feature selection, the objective perturbation and output perturbation strategies are used to produce local learning-based feature selection algorithms with privacy preservation. Meanwhile, we give deep analysis about their privacy preserving property based on the differential privacy model. Some experiments are conducted on benchmark data sets. The experimental results show that our algorithms can preserve the data privacy to some extent and the objective perturbation always obtains higher classification performance than output perturbation when the privacy preserving degree is constant.

Automated seizure detection using EEG has gained increasing attraction in recent years and appeared more and more helpful in both diagnosis and treatment. How to design an appropriate feature extraction method and how to select an efficient classifier are recognized to be crucial in the successful realization. This paper first proposes a new Mahalanobis-similarity-based feature extraction method on the basis of the Mahalanobis distance and discrete wavelet transformation (DWT). Then in order to further improve the performance, this paper designs a fusion feature (MS-SE-FF) in the feature-fusion level, where the Mahalanobis-similarity-based feature characterizing the similarity between signals and the sample-entropy-based feature characterizing the complexity of signals are combined together. Finally, an automated seizure detection method FF-ELM-SD has been built, which is integrated between the novel fusion feature MS-SE-FF and extreme learning machine (ELM). Experimental results demonstrate that the proposed method FF-ELM-SD does a good job in the epileptic seizure detection while preserving the efficiency and simplicity.

The increasing demand for copyright protection has brought extensive interest in video fingerprinting technology, which is considered as an effective way for digital content authentication. In this paper, an innovative method for video authenticity based on the fingerprint of scene frame has been proposed. Since the probability of 5 identical scene frames existing successively in different videos is extremely low, the proposed method firstly extract 5 successive different scene frames to make up the video fingerprint. This video fingerprint is then combined with the ID information of video to form meta fingerprint data. The metadata is stored in Bag-words format, leading to the saving of 75% storage space. In the process of authentication, a binary search of inverted file is applied to accelerate the matching speed. The results of simulation experiments using MATLAB show that the video authenticity method proposed in this paper achieves excellent detection performance, with the average accuracy over 98% at an average speed of 12s per video. This makes the real-time detection become possible on the Internet.

This paper concerns the use of multiple views of a feature set to select a small amount of useful unlabeled data. In the semi-supervised learning (SSL) approach, using a selection strategy, strongly discriminative examples are first selected from unlabeled data and then, together with labeled data, utilized for training a (supervised) classifier or used for re-training the ensemble classifier. In this scenario, the selection strategy plays an important role in improving classification performance. This paper investigates a new selection strategy for a case in which the data are composed of different multiple views: first, multiple views of the data are derived independently; second, each of the views are used to measure corresponding confidence levels with which examples to be selected are evaluated; third, all the confidence levels measured from the multiple views are used as a weighted average to derive the target confidence; this select-and-train process is repeated for a pre-defined number of iterations. The experimental results, obtained using semi-supervised support vector machines for synthetic and real-life benchmark data, demonstrate that the proposed mechanism can compensate for the shortcomings of traditional strategies. In particular, the results demonstrate that when the data is appropriately decomposed into multiple views, this strategy can achieve further improved results in terms of the classification accuracy. HighlightsA small amount of unlabeled data was selected to enhance classification accuracy.To select them efficiently, multiple views of the labeled and unlabeled data were utilized.The confidence levels of the multiple views are referred to derive the target confidence.Run-time characteristics and error rates of the proposed criterion were empirically evaluated.

In this paper, we discuss the long-term time series forecasting using a Multilayer Neural Network with Multi-Valued Neurons (MLMVN). This is a complex-valued neural network with a derivative-free backpropagation learning algorithm. We evaluate the proposed approach using a real-world data set describing the dynamic behavior of an oilfield asset located in the coastal swamps of the Gulf of Mexico. We show that MLMVN can be efficiently applied to univariate and multivariate one-step- and multi-step ahead prediction of reservoir dynamics. This paper is not only intended for proposing to use a complex-valued neural network for forecasting, but to deeper study some important aspects of the application of ANN models to time series forecasting that could be of the particular interest for pattern recognition community. A long-term forecasting of oil production was done using MLMVN.Univariate and multivariate forecasting models were developed.MLMVN is efficient for both multivariate and univariate forecasting models.MLMVN-based prediction combines both regression and pattern recognition approaches.

Clustering ensemble is an important part of ensemble learning. It aims to study and integrate multiple clustering results from different clustering algorithms or same algorithm with different initial parameters for the same dataset. CHAMELEON is a hierarchical clustering algorithm which can discover natural clusters of different shapes and sizes as the result of its merging decision dynamically adapts to the different clustering model characterized. Inspired by the idea of CHAMELEON, the paper proposes a novel clustering ensemble models including semi-supervised method and discusses its application in fault diagnosis of high speed train (HST) running gear. The contributions of this paper include: constructing a sparse graph via the similarity matrix which aggregates multiple clustering results; partitioning the sparse graph (vertex=object, edge weight=similarity) into a large number of relatively small sub-clusters; obtaining the final clustering partition by merging these sub-clusters repeatedly. The experimental results demonstrate that our method outperforms some of state-of-the-art ensemble algorithms regarding the accuracy and stability and recognizes fault patterns of HST running gear effectively.

The prohibitive computational complexity caused by redundant dictionary which is used in signal sparse decomposition has always been perplexing researchers of signal processing all the time. However, large over-complete dictionaries are essential to approximate the signal. In this paper, a decomposition algorithm based on modulation correlation partition (PBMC) is introduced, which improves the process of searching matched atoms in the redundant dictionary of functions. Through analyzing the structure of signals, we unite the frequency factor and phase factor to obtain the 2D modulation factor. The over-complete dictionary is partitioned into several sub-dictionaries according to the modulation correlation of atoms. Each sub-dictionary is represented by a single selected atom which is used in the greedy algorithm. At the end of this paper, experimental results show that the computational complexity of signal sparse decomposition can be adequately reduced by partitioning over-complete dictionary without impacting the decomposition result. HighlightsAn algorithm that raises efficiency of matching pursuit is proposed.Partition strategy is introduced to improve structure of redundant dictionary.The proposed approach is more effective than the traditional ones and has robustness.

From the past few years, the size of the data grows exponentially with respect to volume, velocity, and dimensionality due to wide spread use of embedded and distributed surveillance cameras for security reasons. In this paper, we have proposed an integrated approach for biometric-based image retrieval and processing which addresses the two issues. The first issue is related to the poor visibility of the images produced by the embedded and distributed surveillance cameras, and the second issue is concerned with the effective image retrieval based on the user query. This paper addresses the first issue by proposing an integrated image enhancement approach based on contrast enhancement and colour balancing methods. The contrast enhancement method is used to improve the contrast, while the colour balancing method helps to achieve a balanced colour. Importantly, in the colour balancing method, a new process for colour cast adjustment is introduced which relies on statistical calculation. It adjusts the colour cast and maintains the luminance of the image. The integrated image enhancement approach is applied to the enhancement of low quality images produced by surveillance cameras. The paper addresses the second issue relating to image retrieval by proposing a content-based image retrieval approach. The approach is based on the three features extraction methods namely colour, texture and shape. Colour histogram is used to extract the colour features of an image. Gabor filter is used to extract the texture features and the moment invariant is used to extract the shape features of an image. The use of these three algorithms ensures that the proposed image retrieval approach produces results which are highly relevant to the content of an image query, by taking into account the three distinct features of the image and the similarity metrics based on Euclidean measure. In order to retrieve the most relevant images, the proposed approach also employs a set of fuzzy heuristics to improve the quality of the results further. The results show the proposed approaches perform better than the well-known existing approaches.

A huge number of videos are posted every day on social media platforms such as Facebook and YouTube. This makes the Internet an unlimited source of information. In the coming decades, coping with such information and mining useful knowledge from it will be an increasingly difficult task. In this paper, we propose a novel methodology for multimodal sentiment analysis, which consists in harvesting sentiments from Web videos by demonstrating a model that uses audio, visual and textual modalities as sources of information. We used both feature- and decision-level fusion methods to merge affective information extracted from multiple modalities. A thorough comparison with existing works in this area is carried out throughout the paper, which demonstrates the novelty of our approach. Preliminary comparative experiments with the YouTube dataset show that the proposed multimodal system achieves an accuracy of nearly 80%, outperforming all state-of-the-art systems by more than 20%.

As a classical partitional clustering algorithm, k-means algorithm is sensitive to initial centroids and may malfunction when dealing with datasets which contain clusters with different scales and densities. To improve the effectiveness of k-means algorithm, an outlier factor based partitional clustering analysis method is presented in this paper. Outlier factor is usually used to indicate the degree of an object to be abnormal in the dataset. For the proposed method, it is used to find the core objects. And then the Must-link constraints is generated to put the neighboring core objects into the same cluster. First, a similar-density-array-based outlier factor is proposed to find the core objects in the dataset. Then the neighboring core objects are distributed into the same sub-cluster. The sub-clusters are treated as the representative objects and these representative objects are then clustered following the process of the traditional k-means algorithm. Finally, the non-core objects are assigned to their nearest clusters, respectively. The experiments are performed on four datasets from UCI Machine Learning Repository and a field dataset from a ball mill pulverizing system. The experimental results verify that the effectiveness of our algorithm is high. HighlightsNew method to combine outlier factor with k-means algorithm, and local information is effectively used.The proposed method is compared with two classical constrained k-means algorithms.Besides UCI datasets, a field dataset from a ball mill pulverizing system is used in the experiment.An outlier factor is proposed in the paper and is compared with LOF and COF in the experiment.The impact of the parameter of the outlier factors are tested and discussed in detail in the experiment.

The goal of Multiple Kernel Learning (MKL) is to combine kernels derived from multiple sources in a data-driven way with the aim to enhance the accuracy of a target kernel machine. State-of-the-art methods of MKL have the drawback that the time required to solve the associated optimization problem grows (typically more than linearly) with the number of kernels to combine. Moreover, it has been empirically observed that even sophisticated methods often do not significantly outperform the simple average of kernels. In this paper, we propose a time and space efficient MKL algorithm that can easily cope with hundreds of thousands of kernels and more. The proposed method has been compared with other baselines (random, average, etc.) and three state-of-the-art MKL methods showing that our approach is often superior. We show empirically that the advantage of using the method proposed in this paper is even clearer when noise features are added. Finally, we have analyzed how our algorithm changes its performance with respect to the number of examples in the training set and the number of kernels combined.

Increasing deployment of intermittent power generation from renewable resources in the smart grid, such as photovoltaic (PV) or wind farm, will cause large system frequency fluctuation when the load-frequency control (LFC) capacity is not enough to compensate the unbalance of generation and load demand. Even worse, the system inertia will decrease when the smart grid is in island operating mode, which would degrade system damping and cause system instability. Meanwhile, electric vehicles (EVs) will be widely used by customers in the near future, where the EV station could be treated as dispersed battery energy storage. Therefore, the vehicle-to-grid (V2G) technology can be employed to compensate for inadequate LFC capacity, thus improving the island smart grid frequency stability. In this paper, an on-line reinforcement learning (RL) based method, called goal representation adaptive dynamic programming (GrADP), is employed to adaptive control of units in an island smart grid. In the controller design, adaptive supplementary control signals are provided to proportional-integral-derivative (PID) controller by GrADP in a real-time manner. Comparative simulation studies on a benchmark smart grid with micro-turbine (MT), EVs, PV array and wind power are carried out among the GrADP controller, the original PID controller and the particle swarm optimization (PSO) based fuzzy logic controller. Simulation results demonstrate competitive performance and satisfied learning ability of the GrADP based coordinate controller. Moreover, the impact of signal transmission delay on the control performance is also considered, and suggestions to address this issue are given in the paper.

In order to face with various contexts and situations, autonomous robots should be endowed with many different sensors and behaviors. These requirements pose new challenges such as the coordination of multiple parallel activities and the efficient use of the limited sensorial and cognitive resources. These challenges are often tackled by relying on a priori and well defined coordination schema among behaviors, and on fixed periodic or ad hoc monitoring strategies. Our working hypothesis is that adaptive control strategies, inspired by natural cyclic processes, can be used to cope with these problems. However, the development of these flexible strategies may result hard to be modeled and implemented. Recently, the possibility of abstracting an implementation view into an architectural design is getting more achievable. Hence, in this paper, we propose a modeling framework that allows developers to model simple behavior-based robotic systems enhanced by the use of Central Pattern Generators (CPGs) for modulating the sensor-motor loops. Different from other approaches, the use of CPGs, here, is to efficiently exploit the limited sensorial and cognitive resources, and to coordinate the multiple activities the robot is endowed with, by balancing sensors elaboration and action execution. The framework can use these models to generate robotic control executable code for analysis purposes. In this paper, we focus on parallel behaviors managing analysis.

Image feature is a significant factor for the success of robust face recognition. Recently sparse representation based classifier (SRC) has been widely applied to robust face recognition by using sparse representation residuals to tolerate disturbed image features (e.g., occluded pixels). In order to deal with more complicated image variations, robust representation based classifier, which estimates feature weights (e.g., low weight values are given to the pixels with big representation residuals), has attracted much attention in recent work. Although these methods have achieved improved performance by estimating feature weights independently, structured information and prior knowledge of image features are ignored in these works, resulting in unsatisfactory performance in some challenging cases. Thus how to better learn image feature weight to fully exploit structure information and prior knowledge is still an open question in robust face recognition. In this paper, we proposed a novel joint representation and pattern learning (JRPL) model, in which the feature pattern weight is simultaneously learned with the representation of query image. Especially a feature pattern dictionary, which captures structured information and prior knowledge of image features, are constructed to represent the unknown feature pattern weight of a query image. An efficient algorithm to solve JRPL was also presented in this paper. The experiments of face recognition with various variations and occlusions on several benchmark datasets clearly show the advantage of the proposed JRPL in accuracy and efficiency.

This paper focuses on a new kind of artificial neural networks - the Z-transform artificial neural networks (ZTANNs). It is proposed to use the Z-transform instead of ordinary weights and a linear activation function of an artificial neuron. This extension allows to use artificial neural networks in new areas. The ordinary description of artificial neural networks is a special case of the description proposed in this paper. It also contains a description of the use of the ZTANN for automatic identification of objects in digital control system.

There are a large number of unlabeled examples in real-world application, and if the labels of these unlabeled examples are given manually, then the cost will be very high. The problem about how to label these massive unlabeled instances with the minimal cost is paid more and more attention. Active learning efficiently solves this bottleneck by selecting the most informative examples from the unlabeled examples and establishing a classifier with a higher classifier accuracy to label unlabeled examples, which greatly improves work efficiency. In this paper, we compare two kinds of traditional active learning algorithms relying on a single classifier, namely Gaussian process and margin sampling active learning, in two aspects of classification error rates and computing time. Moreover, we compare their improved versions (GPMAL and IMS) which apply the manifold-preserving graph reduction (MPGR) algorithm. MPGR constructs a subset which well exploits the structural spatial connectivity and spatial diversity among examples. By using MPGR, an active learner selects the informative and representative candidates from the subset instead of the whole unlabeled data set. In addition, a comparison with a state-of-the-art active learning method, QUIRE, is provided. Experimental results on multiple data sets show that both GPMAL and IMS have their own advantages. HighlightsThis paper compares Gaussian process and margin sampling active learning.Moreover, their improved versions GPMAL and IMS are compared.Experimental results show that the two methods have their own advantages.

In this paper, we will introduce a new method to investigate the best visualization of remote sensing imagery data sets on graphics processing units (GPU). The main shortcoming of classical dimension reduction methods is that their visualizations inevitably lose a significant amount of information in remote sensing imagery. Our proposed method (Enhancement Trustworthy (ET)) preserves the color distance as noticeably as possible for high dimension space by defining parallel-supervised extra step to enhance the quality of visualization. The ET classifies the visualization colors into false and true, and constructs a trustworthiness of CIE color space to improve the trustworthy value of false colors. The trustworthiness value of points inside CIE color space illustrate the locations of false and true projected colors in the visualization. Our method is supervised, where the user can select the interested points to enhance by ET. The proposed method is general enhancement step which can be used to enhance any visualization, for example, we used it to enhance the visualizations of remote sensing imagery data sets, which are carried by PCA, SPE and Isomap methods. The results showed the ability of ET to overcome the false colors and return the more reliable true colors. The colors of enhance visualizations showed more details which represent the real neighborhood relation in original data sets. HighlightsIn this paper, we will introduce a new method to investigate the best visualization of remote sensing imagery data sets on graphics processing units (GPU).Our enhanced method (ET) preserves the color distance for remote sensing imagery by defining parallel supervised extra step to enhance the quality of visualization.The ET classifies the visualization colors into false and true, and constructs a trustworthiness of CIE color space.The trustworthiness value of points inside CIE color space illustrate the locations of false and true projected colors in the visualization.The proposed method is general enhancement step which can be used to enhance any visualization.

The human finger is a three-dimensional object. More information will be provided if 3D fingerprint images are available compared with 2D fingerprints. This paper explores 3D fingerprint features, as well as their possible applications. Novel fingerprint features, which are defined as Curvature Features (e.g. curve-skeleton, overall maximum curvatures), are for the first time proposed and investigated in this paper. Those features are then employed to assist more accurate fingerprint matching or classify human gender after analyzing their characteristics. A series of experiments are conducted to evaluate the effectiveness of employing these novel fingerprint features to fingerprint recognition based on the established database with 541 fingers. Results show that an Equal error Rate (EER) of ~15% can be achieved when only curve-skeleton is used for recognition. But, promising EER of ~3.4% is realized by combining curve-skeleton with classical 2D fingerprint features for recognition that indicates the prospect of 3D fingerprint recognition. The proposed overall maximum curvatures are found to be helpful for human gender classification.

In this paper, through extension of the present methods and based on error minimization, two fast and efficient layer-by-layer pre-training methods are proposed for initializing deep neural network (DNN) weights. Due to confrontation with a large number of local minima, DNN training often does not converge. By proper initializing of DNN weights instead of random values at the beginning of the training, it is possible to avoid many local minima. The first version of the proposed method is for pre-training the deep bottleneck neural network (DBNN) in which the DBNN is broken down to some corresponding single-hidden-layer bottleneck neural networks (BNN) which must be trained first. The weight values resulting from their training are then applied in the DBNN. The proposed method was utilized to pre-train a five-hidden-layer DBNN to extract the non-linear principal components of face images in the Bosphorus database. A comparison of the randomly initialized DBNN result with pre-trained DBNN by the layer-by-layer pre-training method shows that this method not only increased the convergence rate of training but also improved its generalizability. Furthermore, it has been shown that this method yields higher efficiency and convergence speed in comparison with some of the previous pre-training methods. This paper also presents the bidirectional version of the layer-by-layer pre-training method for hetero-associative DNN pre-training. This method pre-trains DNN weights in forward and backward manner in parallel. Bidirectional layer-by-layer pre-training was utilized to pre-train the classifier DNN weights, and revealed that both the training speed and the recognition rate were improved in Bosphorus and CK+ databases.

Global Positioning System (GPS) and Inertial Navigation System (INS) are two salient technologies delivering a vehicle?s position, velocity and attitude parameters for navigation. The standalone GPS undergoes signal outages (in forest area) and multipath effects (in urban areas), whereas the standalone INS solution accuracy deteriorates with time. To overcome the limitations of standalone GPS and INS, an integrated INS/GPS system is required for continuous, accurate, and reliable navigation solution. This paper proposes a hybrid method of Principal Component Regression (PCR) and Random Forest Regression (RFR) for INS and GPS data fusion.Traditional filter based fusion techniques like Kalman filter (KF), linearized KF, and particle filter (PF) have deficiencies related to sensor error modeling, immunity to noise, and computational load. Artificial Neural Networks (ANN) are introduced to overcome these limitations, but they show poor generalization capability in the presence of noise and suffer from over fitting. Recently, RFR is proposed as an efficient fusion technique due to its flexibility to handle non-linear input-output relationships and overcome the over fitting problems. The limitation with RFR is that it cannot handle both linear and non-linear input-output relationships effectively Hence, the paper introduces PCR, which can model linear input-output relationships. Thus, a hybrid approach of RFR and PCR provides enhanced modeling of input-output relationships leading to increased prediction accuracy and thereby improved integrated INS/GPS system performance. The results are validated through five simulated GPS outages taken on real field test data. The proposed model showed a maximum of 45.06% decrement in positional error when compared to RFR.

This paper summarizes a multivariate statistical method called kernel PLS in its use of prediction. PLS has been widely used in the multivariate statistical process monitoring but is only effective in linear condition. This leads to further study and construction of a variant of PLS. Kernel PLS algorithm has been proposed to solve this problem. KPLS establishes relationship between input and output variables in a high-dimensional space. Input data set can be considered as linear in this space. This paper discusses the prediction model construction process and shows nonlinear examples to demonstrate the effectiveness of KPLS. This prediction method has proven to be powerful in many areas, such as chemometrics, bioinformatics and neuroscience.

This paper proposes using the Parallel Layer Perceptron (PLP) network, instead of the Single Layer Feedforward neural network (SLFN) in the Extreme Learning Machine (ELM) framework. Differently from the SLFNs which consider cascade layers, the PLP is designed to accomplish also parallel layers, being the SLFN its particular case. This paper explores a particular PLP configuration which considers a nonlinear layer in parallel with a linear layer. For n inputs and m nonlinear neurons, it provides ( n + 1 ) m linear parameters, while the SLFN would have only m linear parameters (one for each hidden neuron). Since the ELM is based on adjusting only the linear parameters using the least squares estimate (LSE), the PLP network provides more freedom for the proper adjustment. Results from 12 regression and 6 classification problems are presented considering the training and test errors, the linear vector norm and the system condition number. They point out that the PLP-ELM framework is more efficient than the SLFN-ELM approach.

Resting state functional Magnetic Resonance Imaging (rs-fMRI) is increasingly used for the identification of image biomarkers of brain diseases or psychiatric conditions, such as Schizophrenia. The machine learning approach followed in this paper consists in performing feature extraction and subsequent classification experiments. Feature extraction methods that preserve spatial information allow to recover the anatomical localization of the voxels that provide discriminant information. Such locations may be further studied to assess their biological meaning as biomarkers for the disease. The power of this approach lies in the predictive accuracy of the classifier, so that features leading to higher accuracy results are assumed to have greater value as biomarkers. In this paper we apply this approach to brain local activity measures computed over rs-fMRI data from Schizophrenia patients and healthy control subjects obtained from a publicly available database (COBRE), which allows for the confirmation or falsification of our results. The extensive experimental work provides evidence that local activity measures, such as Regional Homogeneity (ReHo), may be useful for the intended purposes.

As many devices equipped with various sensors have recently proliferated, the fusion methods of various information and data from different sources have been studied. Bayesian network is one of the popular methods that solve this problem to cope with the uncertainty and imprecision. However, because a monolithic Bayesian network has high computational and design complexities, it is hard to apply to realistic problems. In this paper, we propose a modular Bayesian network system to extract context information by cooperative inference of multiple modules, which guarantees reliable inference compared to a monolithic Bayesian network without losing its strength like the easy management of knowledge and scalability. The proposed method preserves inter-modular dependencies by virtual linking and has lower computational complexity in complicated environments. The inter-modular d-separation controls local information to be delivered only to relevant modules. We verify that the proposed modular Bayesian network is enough to keep inter-modular causalities in a time-saving manner. This paper shows a possibility that a context-aware system would be easily constructed by mashing up Bayesian network fractions independently designed or leaned in different domains.

In this paper, least trimmed squares (LTS) estimators, frequently used in robust (or resistant) linear parametric regression problems, will be generalized to nonparametric LTS neural networks for nonlinear regression problems. Emphasis is put particularly on the robustness against outliers. This provides alternative learning machines when faced with general nonlinear learning problems. Simple weight updating rules based on gradient descent and iteratively reweighted least squares (IRLS) algorithms will be provided. The important parameter of trimming percentage for the data at hand can be determined by cross validation. Some simulated and real-world data will be used to illustrate the use of LTS neural networks. We will compare the robustness against outliers for usual neural networks with least squares criterion and the proposed LTS neural networks. Simulation results show that the LTS neural networks proposed in this paper have good robustness against outliers. Nonparametric least trimmed squares neural networks are proposed.We emphasize the issue of robustness against outliers.The trimming percentage for the data at hand is determined by cross validation.

RGB-D cameras are not only able to provide color (Red-Green-Blue -RGB-) information from the scene but also a relatively accurate cloud of 3D points. Using information coming from this organized cloud, it is possible to define around each image pixel a small planar patch and obtain its normal vector. Within the framework of the combinatorial pyramid, this paper describes a method to abstract from these normals to parametric surface models. The method works at two consecutive stages. Firstly, normals are hierarchically grouped to divide up the image into superpixels. These superpixels capture small patches on the scene that belong to the same surface. Then, they are merged to segment the scene into simple geometric models. Curvature information and model information are used to divide up the image into planes, cylinders and/or spheres. This paper shows how, in the higher levels of abstraction of the combinatorial pyramid, scenes can be described using these geometric items and their topological relationships.

The performance of geomagnetic aided navigation is closely related to the selection of geomagnetic matching area. This paper mainly studies the selection method of multi-parameter geomagnetic matching area based on the projection pursuit model, and then adopts the firefly algorithm, particle swarm optimization algorithm and differential evolution algorithm to obtain the optimal projection direction. After that, we compare the optimizing results comprehensively and give the evaluation of navigation performance in geomagnetic matching area to provide the basis for the selection of matching area. The implementation results show that when there are large differences between the characteristic parameters, the computation efficiency of firefly algorithm performs significantly better than the other two optimization algorithms. And under the same experimental conditions, the optimal geomagnetic matching area obtained by the method presented in this paper has the minimum matching position error and optimal navigation performance.

The unsolicited bulk messages are widespread in the applications of short messages. Although the existing spam filters have satisfying performance, they are facing the challenge of an adversary who misleads the spam filters by manipulating samples. Until now, the vulnerability of spam filtering technique for short messages has not been investigated. Different from the other spam applications, a short message only has a few words and its length usually has an upper limit. The current adversarial learning algorithms may not work efficiently in short message spam filtering. In this paper, we investigate the existing good word attack and its counterattack method, i.e. the feature reweighting, in short message spam filtering in an effort to understand whether, and to what extent, they can work efficiently when the length of a message is limited. This paper proposes a good word attack strategy which maximizes the influence to a classifier with the least number of inserted characters based on the weight values and also the length of words. On the other hand, we also proposes the feature reweighting method with a new rescaling function which minimizes the importance of the feature representing a short word in order to require more inserted characters for a successful evasion. The methods are evaluated experimentally by using the SMS and the comment spam dataset. The results confirm that the length of words is a critical factor of the robustness of short message spam filtering to good word attack.

During the last decade, multi-label learning has attracted the attention of more and more researchers in machine learning field due to wide real-world applications. Existing approaches often predict an unseen example for all labels based on the same feature vector. However, this strategy might be suboptimal since different labels usually depend on different aspects of the feature vector. Furthermore, for each label there is close relationship between positive and negative instances, which is quite informative for classification. In this paper, we propose a new algorithm called ML-DFL, which trains a model for each label with newly constructed discriminative features. In order to form these features, we also propose a spectral clustering algorithm SIA to find the closely located local structures between positive and negative instances, which are assumed to be of more discriminative information, and then transform the original data set by consulting the clustering results in a simple but effective way. Comprehensive experiments are conducted on a collection of benchmark data sets. The results clearly validate the superiority of ML-DFL to various competitors. This paper proposes a novel multi-label learning algorithm, Multi-label Learning with Discriminative Features for each Label (ML-DFL), based on binary relevance by exploiting the correlations between positive and negative instances for each label.To investigate the correlations as stated in Point 1, we propose a new spectral clustering algorithm Spectral Instance Alignment (SIA).Substantial experiments across six data sets from three different application domains validate the superiority of our proposed ML-DFL to five competitors.

Shadows are physical phenomena observed in most natural scenes. Shadows in an image can cause segmentation, tracking, or recognition algorithms to fail. In this paper, we give a research on the assumptions introduced in most researches about the complexity of shadow removal from a single image and the classification of shadow images. The purpose of this paper is twofold: in the first place, it provides a comprehensive survey of the problems and challenges which may occur when removing shadows from a single image. In the second place, an idea of shadow removal is presented, in which an attempt is made to overcome some of the fundamental problems such as illumination, color constancy, and type of reflection. Experimental results show that the proposed method can detect and eliminate the shadows effectively.

The novel method of deception detecting based on speech signal is proposed in this study. Extracting prosodic and non-linear dynamics (NLD) feature sets from speech signal and applying relevance vector machine (RVM) classification method is the primary target of this paper. Here, the sustained speaker-depended phonation samples of deception and non-deception were applied. In this paper, 30 prosodic features and 18 NLD features were selected which show significant correlations to deception state. Moreover, the RVM classification model based on sparse bayesian learning (SBL) was introduced which is a bayesian extension of the support vector machine (SVM) and not be restricted by the Mercer׳s condition. In the experiments, the deception corpus of Soochow University is exploited to test our approach. Firstly, the experiment of feature performance test was carried out. It is demonstrated that the combination of prosodic and NLD features is the most effectively method for detecting deceptive speech. Secondly, the property of RVM classification model is measured. The experiment results show that RVM technology requires much fewer basic functions and demands much less decision time than the SVM algorithm. Furthermore, during the test of correct classification accuracy of RVM, it shows that RVM is much higher than the classical model of SVM and radial basis function neural network (RBFNN). Finally, the robustness test of RVM is processed and received better performance than SVM and RBFNN methods when the signal to noise ratio (SNR) falls. In general, the proposed deception detecting method based on the combined features and RVM classifier is novel, convenient and practical, moreover, it achieves higher classification accuracy, less detection time, property of generalization, and the strong robustness.

The purpose of this paper is to exhibit the internal process of the Growing Functional Modules (GFM) controllers. These learning based controllers are graphically designed by the user, interconnecting four kinds of components: Global Goals, Acting Modules, Sensing Modules and Sensations. Global Goals specify the intrinsic motivations, Acting and Sensing Modules develop their respective functionalities while Sensations provide the controlled system׳s feedback. GFM Controllers engage in a learning process to satisfy the specified motivations while interacting with the environment and thus can be considered as artificial brains. The traditional systems programming task is replaced by a design phase that consists in graphically configuring and interconnecting the GFM components. This paper gives a description of the aforementioned components and of their specific roles in the learning based control process. The behavior of the controller and its components is exemplified with the characterization of a simple control application.

In this paper we propose a novel Computational Attention Models (CAM) that fuses bottom-up, top-down and salient motion visual cues to compute visual salience in surveillance videos. When dealing with a number of visual features/cues in a system, it is always challenging to combine or fuse them. As there is no commonly agreed natural way of combining different conspicuity maps obtained from different features: face and motion for example, the challenge is thus to find the right mix of visual cues to get a salience map that is the closest to a corresponding gaze map? In the literature many CAMs have used fixed weights for combining different visual cues. This is computationally attractive but is a very crude way of combining the different cues. Furthermore, the weights are typically set in an ad hoc fashion. Therefore in this paper, we propose a machine learning approach, using an Artificial Neural Network (ANN) to estimate these weights. The ANN is trained using gaze maps, obtained by eye tracking in psycho-physical experiments. These weights are then used to combine the conspicuities of the different visual cues in our CAM, which is later applied to surveillance videos. The proposed model is designed in a way to consider important visual cues typically present in surveillance videos, and to combine their conspicuities via ANN. The obtained results are encouraging and show a clear improvement over state-of-the-art CAMs.

This paper presents a system of smart home for home security. Many previous papers on smart home try to address this issue, but most of the published systems rely on various sensors. With the development of video technology, video based smart home becomes more attractive, but there are few literatures discussing a complete system including system architecture, feature extraction, event modeling, decision making and final information retrieval. Motivated by this challenge, we propose a novel systematical video analysis architecture, providing rich source of information about the home environment. The proposed ISEE Smart Home (ISH) is capable of analyzing various abnormal behaviors, providing realtime alarm generation, flexible video retrieval and video synopsis. The contributions can be summarized as follows: (1) we propose and build a complete smart home system based on the intelligent video analysis for home security. (2) We propose a novel foreground segmentation method based on the history pattern, which is very suitable for indoor scene analysis. (3) An exact cutting based Region of Interest (ROI) generation and scene structure modeling method is proposed for fast human detection. (4) A concurrence relation graph of visual words is developed for reliable behavior analysis. The experimental results indicate that the ISH can provide the reliable assistance for better life.

In this paper, we propose a two-stage algorithm for real-time fault detection and identification of industrial plants. Our proposal is based on the analysis of selected features using recursive density estimation and a new evolving classifier algorithm. More specifically, the proposed approach for the detection stage is based on the concept of the density in the data space, which is not the same as the probability density function, but is a very useful measure for abnormality/outliers detection. This density can be expressed by a Cauchy function and can be calculated recursively, which makes it memory and computational power efficient and, therefore, applicable to on-line applications. The identification/diagnosis stage is based on a self-developing (evolving) fuzzy-rule-based classifier system proposed in this paper, called the AutoClass. An important property of AutoClass is that it can start learning "from scratch". Not only do the fuzzy rules not need to be prespecified, but neither do the number of classes for AutoClass (the number may grow, with new class labels being added by the online learning process), in a fully unsupervised manner. In the event that an initial rule base exists, AutoClass can evolve/develop it further based on the newly arrived faulty state data. In order to validate our proposal, we present experimental results from a level control didactic process, where control and error signals are used as features for the fault detection and identification system, but the approach is generic and the number of features can be significant due to the computationally lean methodology, since covariance or more complex calculations, as well as storage of old data, are not required. The obtained results are significantly better than the traditional approaches.

This paper provides a simple forecasting framework for nonlinear and non-stationary time series using Wavelet based nonlinear models. The proposed method exploits the ability of wavelets to detect non-stationarities that may be present in a given time series in combination with higher order nonlinear Volterra Models. The utility of the proposed model is verified using two examples: the first based on a synthetically generated times series with nonlinear and non stationary features; the second case study examined in the paper pertains to forecasting of number of pilgrims visiting the well known religious shrine at Katra in the state of Jammu and Kashmir in India. Further, the proposed model was applied to 3 time series from M3 competition. The results show that the proposed models perform better when compared with the performance of some well known benchmark models. The long term predictive capability of the wavelet based nonlinear models has also been studied separately. Developed Wavelet based model for nonlinear and non-stationary time series.Tested using the tourism time series and other well known time series.Wavelet based model performs better than other benchmark models.

The emergence of ubiquitous sources of streaming data has given rise to the popularity of algorithms for online machine learning. In that context, Hoeffding trees represent the state-of-the-art algorithms for online classification. Their popularity stems in large part from their ability to process large quantities of data with a speed that goes beyond the processing power of any other streaming or batch learning algorithm. As a consequence, Hoeffding trees have often been used as base models of many ensemble learning algorithms for online classification. However, despite the existence of many algorithms for online classification, ensemble learning algorithms for online regression do not exist. In particular, the field of online any-time regression analysis seems to have experienced a serious lack of attention. In this paper, we address this issue through a study and an empirical evaluation of a set of online algorithms for regression, which includes the baseline Hoeffding-based regression trees, online option trees, and an online least mean squares filter. We also design, implement and evaluate two novel ensemble learning methods for online regression: online bagging with Hoeffding-based model trees, and an online RandomForest method in which we have used a randomized version of the online model tree learning algorithm as a basic building block. Within the study presented in this paper, we evaluate the proposed algorithms along several dimensions: predictive accuracy and quality of models, time and memory requirements, bias–variance and bias–variance–covariance decomposition of the error, and responsiveness to concept drift.

In this paper, an effective hybrid optimization strategy by incorporating the adaptive optimization of particle swarm optimization (PSO) into genetic algorithm (GA), namely HPSOGA, is used for determining the parameters of radial basis function neural networks (number of neurons, their respective centers and radii) automatically. While this task depends upon operator׳s experience with trial and error due to lack of prior knowledge, or based on gradient algorithms which are highly dependent on initial values. In this paper, hybrid evolutionary algorithms are used to automatically build a radial basis function neural networks (RBF-NN) that solves a specified problem, related to rainfall forecasting in this case. In HPSOGA, individuals in a new generation are created through three approaches to improve the global optimization performance, which are elitist strategy, PSO strategy and GA strategy. The upper-half of the best-performing individuals in a population are regarded as elites, whereas the half of the worst-performing individuals are regarded as a swarm. The group constituted by the elites are enhanced by selection, crossover and mutation operation on these enhanced elites. HPSOGA is applied to RBF-NN design for rainfall prediction. The performance of HPSOGA is compared to pure GA in these basis function neural networks design problems, showing that the hybrid strategy is of more effective global exploration ability and to avoid premature convergence. Our findings reveal that the hybrid optimization strategy proposed here may be used as a promising alternative forecasting tool for higher forecasting accuracy and better generalization ability.

The principal challenge in biometric authentication is to mitigate the effects of any interference while extracting the biometric features for biometric template generation. Most biometric systems are developed under the assumption that extracted biometrics and the nature of their associated interferences are linear, stationary, and homogeneous. The performance of biometric authentication deteriorates when the underlying assumptions are violated due to nonlinear, nonstationary, and heterogeneous noise. Therefore, a more sophisticated filtering method needs to be developed to deal with these challenges. In this paper, a new Sequential Subspace Estimator (SSE) algorithm for biometric authentication is proposed. In the proposed method, a sequential estimator is being designed in the image subspace which addresses the challenges due to nonlinear, nonstationary, and heterogeneous noise. Furthermore, the proposed method includes a subspace technique that overcomes the computational complexity associated with the sequential estimator. The theoretical foundation of the proposed method along with the experimental results is also presented in this paper. For the experimental evaluation of the proposed method, we use facial images from two public databases: the “Put Face Database” and the “Indian Face Database”. The experimental results demonstrate the superiority of the proposed method in comparison with its counterparts.

This paper proposes a methodology for identifying data samples that are likely to be mislabeled in a c-class classification problem (dataset). The methodology relies on an assumption that the generalization error of a model learned from the data decreases if a label of some mislabeled sample is changed to its correct class. A general classification model used in the paper is OP-ELM; it also provides a fast way to estimate the generalization error by PRESS Leave-One-Out. It is tested on two toy datasets, as well as on real life datasets for one of which expert knowledge about the identified potential mislabels has been sought.

This paper focuses on a new kind of artificial neural networks – the Laplace transform artificial neural networks (LTANN). It is proposed to use the Laplace transform instead of ordinary weights and a linear activation function of an artificial neuron. This extension allows to use artificial neural networks in new areas. The ordinary description of artificial neural networks is a special case of the description proposed in this paper. Three models of different geological structures based on the LTANN are discussed in this paper. Using these models, it is possible to classify an unknown geological structure as the structure without anomaly, the structure with a sinkhole or the structure with a loose zone.

Visual simultaneous localization and mapping (VSLAM) is becoming increasingly popular in research and industry as a solution for mapping an unknown environment with moving cameras. However, classic methods such as the Extended Kalman Filter (EKF)-based VSLAM have two significant limitations: First, their robustness and accuracy drop dramatically when low frame rate cameras are used or sudden changes in camera velocity occur. Second, their dynamic models are expensive to build, or are too simple to simulate complex movements. In this paper, a novel VSLAM approach called conditional simultaneous localization and mapping (C-SLAM) is proposed in which camera state transition is derived from image data using optical flow constraints and epipolar geometry in the prediction stage. This improvement not only increases prediction accuracy but also replaces commonly used predefined dynamic models which require additional computation. Compared to classic VSLAM approaches, C-SLAM performs more accurately in prediction and has high computational efficiency, especially under conditions such as abrupt changes in camera velocity or low camera frame rate. Such advantages are supported by the experimental results and analysis presented in this paper.

This paper proposes a novel optimization method that utilizes a multi-population parallel genetic algorithm (GA) to simultaneously optimize the structure and system latency of evolvable block-based neural networks (BbNNs). In the past, BbNNs have been successfully deployed for various kinds of classification problems with promising results. However, existing BbNN architectures do not explicitly model or optimize the latency of the system. Optimization of system latency is important, because it improves performance, reduces power consumption, and allows a more deterministic behavior that is needed for recurrent modes. The BbNN hardware model proposed in this paper is implemented as a synchronous architecture with registered outputs. This architecture will permit the system latency to be deterministic. To facilitate the automated layout of the neuron block interconnects within the BbNN structure, an array interconnect algorithm is presented. The proposed latency-optimized BbNN is implemented in hardware, and the architecture resembles mobile devices with integrated graphics processing units (GPUs). The hardware implementation is verified and analyzed using various case studies. The power consumption of our prototyping platform was measured, and it is shown that approximately 364 nJ of energy can be saved for every computational cycle (latency) reduced. All presented case studies show that the proposed BbNN model provides an optimal latency value while still matching the high accuracy reported in previous works.

In this paper strategies for the A-Team with Reinforcement Learning (RL) for solving the Resource Constrained Project Scheduling Problem (RCPSP) are proposed and experimentally validated. The RCPSP belongs to the NP-hard problem class. To solve this problem a team of asynchronous agents (A-Team) has been implemented using the JABAT multiagent system. An A-Team is the set of objects including multiple agents and the common memory which through interactions produce solutions of optimization problems. These interactions are usually managed by a static strategy. In this paper the dynamic learning strategies are suggested. The proposed strategies based on reinforcement learning supervising interactions between optimization agents and the common memory. To validate the approach and compare strategies computational experiment has been carried out.

A new fault detection scheme based on the proposed robust one class support vector machine (1-class SVM) is constructed in this paper. 1-class SVM is a special variant of the general support vector machine (SVM) and since only the normal data is required for training, 1-class SVM is widely used in anomaly detection. However, experiments show that 1-class SVM is sensitive to the outliers included in the training data set. To cope with this problem, a robust 1-class SVM is proposed in this paper. With the designed penalty factors, the robust 1-class SVM can depress the influences of outliers. Fault detection scheme is constructed based on the robust 1-class SVM. The simulation example shows that the robust 1-class SVM is superior to the general 1-class SVM, especially when the training data set is corrupted by outliers, and the fault detection scheme based on robust 1-class SVM presents satisfactory performances.

The increasing number of available collaborative tools and their extensive use in many organizational activities has constantly raised the complexity of collaboration engineering. It presumes the design of group decision processes, supported by a wide-range of groupware tools, in an ill-structured, dynamic, and open environment. As many of these processes are recurring by nature, the development of a shared repository to store the collective knowledge and experiences of group decision process designs became a core research topic of collaboration engineering in last few years. The paper presents a human-computer interaction engineering approach to design a software prototype that provides personalized, contextual and actionable recommendations for this problem. The approach emphasizes the computational aspects of collective intelligence, inspired from the stigmergic system designs, to structure these recommendations based on the collective knowledge that reflects not only the design space per se, but the collective experience in exploiting it as well. It is demonstrated by (1) detailing the engineering issues of an implemented prototype for the group decision process design; and (2) explaining its functionalities through a representative set of interaction scenarios. The paper covers the methodological, theoretical and practical aspects of engineering the above mentioned issues.