Search

Highlights • Development of a biological neural network model that allows long term simulation of brain activity. • Optimization of the model using multiple techniques that led to a speed-up of X200. • Presentation of alternative simulation frameworks for long term simulations., Simulations of neural networks can be used to study the direct effect of internal or external changes on brain dynamics. However, some changes are not immediate but occur on the timescale of weeks, months, or years. Examples include effects of strokes, surgical tissue removal, or traumatic brain injury but also gradual changes during brain development. Simulating network activity over a long time, even for a small number of nodes, is a computational challenge. Here, we model a coupled network of human brain regions with a modified Wilson-Cowan model representing dynamics for each region and with synaptic plasticity adjusting connection weights within and between regions. Using strategies ranging from different models for plasticity, vectorization and a different differential equation solver setup, we achieved one second runtime for one second biological time.

In this paper, we propose a control framework for intelligent surgical robots under the Remote Center of Motion (RCM). The goal of a surgical robot is to assist surgeons in performing complex surgeries. RCM constraint implies that the surgical tip attached to the end-effector of the surgical robot does not slide away from the point of the incision while performing surgery. Implementation of a control algorithm to comply with RCM constraints is a complicated task because of the nonlinear model of the surgical robots and stringent conditions of accuracy imposed by the patient’s safety. This paper proposes an optimization-driven approach to perform the surgical maneuver under RCM constraints. We then applied a bio-inspired optimization algorithm to solve the problem efficiently. For testing the performance of ZNNBAS, we used MATLAB to simulate a surgical procedure. A 7-DOF surgical robot (KUKA LBR IIWA 7) was used as a test bench for running the simulations. The simulation results show that the ZNNBAS is comparable with BAS, PSO, and GA and efficiently and robustly performed the task commanded maneuvers while enforcing the RCM constraints.

Quite often a machine learning problem lends itself to be split in several well-defined subproblems, or tasks. The goal of Multi-Task Learning (MTL) is to leverage the joint learning of the problem from two different perspectives: on the one hand, a single, overall model, and on the other hand task-specific models. In this way, the found solution by MTL may be better than those of either the common or the task-specific models. Starting with the work of Evgeniou et al., support vector machines (SVMs) have lent themselves naturally to this approach. This paper proposes a convex formulation of MTL for the L1-, L2- and LS-SVM models that results in dual problems quite similar to the single-task ones, but with multi-task kernels; in turn, this makes possible to train the convex MTL models using standard solvers. As an alternative approach, the direct optimal combination of the already trained common and task-specific models can also be considered. In this paper, a procedure to compute the optimal combining parameter with respect to four different error functions is derived. As shown experimentally, the proposed convex MTL approach performs generally better than the alternative optimal convex combination, and both of them are better than the straight use of either common or task-specific models, With partial support from Spain’s grant TIN2016-76406-P. Work supported also by the UAM–ADIC Chair for Data Science and Machine Learning.

In the process of Human-robot skill transfer, we require the robot to reproduce the trajectory of teacher and expect that the robot can generalize the learned trajectory. For the trajectory after generalization, we expect that the robot arm can accurately track. However, because the model of the robot can not be accurately obtained, some researchers have proposed using a neural network to approximate the unknown term. The parameters of the traditional RBF neural network are usually selected through the empirical and trial-and-error method, which maybe biased and inefficient. In addition, due to the end-effector of the mechanical arm trajectory will be constantly changing according to the needs of the task, when the neural network of compact set cannot contain the whole input vector, the neural network cannot achieve the ideal approximation effect. In this paper, the broad neural network is used to approximate the unknown terms of the robot. This method can reuse the motion controller that has been learned and complete other motions in the robot operating space without relearning its weight parameters. In this paper, the effectiveness of the proposed method is proved by the ultrasound scanning task.

This paper proposes a hybrid architecture based on neural networks, fuzzy systems, and n-uninorms for solving pattern classification problems, termed as ENFS-Uni0 (short for evolving neuro-fuzzy system based on uni-nullneurons). The model can produce knowledge in an on-line (single-pass) and evolving learning context in a particular form of neuro-fuzzy rules representing the dependencies among input features through IF-THEN type relations. The rules antecedents are thereby realized through uni-nullneurons, which are constructed from n-uninorms, leading to the possibility to express both, AND- and OR-connections (and a mixture of these) among the single antecedent parts of a rule (and thus achieving an advanced interpretability aspect of the rules). The neurons’ evolution is done through an extended version of an autonomous data partition method (ADPA). On-line interpretation of the timely evolution of rules is addressed by (i) a concept for tracking the degree of changes of the rules over data stream samples, which may indicate experts/operators how much dynamics is in the process and may be used as a structural active learning component to request operator’s feedback in the case of significant changes and (ii) a concept for updating feature weights incrementally. These weights express the (possibly changing) impact degrees of features on the classification problem: features with low weights can be seen as unimportant and masked out when showing rules to an expert ( → rule length reduction). The rules’ consequents are represented by certainty vectors and are recursively updated by an indicator-based recursive weighted least squares (I-RWLS) approach (one RWLS estimator per class) where the weights are given through the neuron activation levels in order to gain stable local learning. The model proposed in this paper was successfully compared to related hybrid and evolving approaches in the literature for classifying binary and multi-class patterns. The results obtained by the proposed model show an outperformance of the related works in terms of higher accuracy trend lines over time, while offering a high degree of interpretability through coherent neuro-fuzzy rules to solve the classification problems.

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering. These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue, we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different architectures as different "views" of the same data. This approach is based on the assumption that information contained in the different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets, and produces state-of-the-art results for IC., Comment: 21 pages, 16 figures, 10 tables, preprint of our paper published in Neurocomputing

In this paper, we propose a method for obtaining sentence-level embeddings. While the problem of securing word-level embeddings is very well studied, we propose a novel method for obtaining sentence-level embeddings. This is obtained by a simple method in the context of solving the paraphrase generation task. If we use a sequential encoder-decoder model for generating paraphrase, we would like the generated paraphrase to be semantically close to the original sentence. One way to ensure this is by adding constraints for true paraphrase embeddings to be close and unrelated paraphrase candidate sentence embeddings to be far. This is ensured by using a sequential pair-wise discriminator that shares weights with the encoder that is trained with a suitable loss function. Our loss function penalizes paraphrase sentence embedding distances from being too large. This loss is used in combination with a sequential encoder-decoder network. We also validated our method by evaluating the obtained embeddings for a sentiment analysis task. The proposed method results in semantic embeddings and outperforms the state-of-the-art on the paraphrase generation and sentiment analysis task on standard datasets. These results are also shown to be statistically significant., Comment: This work is an extension of our COLING-2018 paper arXiv:1806.00807

Person reidentification (ReID) refers to the task of verifying the identity of a pedestrian observed from nonoverlapping views in a surveillance camera network. It has recently been validated that reranking can achieve remarkable performance improvements in person ReID systems. However, current reranking approaches either require feedback from users or suffer from burdensome computational costs. In this paper, we propose to exploit a density-adaptive smooth kernel technique to achieve efficient and effective reranking. Specifically, we adopt a smooth kernel function to formulate the neighbor relationships among data samples with a density-adaptive parameter. Based on this new formulation, we present two simple yet effective reranking methods, termed \emph{inverse} density-adaptive kernel based reranking (inv-DAKR) and \emph{bidirectional} density-adaptive kernel based reranking (bi-DAKR), in which the local density information in the vicinity of each gallery sample is elegantly exploited. Moreover, we extend the proposed inv-DAKR and bi-DAKR methods to incorporate the available extra probe samples and demonstrate that when and why these extra probe samples are able to improve the local neighborhood and thus further refine the ranking results. Extensive experiments are conducted on six benchmark datasets, including: PRID450s, VIPeR, CUHK03, GRID, Market-1501 and Mars. The experimental results demonstrate that our proposals are effective and efficient., 39 pages, 18 figures and 12 tables. This paper is an extended version of our preliminary work on ICPR 2018

Manual annotation of video segmentation datasets requires an immense amount of human effort, thus, reduction of human annotation costs is an active topic of research. While many papers deal with the propagation of masks through frames of a video, only a few results attempt to optimize annotation task selection. In this paper we present a deep learning based solution to the latter problem and train it using Reinforcement Learning. Our approach utilizes a modified version of the Dueling Deep Q-Network sharing weight parameters across the temporal axis of the video. This technique enables the trained agent to select annotation tasks from the whole video. We evaluate our annotation task selection method by means of a hierarchical supervoxel segmentation based mask propagation algorithm.

It is well known that weighted sum of node states plays an essential role in function implementation of neural networks. Therefore, this paper proposes a new weighted sum synchronization model for memristive neural networks. Unlike the existing synchronization models of memristive neural networks which control each network node to reach synchronization, the proposed model treats the networks as dynamic entireties by weighted sum of node states and makes the entireties instead of each node reach expected synchronization. In this paper, weighted sum complete synchronization and quasi-synchronization are both investigated by designing feedback controller and aperiodically intermittent controller, respectively. Meanwhile, a flexible control scheme is designed for the proposed model by utilizing some switching parameters and can improve anti-interference ability of control system. By applying Lyapunov method and some differential inequalities, some effective criteria are derived to ensure the synchronizations of memristive neural networks. Moreover, the error level of the quasi-synchronization is given. Finally, numerical simulation examples are used to certify the effectiveness of the derived results.

Facial landmarks are highly correlated with each other since a certain landmark can be estimated by its neighboring landmarks. Most of the existing deep learning methods only use one fully-connected layer called shape prediction layer to estimate the locations of facial landmarks. In this paper, we propose a novel deep learning framework named Multi-Center Learning with multiple shape prediction layers for face alignment. In particular, each shape prediction layer emphasizes on the detection of a certain cluster of semantically relevant landmarks respectively. Challenging landmarks are focused firstly, and each cluster of landmarks is further optimized respectively. Moreover, to reduce the model complexity, we propose a model assembling method to integrate multiple shape prediction layers into one shape prediction layer. Extensive experiments demonstrate that our method is effective for handling complex occlusions and appearance variations with real-time performance. The code for our method is available at https://github.com/ZhiwenShao/MCNet-Extension., This paper has been accepted by Neurocomputing

Feature selection plays an important role in reducing irrelevant and redundant features, while retaining the underlying semantics of selected ones. An effective feature selection method is expected to result in a significantly reduced subset of the original features without sacrificing the quality of problem-solving (e.g., classification). In this paper, a non-unique decision measure is proposed that captures the degree of a given feature subset being relevant to different categories. This helps to represent the uncertainty information in the boundary region of a granular model, such as rough sets or fuzzy-rough sets in an efficient manner. Based on this measure, the paper further introduce a differentiation entropy as an evaluator of feature subsets to implement a novel feature selection algorithm. The resulting feature selection method is capable of dealing with either nominal or real-valued data. Experimental results on both benchmark data sets and a real application problem demonstrate that the features selected by the proposed approach outperform those attained by state-of-the-art feature selection techniques, in terms of both the size of feature reduction and the classification accuracy.

Facial expressions play an important role during communications, allowing information regarding the emotional state of an individual to be conveyed and inferred. Research suggests that automatic facial expression recognition is a promising avenue of enquiry in mental healthcare, as facial expressions can also reflect an individual's mental state. In order to develop user-friendly, low-cost and effective facial expression analysis systems for mental health care, this paper presents a novel deep convolution network based emotion analysis framework to support mental state detection and diagnosis. The proposed system is able to process facial images and interpret the temporal evolution of emotions through a new solution in which deep features are extracted from the Fully Connected Layer 6 of the AlexNet, with a standard Linear Discriminant Analysis Classifier exploited to obtain the final classification outcome. It is tested against 5 benchmarking databases, including JAFFE, KDEF,CK+, and databases with the images obtained ‘in the wild’ such as FER2013 and AffectNet. Compared with the other state-of-the-art methods, we observe that our method has overall higher accuracy of facial expression recognition. Additionally, when compared to the state-of-the-art deep learning algorithms such as Vgg16, GoogleNet, ResNet and AlexNet, the proposed method demonstrated better efficiency and has less device requirements. The experiments presented in this paper demonstrate that the proposed method outperforms the other methods in terms of accuracy and efficiency which suggests it could act as a smart, low-cost, user-friendly cognitive aid to detect, monitor, and diagnose the mental health of a patient through automatic facial expression analysis.

Hand gestures are quite suitable for space human-robot interaction (SHRI) because of their natural and convenient features. While the detection and localization of hands are the premise and foundation for SHRI based on hand gestures. But hand gestures are very complicated and hand sizes are very small in some images. These problems make the robust real-time hand detection and local- ization very difficult. In this paper, a feature-map-fused single shot multibox detector (FF-SSD) which is a deep learning network is designed to deal with the problems of hand detection and localization in SHRI. First, the background of the method is introduced in this paper, including an astronaut assistant robot platform, the difficulties of hand detection and localization, and introduction of the state-of-the-art deep learning networks for object detection and localiza- tion. Then, the FF-SSD is proposed for detecting and localizing hands especially pony-size hands. This network magentatakes into consideration both accuracy and speed with balanced performance. And in the experiment part, the FF-SSD is trained and tested on hand databases which include a homemade database and two public databases. At last, the superiority of the proposed method is demonstrated compared with the state-of-the-art methods.

The Denoising Autoencoder (DAE) enhances the flexibility of the data stream method in exploiting unlabeled samples. Nonetheless, the feasibility of DAE for data stream analytic deserves an in-depth study because it characterizes a fixed network capacity that cannot adapt to rapidly changing environments. Deep evolving denoising autoencoder (DEVDAN), is proposed in this paper. It features an open structure in the generative phase and the discriminative phase where the hidden units can be automatically added and discarded on the fly. The generative phase refines the predictive performance of the discriminative model exploiting unlabeled data. Furthermore, DEVDAN is free of the problem-specific threshold and works fully in the single-pass learning fashion. We show that DEVDAN can find competitive network architecture compared with state-of-the-art methods on the classification task using ten prominent datasets simulated under the prequential test-then-train protocol., This paper has been accepted for publication in Neurocomputing 2019. arXiv admin note: substantial text overlap with arXiv:1809.09081

In this paper, we address a challenging task in real-word applications, i.e., automatic reading recognition for pointer meters, called PRM. 1 This application is valuable in the fields of military, industry, and aerospace. However, the accuracy of recognizing the readings of pointer meters by machine vision is, oftentimes, affected by several factors, such as uneven illumination in each image, large range variation of illumination in different images, complex backgrounds, tilting of pointer meters, image blur, and scale change, resulting in the recognized readings with unacceptable accuracy. In this paper, a new robust approach to reading recognition of pointer meters is proposed. The proposed method consists of three main contributions: (1) constructing a novel deep learning algorithm in which the PrRoIPooling is used in lieu of the RoiAlign in the existing Mask-RCNN, (2) classifying the type of pointer meters while fitting the pointer binary mask, and (3) calculating the readings of pointer meters by the proposed angle method. In addition, we also report and release a new dataset for the community. Experiments show that the new algorithm can significantly improve the accuracy of the recognized readings of pointer meters, meanwhile, the proposed approach is also robust to the natural environments and computationally efficient.

Although supervised deep representation learning has attracted enormous attentions across areas of pattern recognition and computer vision, little progress has been made towards unsupervised deep representation learning for image clustering. In this paper, we propose a deep spectral analysis network for unsupervised representation learning and image clustering. While spectral analysis is established with solid theoretical foundations and has been widely applied to unsupervised data mining, its essential weakness lies in the fact that it is difficult to construct a proper affinity matrix and determine the involving Laplacian matrix for a given dataset. In this paper, we propose a SA-Net to overcome these weaknesses and achieve improved image clustering by extending the spectral analysis procedure into a deep learning framework with multiple layers. The SA-Net has the capability to learn deep representations and reveal deep correlations among data samples. Compared with the existing spectral analysis, the SA-Net achieves two advantages: (i) Given the fact that one spectral analysis procedure can only deal with one subset of the given dataset, our proposed SA-Net elegantly integrates multiple parallel and consecutive spectral analysis procedures together to enable interactive learning across different units towards a coordinated clustering model; (ii) Our SA-Net can identify the local similarities among different images at patch level and hence achieves a higher level of robustness against occlusions. Extensive experiments on a number of popular datasets support that our proposed SA-Net outperforms 11 benchmarks across a number of image clustering applications., Comment: arXiv admin note: text overlap with arXiv:2009.05235

The Convolutional Neural Networks (CNNs), in domains like computer vision, mostly reduced the need for handcrafted features due to its ability to learn the problem-specific features from the raw input data. However, the selection of dataset-specific CNN architecture, which mostly performed by either experience or expertise is a time-consuming and error-prone process. To automate the process of learning a CNN architecture, this paper attempts at finding the relationship between Fully Connected (FC) layers with some of the characteristics of the datasets. The CNN architectures, and recently datasets also, are categorized as deep, shallow, wide, etc. This paper tries to formalize these terms along with answering the following questions. (i) What is the impact of deeper/shallow architectures on the performance of the CNN w.r.t. FC layers?, (ii) How the deeper/wider datasets influence the performance of CNN w.r.t. FC layers?, and (iii) Which kind of architecture (deeper/ shallower) is better suitable for which kind of (deeper/ wider) datasets. To address these findings, we have performed experiments with three CNN architectures having different depths. The experiments are conducted by varying the number of FC layers. We used four widely used datasets including CIFAR-10, CIFAR-100, Tiny ImageNet, and CRCHistoPhenotypes to justify our findings in the context of the image classification problem. The source code of this research is available at https://github.com/shabbeersh/Impact-of-FC-layers., This paper is accepted for publication in Neurocomputing Journal

In this paper, we focus on synchronization issue of coupled multistable memristive neural networks (CMMNNs) with time delay under multiple stable equilibrium states. First, we build delayed CMMNNs consisting of one master subnetwork without controller and N−1 identical slave subnetworks with controllers, and every subnetwork has n nodes. Moreover, this paper investigates multistability of delayed CMMNNs with continuous nonmonotonic piecewise linear activation function (PLAF) owning 2 r + 2 corner points. By using the theorems of differential inclusion and fixed point, sufficient conditions are derived such that master subnetwork of CMMNNs can acquire ( r + 2 ) n exponentially stable equilibrium points, stable periodic orbits or hybrid stable equilibrium states. Then, this paper proposes hybrid multisynchronization of delayed CMMNNs related with various external inputs under multiple stable equilibrium states for the first time. There exist ( r + 2 ) n hybrid multisynchronization manifolds in CMMNNs with different initial conditions and external inputs. Finally, two numerical simulations are given to illustrate the effectiveness of the obtained results.

This paper addresses two issues toward practical use of wearable electroencephalogram (EEG) measurement devices. Ocular (eye movement and blink) artifacts often contaminate EEGs and deteriorate the performance of EEG-based brain–computer interfaces (BCIs). Although wearable consumer-grade EEG devices with single electrode allow users to operate BCIs conveniently in daily lives, it remains a challenging issue to attenuate ocular artifacts from single-channel measurements without spatial information. Existing ocular artifact reduction methods are, however, not simple enough for single-channel EEG data in the sense that they require an additional reference channel and/or pre-processing for artifact segment detection. Another issue is how to assess the performance of artifact reduction; the existing studies have used their own datasets that are not accessible from other researchers. Then, this paper makes two major contributions. (1) This paper proposes a novel ocular artifact reduction method, multi-scale dictionary learning (MSDL), which operates under single-channel measurements and without artifact segment detection. (2) We also develop a semi-simulation setting for quantitative evaluation with a publicly available EEG dataset. In particular, we employed BCI Competition IV Dataset 2a, on which the proposed method was compared with state-of-art methods. The proposed technique showed the best performance for recovering artifact-reduced waveforms from single-channel data compared to the other artifact reduction methods. The Matlab scripts for semi-simulation data generation and single-channel artifact reduction are available on GitHub .

In online-learning, the data is incrementally received and the distributions from which it is drawn may keep changing over time. This phenomenon is widely known as concept drift. Such changes may affect the generalization of a learned model to future data. This problem may be exacerbated by the form of the drift itself changing over time. Quantitative measures to describe and analyze the concept drift have been proposed in previous work. A description composed from these measures is called a concept drift map. We believe that these maps could be useful for guiding how much knowledge in the old model should be forgotten. Therefore, this paper presents an adaptive online learning model that uses a concept drift map to regulate the forgetting factor of an extreme learning machine. Specifically, when a batch of new instances are labeled, the distribution of each class on each attribute is firstly estimated, and then it is compared with the distribution estimated in the previous batch to calculate the magnitude of concept drift, which is further used to regulate the forgetting factor and to update the learning model. Therefore, the novelty of this paper lies in that a quantitative distance metric between two distributions constructed on continuous attribute space is presented to construct concept drift map which can be further associated with the forgetting factor to make the learning model adapt the concept drift. Experimental results on several benchmark stream data sets show the proposed model is generally superior to several previous algorithms when classifying a variety of data streams subject to drift, indicating its effectiveness and feasibility.

Currently, knowledge discovery in databases is an essential first step when identifying valid, novel and useful patterns for decision making. There are many real-world scenarios, such as bankruptcy prediction, option pricing or medical diagnosis, where the classification models to be learned need to fulfill restrictions of monotonicity (i.e. the target class label should not decrease when input attributes values increase). For instance, it is rational to assume that a higher debt ratio of a company should never result in a lower level of bankruptcy risk. Consequently, there is a growing interest from the data mining research community concerning monotonic predictive models. This paper aims to present an overview of the literature in the field, analyzing existing techniques and proposing a taxonomy of the algorithms based on the type of model generated. For each method, we review the quality metrics considered in the evaluation and the different data sets and monotonic problems used in the analysis. In this way, this paper serves as an overview of monotonic classification research in specialized literature and can be used as a functional guide for the field.

Twin Support Vector Machines (TWSVMs) have emerged as an efficient alternative to Support Vector Machines (SVM) for learning from imbalanced datasets. The TWSVM learns two non-parallel classifying hyperplanes by solving a couple of smaller sized problems. However, it is unsuitable for large datasets, as it involves matrix operations. In this paper, we discuss a Twin Neural Network (Twin NN) architecture for learning from large unbalanced datasets. The objective functions of the networks in the Twin NN are designed to realize the idea of the Twin SVM with non-parallel decision boudaries for the respective classes, while also being able to reduce model complexity. The Twin NN optimizes the feature map, allowing for better discrimination between classes. The paper also discusses an extension of the Twin NN for multiclass datasets. This architecture trains as many neural networks as the number of classes, and has the additional advantage that it does not have any hyper-parameter which requires tuning. Results presented in the paper demonstrate that the Twin NN generalizes well and scales well on large unbalanced datasets.

In this paper, we study the problem of matrix recovery, which aims to restore a target matrix of authentic samples from grossly corrupted observations. Most of the existing methods, such as the well-known Robust Principal Component Analysis (RPCA), assume that the target matrix we wish to recover is low-rank. However, the underlying data structure is often non-linear in practice, therefore the low-rankness assumption could be violated. To tackle this issue, we propose a novel method for matrix recovery in this paper, which could well handle the case where the target matrix is low-rank in an implicit feature space but high-rank or even full-rank in its original form. Namely, our method pursues the low-rank structure of the target matrix in an implicit feature space. By making use of the specifics of an accelerated proximal gradient based optimization algorithm, the proposed method could recover the target matrix with non-linear structures from its corrupted version. Comprehensive experiments on both synthetic and real datasets demonstrate the superiority of our method.

Feature selection is an important step in building classifiers for high-dimensional data problems, such as EEG classification for BCI applications. This paper proposes a new wrapper method for feature selection, based on a multi-objective evolutionary algorithm, where the representation of the individuals or potential solutions, along with the breeding operators and objective functions, have been carefully designed to select a small subset of features that has good generalization capability, trying to avoid the over-fitting problems that wrapper methods usually suffer. A novel feature ranking procedure is also proposed in order to analyze the stability of the proposed wrapper method. Four different classification schemes have been applied within the proposed wrapper method in order to evaluate its accuracy and stability for feature selection on a real motor imagery dataset. Experimental results show that the wrapper method presented in this paper is able to obtain very small subsets of features, which are quite stable and also achieve high classification accuracy, regardless of the classifiers used., Project TIN2015-67020-P (Spanish “Ministerio de Economía y Competitividad”), European Regional Development Funds (ERDF)

It is a big challenge of computer vision to make machine automatically describe the content of an image with a natural language sentence. Previous works have made great progress on this task, but they only use the global or local image feature, which may lose some important subtle or global information of an image. In this paper, we propose a model with 3-gated model which fuses the global and local image features together for the task of image caption generation. The model mainly has three gated structures. 1) Gate for the global image feature, which can adaptively decide when and how much the global image feature should be imported into the sentence generator. 2) The gated recurrent neural network (RNN) is used as the sentence generator. 3) The gated feedback method for stacking RNN is employed to increase the capability of nonlinearity fitting. More specially, the global and local image features are combined together in this paper, which makes full use of the image information. The global image feature is controlled by the first gate and the local image feature is selected by the attention mechanism. With the latter two gates, the relationship between image and text can be well explored, which improves the performance of the language part as well as the multi-modal embedding part. Experimental results show that our proposed method outperforms the state-of-the-art for image caption generation., Comment: 35 pages, 7 figures, magazine

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and weight sharing, can reduce the number of parameters and increase processing speed during training and inference. However, as the dimension of data becomes higher and the CNN architecture becomes more complicated, the end-to-end approach or the combined manner of CNN is computationally intensive, which becomes limitation to CNN’s further implementation. Therefore, it is necessary and urgent to implement CNN in a faster way. In this paper, we first summarize the acceleration methods that contribute to but not limited to CNN by reviewing a broad variety of research papers. We propose a taxonomy in terms of three levels, i.e. structure level, algorithm level, and implementation level, for acceleration methods. We also analyze the acceleration methods in terms of CNN architecture compression, algorithm optimization, and hardware-based improvement. At last, we give a discussion on different perspectives of these acceleration and optimization methods within each level. The discussion shows that the methods in each level still have large exploration space. By incorporating such a wide range of disciplines, we expect to provide a comprehensive reference for researchers who are interested in CNN acceleration.

This paper studies the consensus of multi-agent systems with faults and mismatches under switched topologies using a delta operator method. Since faults and mismatches can result in failure of the consensus even for a fixed topology with a spanning tree, how to reach a consensus is a complicated and challenging problem under such circumstances especially when part topologies have no spanning tree. Although some works studied the influence of faults and mismatches on the consensus, there is little work on reaching a consensus for the multi-agent systems with faults and mismatches. In this paper, we introduce the delta operator to unify the consensus analysis for continuous, discrete, or sampled systems under one framework. We develop the theories on the delta operator systems first and then apply theories of the delta operator systems to the consensus problems. By converting the consensus problems into stability problems, we investigate and prove consensus and the associated conditions for systems 1) without any fault, 2) with a known fault, and 3) with unknown faults, under switching topologies with matching or mismatching coefficients. Numerical examples are provided and validate the effectiveness of the theoretical results.

This work addresses the coordination problem of multiple robots with the goal of finding specific hazardous targets in an unknown area and dealing with them cooperatively. The desired behaviour for the robotic system entails multiple requirements, which may also be conflicting. The paper presents the problem as a constrained bi-objective optimization problem in which mobile robots must perform two specific tasks of exploration and at same time cooperation and coordination for disarming the hazardous targets. These objectives are opposed goals, in which one may be favored, but only at the expense of the other. Therefore, a good trade-off must be found. For this purpose, a nature-inspired approach and an analytical mathematical model to solve this problem considering a single equivalent weighted objective function are presented. The results of proposed coordination model, simulated in a two dimensional terrain, are showed in order to assess the behaviour of the proposed solution to tackle this problem. We have analyzed the performance of the approach and the influence of the weights of the objective function under different conditions: static and dynamic. In this latter situation, the robots may fail under the stringent limited budget of energy or for hazardous events. The paper concludes with a critical discussion of the experimental results., Comment: 40 pages

Activation function is crucial to the recent successes of deep neural networks. In this paper, we first propose a new activation function, Multiple Parametric Exponential Linear Units (MPELU), aiming to generalize and unify the rectified and exponential linear units. As the generalized form, MPELU shares the advantages of Parametric Rectified Linear Unit (PReLU) and Exponential Linear Unit (ELU), leading to better classification performance and convergence property. In addition, weight initialization is very important to train very deep networks. The existing methods laid a solid foundation for networks using rectified linear units but not for exponential linear units. This paper complements the current theory and extends it to the wider range. Specifically, we put forward a way of initialization, enabling training of very deep networks using exponential linear units. Experiments demonstrate that the proposed initialization not only helps the training process but leads to better generalization performance. Finally, utilizing the proposed activation function and initialization, we present a deep MPELU residual architecture that achieves state-of-the-art performance on the CIFAR-10/100 datasets. The code is available at https://github.com/Coldmooon/Code-for-MPELU .

Pedestrian detection and tracking have become an important field in the computer vision research area. This growing interest, started in the last decades, might be explained by the multitude of potential applications that could use the results of this research field, e.g. robotics, entertainment, surveillance, care for the elderly and disabled, and content-based indexing. In this survey paper, vision-based pedestrian detection systems are analysed based on their field of application, acquisition technology, computer vision techniques and classification strategies. Three main application fields have been individuated and discussed: video surveillance, human-machine interaction and analysis. Due to the large variety of acquisition technologies, this paper discusses both the differences between 2D and 3D vision systems, and indoor and outdoor systems. The authors reserved a dedicated section for the analysis of the Deep Learning methodologies, including the Convolutional Neural Networks in pedestrian detection and tracking, considering their recent exploding adoption for such a kind systems. Finally, focusing on the classification point of view, different Machine Learning techniques have been analysed, basing the discussion on the classification performances on different benchmark datasets. The reported results highlight the importance of testing pedestrian detection systems on different datasets to evaluate the robustness of the computed groups of features used as input to classifiers.

The paper considers a class of nonsmooth neural networks where hard-limiter saturation nonlinearities are used to constrain solutions of a linear system with concentrated and distributed delays to evolve within a closed hypercube of R n . Such networks are termed delayed linear systems in saturated mode (D-LSSMs) and they are a generalization to the delayed case of a relevant class of neural networks previously introduced in the literature. The paper gives a rigorous foundation to the D-LSSM model and then it provides a fundamental result on convergence of solutions toward equilibrium points in the case where there are nonsymmetric cooperative (nonnegative) interconnections between neurons. The result ensures convergence for any finite value of the maximum delay and is physically robust with respect to perturbations of the interconnections. More importantly, it encompasses situations where there exist multiple stable equilibria, thus guaranteeing multistability of cooperative D-LSSMs. From an application viewpoint the delays in combination with the property of multistability make D-LSSMs potentially useful in the fields of associative memories, motion detection and processing of temporal patterns.

Computer simulation, which has been proved to be an effective approach to problem solving, is nowadays widely used in modern science. However, it requires a lot of computing resources, which are difficult for general users to acquire. In this paper, we design a Web based system to implement on-line simulation system for ordinary users. As a useful example, the simulation of one type of collision detection model is presented in this paper. Moreover, the software application of simulation is offered as a service on Web. Meanwhile, the incorporation of general vector machine (GVM, a type of neural network) to intelligently predict the relationship between simulation parameters and computation resources is presented, which could further provide more information for system monitoring and scheduling. The system has demonstrated efficiency and intuitiveness for users of this type of applications.

BrainComputer Interface (BCI) systems analyze brain signals to generate control commands for computer applications or external devices. Utilized as alternative communication channel, BCIs have the potential to assist people with severe motor disabilities to interact with their environment and to participate in daily life activities. Handicapped people from all age groups could benefit from such BCI technologies. Although some papers have previously reported slightly worse BCI performance by older subjects, in many studies BCI systems were tested with young subjects only.In the presented paper age-associated differences in BCI performance were investigated. We compared accuracy and speed of a steady-state visual evoked potential (SSVEP)-based BCI spelling application controlled by participants of two different equally sized age groups. Twenty subjects (eleven female and nine male) participated in this study; each age group consisted of ten subjects, ranging from 19 to 27 years and from 64 to 76 years. Our results confirm that elderly people may have a deteriorated information transfer rate (ITR). The mean (SD) ITR of the young age group was 27.36 (6.50) bit/min while the elderly people achieved a significantly lower ITR of 16.10 (5.90) bit/min. The average time window length associated with the signal classification was usually larger for the participants of advanced age. These findings show that the subject age must be taken into account during the development of SSVEP-based applications.

In this paper, a self-organizing map (SOM) neural network is used to visualize corrective actions of failure modes and effects analysis (FMEA). SOM is a popular unsupervised neural network model that aims to produce a low-dimensional map (typically a two-dimensional map) for visualizing high-dimensional data. With regards to FMEA, it is a popular methodology to identify potential failure modes for a product or a process, to assess the risk associated with those failure modes, also, to identify and carry out corrective actions to address the most serious concerns. Despite the popularity of FMEA in a wide range of industries, two well-known shortcomings are the complexity of the FMEA worksheet and its intricacy of use. To the best of our knowledge, the use of computation techniques for solving the aforementioned shortcomings is limited. The use of SOM in FMEA is new. In this paper, corrective actions in FMEA are described in their severity, occurrence and detect scores. SOM is then used as a visualization aid for FMEA users to see the relationship among corrective actions via a map. Color information from the SOM map is then included to the FMEA worksheet for better visualization. In addition, a Risk Priority Number Interval is used to allow corrective actions to be evaluated and ordered in groups. Such approach provides a quick and easily understandable framework to elucidate important information from a complex FMEA worksheet; therefore facilitating the decision-making tasks by FMEA users. The significance of this study is two-fold, viz., the use of SOM as an effective neural network learning paradigm to facilitate FMEA implementations, and the use of a computational visualization approach to tackle the two well-known shortcomings of FMEA.

[EN] This paper presents an approach of a social emotional model, which allows to extract the social emotion of a group of intelligent entities. The emotional model PAD allows to represent the emotion of an intelligent entity in 3-D space, allowing the representation of different emotional states. The social emotional model presented in this paper uses individual emotions of each one of the entities, which are represented in the emotional space PAD. Using a social emotional model within intelligent entities allows the creation of more real simulations, in which emotional states can influence decision-making. The result of this social emotional mode is represented by a series of examples, which are intended to represent a number of situations in which the emotions of each individual modify the emotion of the group. Moreover, the paper introduces an example which employs the proposed model in order to learn and predict future actions trying to influence in the social emotion of a group of people., This work is partially supported by the MINECO/FEDER TIN2015-65515-C4-1-R and the FPI grant AP2013-01276 awarded to Jaime-Andres Rincon.

Dictionary learning has played an important role in the success of sparse representation. Although discriminative synthesis dictionary learning for sparse representation with a high-computational-complexity l0 or l1 norm constraint has been well studied for image classification, jointly and discriminatively learning an analysis dictionary and a synthesis dictionary is still in its infant stage. As a dual of synthesis dictionary, the recently developed analysis dictionary can provide a complementary view of data representation, which can have a much lower time complexity than sparse synthesis representation. Although several class-specific analysis-synthesis dictionary, which may have a big correlation between different classes' dictionaries, have been developed, how to learn a more compact and discriminative universal analysis-synthesis dictionary is still open. In this paper, to provide a more complete view of discriminative data representation, we propose a novel model of discriminative analysis-synthesis dictionary learning (DASDL), in which a linear classifier based on the coding coefficient is jointly learned with the dictionary pair, thus the performance of the classifier and the representational power of the dictionary pair being considered at the same time by the same optimization procedure. The size of the learned dictionaries can be very small since the analysis-synthesis dictionary is shared by all class data. An iterative algorithm to efficiently solve the proposed DASDL is presented in this paper. The experiments on face recognition, gender classification, action recognition and image classification clearly show the superiority of the proposed DASDL.

Traffic guidance and prompt information could induce the change of traffic states on road sections, and in turn the effects of these changes will be transited to their relative upstream and downstream sections, which lead to dynamic variations in traffic states of urban regional road networks. In this paper, the rule of dynamic transition in traffic state of urban road networks under the effect of traffic information is studied. Specifically, the hidden Markov model is selected to represent the dynamic transition process. Then Expectation Maximization (EM) Algorithm is presented for potential traffic state estimation. Finally, verification is carried out through simulation with Variable Message Sign (VMS) selected as the information release terminal and then a certain regional road network in Beijing is chosen as the study object. The results show that the model in this paper can describe transition process of road traffic state under the effect of VMS information, and also can be used for real-time traffic state estimation of urban road networks. The study has both theoretical and practical values in evaluation of service quality of traffic information and in making traffic dispersion and control strategies for traffic management department.

In today׳s society where audio–visual content is ubiquitous, violence detection in movies and Web videos has become a decisive functionality, e.g., for providing automated youth protection services. In this paper, we concentrate on two important aspects of video content analysis: Time efficiency and modeling of concepts (in this case, violence modeling). Traditional approaches to violent scene detection build on audio or visual features to model violence as a single concept in the feature space. Such modeling does not always provide a faithful representation of violence in terms of audio–visual features, as violence is not necessarily located compactly in the feature space. Consequently, in this paper, we target to close this gap. To this end, we present a solution which uses audio–visual features (MFCC-based audio and advanced motion features) and propose to model violence by means of multiple (sub)concepts. To cope with the heavy computations induced by the use of motion features, we perform a coarse-to-fine analysis, starting with a coarse-level analysis with time efficient audio features and pursuing with a fine-level analysis with advanced features when necessary. The results demonstrate the potential of the proposed approach on the standardized datasets of the latest editions of the MediaEval Affect in Multimedia: Violent Scenes Detection (VSD) task of 2014 and 2015.

Pedestrians, in videos taken from fixed cameras, tend to appear and disappear at precise locations such as doors, gateways or edges of the scene: we refer to locations where pedestrians appear as sources (potential origins) and the locations where they disappear as sinks (potential destinations). The detection of these points and the characterization of the flows connecting them represent a typical preliminary step in most pedestrian studies and it can be supported by computer vision approaches. In this paper we propose an algorithm in which a scene is overlaid by a grid of particles initializing a dynamical system defined by optical flow, a high level global motion information. Time integration of the dynamical system produces short particle trajectories (tracklets), representing dense but short motion patterns in segments of the scene; tracklets are then extended into longer tracks that are grouped using an unsupervised clustering algorithm, where the similarity is measured by the Longest Common Subsequence. The analysis of these clusters supports the identification of sources and sinks related to a single video segment. Local segment information is finally combined to achieve a global set of traces identifying sources and sinks, and characterizing the flow of pedestrians connecting them. The paper presents the defined technique and it discusses its application in a real-world scenario.

In this paper, we examine the problem of learning sparse representations of visual patterns in the context of artificial and biological vision systems. There are a myriad of strategies for sparse coding that often result in similar feature properties for the learned feature set. Typically this results in a bank of Gabor-like or edge filters that are sensitive to a range of distinct angular and radial frequencies. The theory and experimentation that is presented in this paper serves to provide a better understanding of a number of specific properties related to low-level feature learning. This includes close examination of the role of phase pairing in complex cells, the role of depth information and its relationship to variation of intensity and chroma, and deriving hybrid features that borrow from both analytic forms and statistical methods. Together, these specific examples provide context for more general discussion of effective strategies for feature learning. In particular, we make the case that imposing additional constraints on mechanisms for feature learning inspired by biological vision systems can be useful in guiding constrained optimization towards convergence, or specific desirable computational properties for representation of visual input in artificial vision systems.

From the past few years, the size of the data grows exponentially with respect to volume, velocity, and dimensionality due to wide spread use of embedded and distributed surveillance cameras for security reasons. In this paper, we have proposed an integrated approach for biometric-based image retrieval and processing which addresses the two issues. The first issue is related to the poor visibility of the images produced by the embedded and distributed surveillance cameras, and the second issue is concerned with the effective image retrieval based on the user query. This paper addresses the first issue by proposing an integrated image enhancement approach based on contrast enhancement and colour balancing methods. The contrast enhancement method is used to improve the contrast, while the colour balancing method helps to achieve a balanced colour. Importantly, in the colour balancing method, a new process for colour cast adjustment is introduced which relies on statistical calculation. It adjusts the colour cast and maintains the luminance of the image. The integrated image enhancement approach is applied to the enhancement of low quality images produced by surveillance cameras. The paper addresses the second issue relating to image retrieval by proposing a content-based image retrieval approach. The approach is based on the three features extraction methods namely colour, texture and shape. Colour histogram is used to extract the colour features of an image. Gabor filter is used to extract the texture features and the moment invariant is used to extract the shape features of an image. The use of these three algorithms ensures that the proposed image retrieval approach produces results which are highly relevant to the content of an image query, by taking into account the three distinct features of the image and the similarity metrics based on Euclidean measure. In order to retrieve the most relevant images, the proposed approach also employs a set of fuzzy heuristics to improve the quality of the results further. The results show the proposed approaches perform better than the well-known existing approaches.

Increasing deployment of intermittent power generation from renewable resources in the smart grid, such as photovoltaic (PV) or wind farm, will cause large system frequency fluctuation when the load-frequency control (LFC) capacity is not enough to compensate the unbalance of generation and load demand. Even worse, the system inertia will decrease when the smart grid is in island operating mode, which would degrade system damping and cause system instability. Meanwhile, electric vehicles (EVs) will be widely used by customers in the near future, where the EV station could be treated as dispersed battery energy storage. Therefore, the vehicle-to-grid (V2G) technology can be employed to compensate for inadequate LFC capacity, thus improving the island smart grid frequency stability. In this paper, an on-line reinforcement learning (RL) based method, called goal representation adaptive dynamic programming (GrADP), is employed to adaptive control of units in an island smart grid. In the controller design, adaptive supplementary control signals are provided to proportional-integral-derivative (PID) controller by GrADP in a real-time manner. Comparative simulation studies on a benchmark smart grid with micro-turbine (MT), EVs, PV array and wind power are carried out among the GrADP controller, the original PID controller and the particle swarm optimization (PSO) based fuzzy logic controller. Simulation results demonstrate competitive performance and satisfied learning ability of the GrADP based coordinate controller. Moreover, the impact of signal transmission delay on the control performance is also considered, and suggestions to address this issue are given in the paper.

International audience; In this paper we study the problem of the assignment of road paths to vehicles. Due to the assumption that a low percentage of vehicles follow the routes proposed by route guidance systems (RGS) and the increase of the use of the same, the conventional RGS might shortly result obsolete. Assuming a complete road network information at the disposal of RGSs, their proposed paths are related with user optimization which in general can be arbitrarily more costly than the system optimum. However, the user optimum is fair for the drivers of the same Origin-Destination (O-D) pair but it doesn't guarantee fairness for different O-D pairs. Contrary, the system optimum can produce unfair assignments both for the vehicles of the same as of different O-D pairs. This is the reason why, in this paper, we propose an optimization model which bridges this gap between the user and system optimum , and propose a new mathematical programming formulation based on Nash Welfare optimization which results in a good egalitarian and utilitarian welfare for all O-D pairs. To avoid the issues with the lack of robustness related with the centralized implementation, the proposed model is highly distributed. We test the solution approach through simulation and compare it with the conventional user-and system-optimization.; Dans cet article, nous étudions le problème de l’attribution de chemins à des véhicules. En supposant qu'un faible pourcentage de véhicules empruntent les itinéraires proposés par les systèmes de guidage d'itinéraire (RGS) et leur utilisation accrue, le système RGS conventionnel pourrait bientôt devenir obsolète. En supposant que les RGS disposent d'informations complètes sur le réseau routier, leurs chemins proposés sont liés à l'optimisation de l'utilisateur, qui peut en général être arbitrairement plus coûteuse que l'optimum du système. Toutefois, l'optimum d'utilisation est équitable pour les conducteurs de la même paire origine-destination (O-D), mais il ne garantit pas l'équité pour différentes paires O-D. Au contraire, l’optimum du système peut produire des affectations déloyales, que ce soit pour les véhicules de même ou de différentes paires O-D. C'est la raison pour laquelle, dans cet article, nous proposons un modèle d'optimisation qui comble ce fossé entre l'utilisateur et l'optimum du système, ainsi qu'une nouvelle formulation de programmation mathématique basée sur l'optimisation du bien-être de Nash, qui donne un bon bien-être égalitaire et utilitaire pour tous. paires. Pour éviter les problèmes de manque de robustesse liés à la mise en œuvre centralisée, le modèle proposé est très distribué. Nous testons l'approche de la solution par la simulation et la comparons à l'optimisation conventionnelle de l'utilisateur et du système.

The emergence of ubiquitous sources of streaming data has given rise to the popularity of algorithms for online machine learning. In that context, Hoeffding trees represent the state-of-the-art algorithms for online classification. Their popularity stems in large part from their ability to process large quantities of data with a speed that goes beyond the processing power of any other streaming or batch learning algorithm. As a consequence, Hoeffding trees have often been used as base models of many ensemble learning algorithms for online classification. However, despite the existence of many algorithms for online classification, ensemble learning algorithms for online regression do not exist. In particular, the field of online any-time regression analysis seems to have experienced a serious lack of attention. In this paper, we address this issue through a study and an empirical evaluation of a set of online algorithms for regression, which includes the baseline Hoeffding-based regression trees, online option trees, and an online least mean squares filter. We also design, implement and evaluate two novel ensemble learning methods for online regression: online bagging with Hoeffding-based model trees, and an online RandomForest method in which we have used a randomized version of the online model tree learning algorithm as a basic building block. Within the study presented in this paper, we evaluate the proposed algorithms along several dimensions: predictive accuracy and quality of models, time and memory requirements, bias–variance and bias–variance–covariance decomposition of the error, and responsiveness to concept drift.

Probabilistic K-nearest neighbour (PKNN) classification has been introduced to improve the performance of the original K-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, K. The contribution of this paper is to incorporate the uncertainty in K into the decision making, and consequently to provide improved classification with Bayesian model averaging. Indeed the problem of assessing the uncertainty in K can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, we develop a new functional approximation algorithm to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, the algorithms avoid cross validation by adopting Bayesian framework. The performance of the proposed approaches is evaluated on several real experimental datasets. HighlightsIt finds the optimal number of neighbours K for probabilistic KNN (PKNN).It reconstructs the density of K for the PKNN.It incorporates the uncertainty in K into the decision making.It provides improved PKNN classification using Bayesian model averaging.

Slack variables are utilized in optimisation problems in order to build soft margin classifiers that allow for more flexibility during training. A robust binary classification algorithm that is based on the minimisation of the energy of slack variables, called the Mean Squared Slack (MSS), is proposed in this paper. Initially, the algorithm is analysed for the linear case, where the minimum mean squared slack is attained as a separating vector. Next, the kernel trick is exploited to facilitate computation of non-linear separating hyperplanes. For this paper, two kernels are tested, namely the radial basis function (RBF) and the polynomial kernel. In order to ensure a time and memory efficient system that converges in a few iterations four strategies are applied so as to withhold just a subset of feature vectors that are misclassified during training. Aiming to the automatic optimisation of the kernel parameters a modern combination of particle swarm optimisation (PSO) with artificial immune system (AIS) is tested. The aforementioned evolutionary methods are combined in a parallel architecture. Four datasets of diverse nature are exploited for performance evaluation, namely the iris, the SPECTheart, the vertebral column, and the wine quality datasets. Simulation experiments demonstrate high classification accuracy in a number of benchmark datasets.

This paper describes the algorithms used to estimate the state of consumption of a pump based on dynamic pressure or vibrations. To create algorithms, the author used computational intelligence methods in the form of neural networks. In order to perform the analysis, data analysis systems were designed based on three neural networks: multilayer perceptron neural network (MLP), generalized regression neural network (GRNN) and probabilistic neural network (PNN). Processing of the input signal in the final result of the analysis consisted of several steps. First, the measurement data were preprocessed (delete constant component, normalization, standardization, reduction, fast Fourier transform (FFT), etc.), and training and test sets were prepared using the matrices with the expected system answers. The last step was the analysis, consisting of design data analysis systems based on artificial neural networks and their learning and testing. On the basis of the obtained results the effectiveness of neural networks and the methods of pre-processing of the signals applied to approximate the state of consumption of the displacement pump were evaluated. Design systems were evaluated based on accuracy (generated error) and complexity (number of parameters and training time) criteria. The main contribution of the paper is to design and compare methods for pre-processing the signal, and to design and compare the effectiveness of the three neural networks in the diagnosis consumption of a positive displacement pump.

In this paper, an intelligent swarm based-wavelet neural network for affective mobile designed is presented. The contribution on this paper is to develop a new intelligent particle swarm optimization (iPSO), where a fuzzy logic system developed based on human knowledge is proposed to determine the inertia weight for the swarm movement of the PSO and the control parameter of a newly introduced cross-mutated operation. The proposed iPSO is used to optimize the parameters of wavelet neural network. An affective design of mobile phones is used to evaluate the effectiveness of the proposed iPSO. It has been found that significantly better results in a statistical sense can be obtained by the iPSO comparing with the existing hybrid PSO methods.