Search

This paper proposes a method to detect the high-frequency oscillations (HFOs) in epileptic seizure onset zones (SOZs) localization using semi-supervised k-means and mean shift algorithm. Wavelet entropy (WE) and teager energy operator (TEO) are adopted to distinguish HFOs from normal electroencephalogram (EEG). Labeled data are used to initialize the clustering center of semi-supervised k-means algorithm, and unlabeled data are employed to obtain physiological and suspected pathological HFOs. For the suspected pathological HFOs, the mean shift algorithm is used for clustering, and the results are analyzed by the spectral center algorithm to locate SOZs. By comparing the EEG data of five patients with the results of the other three methods, it can be seen that the method proposed in this paper has good sensitivity and specificity, which is helpful for accurate localization before clinical epilepsy surgery.

This paper presents a method of water quality monitoring using a Feedback self correcting system combined with a Densely Connected Convolution Network. We find an effective method to correct the model output and innovate the method of biological water quality monitoring. Fish movement trajectory is a comprehensive expression of various water quality classification characteristics used in all the literature, and it is an important basis for classification of biological water quality. In this paper, we use the image segmentation method of Mask-RCNN to obtain the centroid coordinates of the fish and draw the trajectory image of the fish in a certain period of time. The trajectory image data sets are divided into normal and abnormal water quality. Densely connected convolution network(DenseNet) is used to classify the quality of water. The experiment is based on normal and abnormal water quality image data, and the model correction system with deviation feedback can be designed by the output of softmax. The learning ability of the classification model in practical application is greatly improved and enhance the stability of the detection system. The experimental results show that the water quality identification rate of the model reaches 99.38%, which is far higher than that of all previous water quality classification models.

This paper addresses the issue of Alzheimer’s disease (AD) detection from Magnetic Resonance Images (MRIs). Existing AD detection methods rely on global feature learning from the whole brain scans, while depending on the tissue types, AD related features in different tissue regions, e.g. grey matter (GM), white matter (WM), and cerebrospinal fluid (CSF), show different characteristics. In this paper, we propose a deep learning method for multi-scale feature learning based on segmented tissue areas. A novel deep 3D multi-scale convolutional network scheme is proposed to generate multi-resolution features for AD detection. The proposed scheme employs several parallel 3D multi-scale convolutional networks, each applying to individual tissue regions (GM, WM and CSF) followed by feature fusions. The proposed fusion is applied in two separate levels: the first level fusion is applied on different scales within the same tissue region, and the second level is on different tissue regions. To further reduce the dimensions of features and mitigate overfitting, a feature boosting and dimension reduction method, XGBoost, is utilized before the classification. The proposed deep learning scheme has been tested on a moderate open dataset of ADNI (1198 scans from 337 subjects), with excellent test performance on randomly partitioned datasets (best 99.67%, average 98.29%), and good test performance on subject-separated partitioned datasets (best 94.74%, average 89.51%). Comparisons with state-of-the-art methods are also included.

This paper investigates the consensus asymptotic convergence problem for a class of nth-order strict-feedback multiagent systems, which include the input quantization, actuator faults, unknown nonlinear functions and directed communication topology. Because the upper bounds of the time-varying stuck faults and external disturbance are commonly difficult to accurately determine, it is also assumed that these upper bounds are unknown in this paper. First, a group of first-order filters are designed to estimate the bounds of the reference signal for each agent. Second, smooth functions are introduced to compensate the effect of quantization and bounded stuck faults. Meanwhile, a new back-stepping method is used to propose an intermediate control law and an adaptive design procedure, and the final distributed control protocols are established. All closed-loop signals are uniformly bounded, and the tracking errors asymptotically converge to zero. Finally, a practical example simulation is provided to demonstrate the effectiveness of the proposed scheme.

Virtualization in the data center network is used to overcome the resistance of Internet ossification. Node mapping in virtual network embedding (VNE) process is more important than link mapping because each node mapping has a lot of choices with high probability. Historical data always have valuable information which displays how to be mapped virtual networks with special features. Previous works use physical network historical information, and they do not pay attention to the characteristics of virtual networks during the embedding process. To address this problem, this paper proposes SoGVNE (Self-organized Green Virtual Network Embedding) which uses previous successful virtual node mapping information to train a self-organizing map (SOM) neural network. Then, it will respond to future requests quickly and with fewer resources. SoGVNE uses virtual networks historical data and is able to embed more than one virtual network concurrently. Using supervised SOM makes the SoGVNE aware of resources with renewable energy and uses them to manage energy consumption. This paper evaluates SoGVNE with reinforcement-learning, Monte-Carlo and Presto methods. The results indicate the high efficiency of the proposed method in learning the node mapping process and achieving high profit and acceptance ratio with low cost.

In this paper, the robust H∞ state estimator is designed for the two-dimensional (2-D) systems in Fornasini-Machesini second model under logarithmic quantization and Round-Robin protocol. The norm-bounded uncertainties are also involved in the considered system, which are governed by two stochastic variables satisfying the Bernoulli distribution. As a kind of static scheduling, the Round-Robin protocol assigns the communication access rights to the sensors in the chronological order. The aim of this paper is to design a state estimator so that the addressed estimation error system satisfies a predefined H∞ performance index. Sufficient conditions are given to ensure the 2-D augmented error system to be robustly stable via intensive stochastic analysis, then the results are further extended to achieve the robust H∞ stability for the augmented error system. Explicit expressions of the estimator gains are also given by solving certain matrix inequalities. Finally, a numerical example is presented to demonstrate the effectiveness of the proposed estimation scheme.

This paper addresses two issues toward practical use of wearable electroencephalogram (EEG) measurement devices. Ocular (eye movement and blink) artifacts often contaminate EEGs and deteriorate the performance of EEG-based brain–computer interfaces (BCIs). Although wearable consumer-grade EEG devices with single electrode allow users to operate BCIs conveniently in daily lives, it remains a challenging issue to attenuate ocular artifacts from single-channel measurements without spatial information. Existing ocular artifact reduction methods are, however, not simple enough for single-channel EEG data in the sense that they require an additional reference channel and/or pre-processing for artifact segment detection. Another issue is how to assess the performance of artifact reduction; the existing studies have used their own datasets that are not accessible from other researchers. Then, this paper makes two major contributions. (1) This paper proposes a novel ocular artifact reduction method, multi-scale dictionary learning (MSDL), which operates under single-channel measurements and without artifact segment detection. (2) We also develop a semi-simulation setting for quantitative evaluation with a publicly available EEG dataset. In particular, we employed BCI Competition IV Dataset 2a, on which the proposed method was compared with state-of-art methods. The proposed technique showed the best performance for recovering artifact-reduced waveforms from single-channel data compared to the other artifact reduction methods. The Matlab scripts for semi-simulation data generation and single-channel artifact reduction are available on GitHub .

Probabilistic models have proven their strength to model many natural phenomena as close as possible to reality, in particular, the probabilistic self organizing map (PRSOM) that belongs to the unsupervised learning models. It allows to provide an estimation of the probability density function through the likelihood maximization. This latter function depends on several parameters given by the model architecture. In this context, the aim of this paper is to deal with the architecture choice problem that consists in determining the optimal number of components needed for a better performance. In this paper, we propose a new optimization model that describes the problem above. We present the architecture of PRSOM in a mathematical system of a non linear objective function with mixed variables under linear and quadratic constraints. Due to the complexity of the resolution, we suggest the continuous Hopfield neural network (CHN) that we support by a deep stability analysis. Performance of the proposed model is demonstrated through the dataset clustering.

In this paper, the resilient state estimation problem is discussed for a class of discrete nonlinear complex networks with time-delay subject to the stochastic communication protocol (SCP). The SCP is employed to determine which node has the priority of accessing to the communication networks at each transmission instant. A switch-based approach is utilized to establish the updating rule of the measurement output under the SCP. The aim of the paper is to design a resilient state estimator for the addressed complex networks with time-delay and SCP, where sufficient conditions are given to ensure that the estimation error is exponentially ultimate bounded in the mean-square sense and the asymptotic upper bound of the estimation error is given. Moreover, the expression forms of the desired estimator gains are obtained via the solutions to a set of linear matrix inequalities. Finally, a numerical example is provided to illustrate the validity of the proposed resilient state estimation method.

This paper focuses on stability and synchronization of fractional-order Hopfield neural networks. By taking information on activation functions into account, two novel convex Lyapunov functions are constructed: one is a fractional-order-dependent Lyapunov function, and the other is a new quadratic Lyapunov function. Based on these two Lyapunov functions, together with a fractional-order differential inequality, a fractional-order-dependent Mittag–Leffler stability criterion is derived for fractional-order Hopfield neural networks, which is in the form of linear matrix inequalities (LMIs). Moreover, a Mittag–Leffler synchronization criterion in terms of LMIs is presented for drive-response fractional-order Hopfield neural networks under linear control. Finally, three numerical examples are provided to indicate the benefits and less conservatism of the obtained criteria in this paper.

This paper presents a comprehensive review of the significant studies exploited Artificial Neural Networks (ANNs) in BEA (Building Energy Analysis). To achieve a full coverage of the relevant studies to the scope of the research, a three-decade time span of the publishing date of the existing studies was taken into account. The review focuses on the studies utilized ANN to analyze the energy-related issues associated with buildings in major areas, including modeling of water heating and cooling systems, heating and cooling loads prediction, modeling heating ventilation air conditioning systems, indoor air temperature prediction, and building energy consumption prediction. Moreover, the findings of the abundant reviewed studies along with the potential future research to be carried out are discussed elaborately. Regarding the comprehensive review conducted, it is found out that the majority of studies focused on building energy consumption and indoor air temperature prediction. Additionally, it is observed that there has been a growing interest in the application of newly-developed ANNs to BEA areas, such as general regression neural network and recurrent neural network, due to their abilities in improving the modeling and prediction of buildings energy analysis. It is believed that this thorough review paper is useful for the researchers and scientific engineers working on the application of AI-based techniques to the building-energy-related areas to find out the relevant references and current state of the field.

In online-learning, the data is incrementally received and the distributions from which it is drawn may keep changing over time. This phenomenon is widely known as concept drift. Such changes may affect the generalization of a learned model to future data. This problem may be exacerbated by the form of the drift itself changing over time. Quantitative measures to describe and analyze the concept drift have been proposed in previous work. A description composed from these measures is called a concept drift map. We believe that these maps could be useful for guiding how much knowledge in the old model should be forgotten. Therefore, this paper presents an adaptive online learning model that uses a concept drift map to regulate the forgetting factor of an extreme learning machine. Specifically, when a batch of new instances are labeled, the distribution of each class on each attribute is firstly estimated, and then it is compared with the distribution estimated in the previous batch to calculate the magnitude of concept drift, which is further used to regulate the forgetting factor and to update the learning model. Therefore, the novelty of this paper lies in that a quantitative distance metric between two distributions constructed on continuous attribute space is presented to construct concept drift map which can be further associated with the forgetting factor to make the learning model adapt the concept drift. Experimental results on several benchmark stream data sets show the proposed model is generally superior to several previous algorithms when classifying a variety of data streams subject to drift, indicating its effectiveness and feasibility.

This paper is concerned with the issue of the finite-time adaptive synchronization and finite-time synchronization of memristive neural networks with discontinuous activation functions and mixed time-varying delays. For synchronizing the drive-response memristive neural networks in finite time, an adaptive state-feedback controller and a state-feedback controller are proposed, respectively. Then by using the theories of differential inclusions and set-valued map, the synchronization issue of drive-response memristive neural networks with discontinuous activation functions and mixed time-varying delays is transformed into the stabilization issue of the error system. Moreover, based on the stability theory, Forti Lemma and Hardy inequality, some novel algebraic synchronization criteria are deduced to ensure the finite-time adaptive synchronization and finite-time synchronization of memristive neural networks with discontinuous activation functions and mixed time-varying delays under the adaptive state-feedback controller and the state-feedback controller. And the settling times for finite-time adaptive synchronization and finite-time synchronization are given. Furthermore, it is hard to estimate the initial conditions for a large system, so the settling times in this paper are not dependent on initial conditions of system. Finally, an example is provided to demonstrate the effectiveness of the obtained results.

The Multi-Layered Echo-State Network (ML-ESN) is a recently developed, highly powerful type of recurrent neural network. It has succeeded in dealing with several non-linear benchmark problems. On account of its rich dynamics, ML-ESN is exploited in this paper, for the first time, as a recurrent Autoencoder (ML-ESNAE) to extract new features from original data representations. Further, the challenging and crucial task of optimally determining the ML-ESNAE architecture and training parameters is addressed, in order to extract more efficient features from the data. Traditionally, in a ML-ESN, the number of parameters (hidden neurons, sparsity rates, weights) are randomly chosen and manually altered to achieve a minimum learning error. On one hand, this random setting may not guarantee best generalization results. On the other, it can increase the network’s complexity. In this paper, a novel bi-level evolutionary optimization approach is thus proposed for the ML-ESNAE, to deal with these challenges. The first level offers Pareto multi-objective architecture optimization, providing maximum learning accuracy while maintaining a reduced complexity target. Next, every Pareto optimal solution obtained from the first level undergoes a mono-objective weights optimization at the second level. Particle Swarm Optimization (PSO) is used as an evolutionary tool for both levels 1 and 2. An empirical study shows that the evolved ML-ESNAE produces a noticeable improvement in extracting new, more expressive data features from original ones. A number of application case studies, using a range of benchmark datasets, show that the extracted features produce excellent results in terms of classification accuracy. The effectiveness of the evolved ML-ESNAE is demonstrated for both noisy and noise-free data. In conclusion, the evolutionary ML-ESNAE is proposed as a new benchmark for the evolutionary AI and machine learning research community.

Currently, knowledge discovery in databases is an essential first step when identifying valid, novel and useful patterns for decision making. There are many real-world scenarios, such as bankruptcy prediction, option pricing or medical diagnosis, where the classification models to be learned need to fulfill restrictions of monotonicity (i.e. the target class label should not decrease when input attributes values increase). For instance, it is rational to assume that a higher debt ratio of a company should never result in a lower level of bankruptcy risk. Consequently, there is a growing interest from the data mining research community concerning monotonic predictive models. This paper aims to present an overview of the literature in the field, analyzing existing techniques and proposing a taxonomy of the algorithms based on the type of model generated. For each method, we review the quality metrics considered in the evaluation and the different data sets and monotonic problems used in the analysis. In this way, this paper serves as an overview of monotonic classification research in specialized literature and can be used as a functional guide for the field.

Twin Support Vector Machines (TWSVMs) have emerged as an efficient alternative to Support Vector Machines (SVM) for learning from imbalanced datasets. The TWSVM learns two non-parallel classifying hyperplanes by solving a couple of smaller sized problems. However, it is unsuitable for large datasets, as it involves matrix operations. In this paper, we discuss a Twin Neural Network (Twin NN) architecture for learning from large unbalanced datasets. The objective functions of the networks in the Twin NN are designed to realize the idea of the Twin SVM with non-parallel decision boudaries for the respective classes, while also being able to reduce model complexity. The Twin NN optimizes the feature map, allowing for better discrimination between classes. The paper also discusses an extension of the Twin NN for multiclass datasets. This architecture trains as many neural networks as the number of classes, and has the additional advantage that it does not have any hyper-parameter which requires tuning. Results presented in the paper demonstrate that the Twin NN generalizes well and scales well on large unbalanced datasets.

Feature selection is of great importance for two possible scenarios: (1) prediction, i.e., improving (or minimally degrading) the predictions of a target variable while discarding redundant or uninformative features and (2) discovery, i.e., identifying features that are truly dependent on the target and may be genuine causes to be determined in experimental verifications (for example for the task of drug target discovery in genomics). In both cases, if variables have a large number of missing values, imputing them may lead to false positives; features that are not associated with the target become dependent as a result of imputation. In the first scenario, this may not harm prediction, but in the second one, it will erroneously select irrelevant features. In this paper, we study the risk/benefit trade-off of missing value imputation in the context of feature selection, using causal graphs to characterize when structural bias arises. Our aim is also to investigate situations in which imputing missing values may be beneficial to reduce false negatives, a situation that might arise when there is a dependency between feature and target, but the dependency is below the significance level when only complete cases are considered. However, the benefits of reducing false negatives must be balanced against the increased number of false positives. In the case of binary target variable and continuous features, the t-test is often used for univariate feature selection. In this paper, we also introduce a de-biased version of the t-test allowing us to reap the benefits of imputation, while not incurring the penalty of increasing the number of false positives.

With reciprocal problem widely arising in various scientific computation and engineering fields in recent decades, it is always assumed that the computing process is free of measurement noises. However, time is precious for time-varying reciprocal problem, which is considered to be an important operation in a floating-point divider in practice and optimization. Therefore, mathematical models with inherent noise tolerance are needed to compute reciprocal problem in real time. In this paper, different modified zeroing neural dynamics models are proposed, analyzed and investigated for the solution of time-varying reciprocal problem with inherent tolerance to noises. Moreover, theoretical analyses show that the modified zeroing neural dynamics globally/exponentially converge to the exact solution of the time-varying reciprocal problem with measurement noises. Furthermore, the zeroing neural dynamic models with different activation functions are employed for comparison with the modified zeroing neural dynamic model. Finally, some numerical simulations are reported and analyzed to substantiate the feasibility and superiority of the developed zeroing neural dynamic for time-varying reciprocal problem with inherent tolerance to noises. This paper develops a systematic approach on exploiting control techniques to design zeroing neural dynamic models for robustly and accurately solving time-varying reciprocal problems.

This paper focuses on the early detection problem of abrupt stall inception in axial compressors subject to non-uniform inflow. The considered stall inception is small and abrupt. Such a stall inception is not only easy to cause system instability, but also difficult to be rapidly detected. In this paper, a novel abrupt stall detection approach is proposed for axial compressors with non-uniform inflow by using the deterministic learning algorithm. Firstly, a high-order distortion dynamic mode is derived to generate the normal and stall inception situations. Subsequently, dynamic features of these different situations are extracted and stored in constant radial basis function networks, which are regarded as the training patterns. Based on dynamic pattern recognition, the average L1 norms of the state errors are used as the measure of the dynamical differences between the test pattern and the training patterns, and then a stall detection method is developed to rapidly decide whether the test patterns are the abrupt stall inception. Simulation studies are given to show that the proposed stall detection approach is sensitive to small dynamic changes of axial compressors, thereby rapidly detecting the abrupt stall inception.

In recent years, person re-identification (ReID) has received much attention since it is a fundamental task in intelligent surveillance systems and has widespread application prospects in numerous fields. Given an image of a pedestrian captured from one camera, the task is to identify this pedestrian from the gallery set captured by other multiple cameras. It is a challenging issue since the appearance of a pedestrian may suffer great changes across different cameras. The task has been greatly boosted by deep learning technology. There are mainly six types of deep learning-based methods designed for this issue, i.e. identification deep model, verification deep model, distance metric-based deep model, part-based deep model, video-based deep model and data augmentation-based deep model. In this paper, we first give a comprehensive review of current six types of deep learning methods. Second, we present the detailed descriptions of existing person ReID datasets. Then, some state-of-the-art performances of methods over recent years on several representative ReID datasets are summarized. Finally, we conclude this paper and discuss the future directions of the person ReID.

In this paper, a containment control issue for discrete-time second-order multi-agents (MASs) with asynchronous intermittent communication is analyzed, where each agent communicates with some of its neighbors at certain discrete time according to its own clock rather than the complete discrete process and all agents’ clocks are independent of each other. By constructing the spanning subgraph of communication topology at each discrete time, the stability analysis of containment control under the asynchronous intermittent communication is equivalently transformed into the products convergence of nonnegative matrices. Hybrid tools including matrix theory and the composite binary relation are utilized to analyze this convergence issue. Unlike the existing results on the containment control with fixed topology, in which the followers will gradually enter into the convex hull constructed by the leaders and their final positions are fixed, it is theoretically proved in this paper that although the followers will enter into the convex hull of the leaders under the asynchronous intermittent communication, their positions after entering the convex hull may always be time-varying. Finally, the theoretical finding is verified by a simulation example.

This paper tackles the challenging problem of multi-shot person re-identification with Convolutional Neural Network (CNN). As no prior information about how importance each instance plays, it is non-trivial to exploit the interaction information shared by the multi-shot images to help identification. Traditional CNN is in single-shot architecture, then how to utilize the interaction information provided by multi-shot images becomes an important problem to solve. Furthermore, as data augmentation methods are not strictly label-preserving, it increases the difficulty to select discriminative instance for CNN training. In this paper, we propose a weakly supervised CNN framework named Multi-Instance Convolutional Neural Network (MICNN) to solve the aforementioned problem. We develop two paradigms, i.e., Embedding-Space paradigm and Instance-Space paradigm, which re-formulate the person re-identification problem as a multi-instance verification problem with part-based features extracted by neural network. We respectively devise a specific bag-level loss function which incorporates the characteristics of the multi-instance problem for each paradigm. Experiments show that the proposed IS method outperforms many related state-of-the-art techniques on four benchmark datasets: CUHK03, SYSUm, RAiD and Market-1501.

This paper investigates the leakage delay-dependent global asymptotic stability problem for a class of complex-valued neural networks (CVNNs) with discrete and distributed time-varying delays. In order to handle this issue easily, an appropriate Lyapunov–Krasovskii functional (LKF) is constructed with some augmented delay-dependent terms. By employing integral inequalities, several delay-dependent sufficient conditions are derived that ensure the global asymptotic stability of the considered system model. Moreover, the results obtained in this paper have expressed in terms of complex-valued linear matrix inequalities (LMIs), whose feasible solutions can be easily verified by effective YALMIP control toolbox in MATLAB LMI. Finally, two benchmark illustrative examples are given to show the effectiveness and advantages of the proposed results.

Feature selection is an important step in building classifiers for high-dimensional data problems, such as EEG classification for BCI applications. This paper proposes a new wrapper method for feature selection, based on a multi-objective evolutionary algorithm, where the representation of the individuals or potential solutions, along with the breeding operators and objective functions, have been carefully designed to select a small subset of features that has good generalization capability, trying to avoid the over-fitting problems that wrapper methods usually suffer. A novel feature ranking procedure is also proposed in order to analyze the stability of the proposed wrapper method. Four different classification schemes have been applied within the proposed wrapper method in order to evaluate its accuracy and stability for feature selection on a real motor imagery dataset. Experimental results show that the wrapper method presented in this paper is able to obtain very small subsets of features, which are quite stable and also achieve high classification accuracy, regardless of the classifiers used., Project TIN2015-67020-P (Spanish “Ministerio de Economía y Competitividad”), European Regional Development Funds (ERDF)

Aiming at process signal classification combined with fuzzy decision rules, a fuzzy process neural network (FPNN) is proposed in this paper. The FPNN is structured with a process signal input layer, a fuzzy process neuron (FPN) hidden layer, a signal pattern layer, and a fuzzy decision output layer. The spatio-temporal aggregation in the FPN is taken as a generalized inner product operation, which can measure the fuzzy similarity of the distributed features among process signals and FPN uses an exponential fuzzy membership function as activation function. The FPNN selectively sums the output of the FPN hidden layer to the pattern layer according to the category of input signal. By linking a Takagi-Sugeno fuzzy classifier behind the pattern layer, direct classification of the process signals is achieved. The FPNN can improve the deficiencies of existing time-varying signal classification methods, such as the complete training data sets are required, and the information processing processes and algorithms are complicated. In this paper, the theoretical properties of the FPNN are analyzed, and the comprehensive learning algorithm for FPNNs is given. The discrimination of reservoir water flooding condition based on multi-channel well logging process signals was used as an example for experimental analysis; the results verify the validity of the model and algorithm.

Hand pose estimation plays an important role in human–computer interaction. The traditional way is to deal with a video stream frame by frame. However, since the gesture in the video is changing continuously, the adjacent frames must be highly related to each other. Therefore, the input of the neural network in this paper was set to be a series of contiguous video frames in order to make use of the relevance that the adjacent frames have. In this paper, a convolutional recurrent neural network (CRNN) module is proposed, which combines the characteristics of Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), and can significantly improve the accuracy of the network.The impact of the location of the CRNN/RNN module in the network is also discussed in this paper. Finally, we demonstrated that our approach significantly outperforms the current state-of-the-art techniques in the NYU Hand dataset.

In this paper, we present a multi-view transfer learning model named Multi-view Transfer Discriminative Model (MTDM) for both image and text classification tasks. Transfer learning, which aims to learn a robust classifier for the target domain using data from a different distribution, has been proved to be effective in many real-world applications. However, most of the existing transfer learning methods map across domain data into a high-dimension space which the distance between domains is closed. This strategy always fails in the multi-view scenario. On the contrary, the multi-view learning methods are also difficult to extend in the transfer learning settings. One of our goals in this paper is to develop a model which can perform better in both multi-view and transfer learning settings. On the one hand, the problem of multi-view is implemented by the paradigm of learning using privileged information (LUPI), which could guarantee the principle of complementary and consensus. On the other hand, the model adequately utilizes the source domain data to build a robust classifier for the target domain. We evaluate our model on both image and text classification tasks and show the effectiveness compared with other baseline approaches.

This paper is concerned with the finite-frequency bounded realness analysis and finite-frequency H∞ distributed filter design problem for the discrete-time networked system which consists of heterogeneous subsystems interconnected over an arbitrary network. The H∞ performance analysis of networked systems is generalized from the entire-frequency region to some finite-frequency regions. A necessary and sufficient condition in the centralized manner is established to check the finite-frequency bounded realness property of networked systems. Moreover, in order to design a distributed filter and reduce the computational effort, a sufficient condition in terms of individual subsystems is developed. Based on the new condition derived, a sufficient condition is given for the existence of the distributed filter and an LMI method is then presented for designing the distributed filter. A practical example is included to illustrate the proposed approach in this paper and it is shown that the developed finite-frequency bounded realness condition is less conservative than the existing entire-frequency one.

This paper is concerned with the H∞ fuzzy proportional-integral-derivative (PID) control problem for delayed Takagi–Sugeno (T-S) fuzzy systems in the discrete-time setting. Based on the current and historical measurement data, a novel T-S fuzzy PID controller is developed with which the integral-loop is generated based on a fixed number of past measurements for the purpose of reducing the computational burden, where a special augmentation scheme is adopted to simplify the closed-loop system. The aim of this paper is to design the PID controller parameters such that the closed-loop T-S fuzzy system is exponentially stable and the prescribed H∞ disturbance attenuation performance is achieved. By adopting the Lyapunov stability theory and the linear matrix inequality technology, sufficient conditions are obtained for the existence of the desired fuzzy PID controllers. In addition, an iterative optimization procedure is proposed to design the controller parameters according to the cone complementarity linearization algorithm. Finally, a numerical simulation example is exploited to demonstrate the usefulness and effectiveness of our proposed design scheme.

In this paper, neural networks (NNs)-based adaptive dynamic surface control is investigated for a class of pure-feedback nonlinear systems with unknown time delay and input hysteresis nonlinearities. The design difficulty appeared in this paper due to time delay is handled by combining neural networks with finite covering lemma instead of using Krasovskii functionals, and the assumptions on the time-delay functions are removed based the finite covering lemma and NNs. Meanwhile, completely unknown backlash-like hysteresis control input that frequently exists in practice is also considered. By combining the adaptive backstepping recursive design technique with the universal approximation ability of neural networks, an adaptive neural control algorithm is systemically designed for the system under consideration, and the explosion of complexity exists in traditional backstepping design is avoided by using dynamic surface technique. It is shown that the designed adaptive controller can guarantee that all the signals are ultimately uniformly bounded and the desired signal can be tracked with a small domain of the origin. A numerical example and an example of a real plant for adjustable metal cutting system are provided to show the feasibility of the proposed method.

Traditional recommendation methods based on matrix factorization techniques have yielded immense success because of their good scalability. However, they still face the problem of data sparsity, which may lead to a reduction in recommendation performance. As it is hard to learn good latent features in the sparse user-item rating matrix. In recent years, deep learning is very appealing in learning effective representations. Its non-linear characteristics just remedy the shortcomings of matrix factorization. In this paper, a novel method deep variational matrix factorization recommendation (DVMF) is proposed for large scale sparse dataset. DVMF is based on latent factors to predict the ratings. The latent features of the users and items are respectively obtained through a deep nonlinear structure. Based on the latent factors and combined with matrix factorization method, the paper presents algorithm optimization method of DVMF. The experiments on three real-world datasets from different domains show that DVMF is able to provide higher accuracy than recommendation algorithms based on matrix factorization or deep learning individually on large scale sparse dataset.

In this paper, we study the problem of matrix recovery, which aims to restore a target matrix of authentic samples from grossly corrupted observations. Most of the existing methods, such as the well-known Robust Principal Component Analysis (RPCA), assume that the target matrix we wish to recover is low-rank. However, the underlying data structure is often non-linear in practice, therefore the low-rankness assumption could be violated. To tackle this issue, we propose a novel method for matrix recovery in this paper, which could well handle the case where the target matrix is low-rank in an implicit feature space but high-rank or even full-rank in its original form. Namely, our method pursues the low-rank structure of the target matrix in an implicit feature space. By making use of the specifics of an accelerated proximal gradient based optimization algorithm, the proposed method could recover the target matrix with non-linear structures from its corrupted version. Comprehensive experiments on both synthetic and real datasets demonstrate the superiority of our method.

It is a big challenge of computer vision to make machine automatically describe the content of an image with a natural language sentence. Previous works have made great progress on this task, but they only use the global or local image feature, which may lose some important subtle or global information of an image. In this paper, we propose a model with 3-gated model which fuses the global and local image features together for the task of image caption generation. The model mainly has three gated structures. 1) Gate for the global image feature, which can adaptively decide when and how much the global image feature should be imported into the sentence generator. 2) The gated recurrent neural network (RNN) is used as the sentence generator. 3) The gated feedback method for stacking RNN is employed to increase the capability of nonlinearity fitting. More specially, the global and local image features are combined together in this paper, which makes full use of the image information. The global image feature is controlled by the first gate and the local image feature is selected by the attention mechanism. With the latter two gates, the relationship between image and text can be well explored, which improves the performance of the language part as well as the multi-modal embedding part. Experimental results show that our proposed method outperforms the state-of-the-art for image caption generation., Comment: 35 pages, 7 figures, magazine

In the era of big data, all types of data with increasing samples and high-dimensional attributes are demonstrating their important roles in various fields, such as data mining, pattern recognition and machine learning, etc. Meanwhile, machine learning algorithms are being effectively applied in large-scale data processing. This paper mainly reviews the classical dimensionality reduction and sample selection methods based on machine learning algorithms for large-scale data processing. Firstly, the paper provides a brief overview to the classical sample selection and dimensionality reduction methods. Then, it pays attention to the applications of those methods and their combinations with the classical machine learning methods, such as clustering, random forest, fuzzy set, and heuristic algorithms, particularly deep leaning methods. Furthermore, the paper primarily introduces the application frameworks that combine sample selection and dimensionality reduction in the context of two aspects: sequential and simultaneous, which almost all get the ideal results in the processing of the large-scale training data contrasting to the original models. Lastly, we further conclude that sample selection and dimensionality reduction methods are essential and effective for the modern large-scale data processing. In the future work, the machine learning algorithms, especially the deep learning methods, will play a more important role in the processing of large-scale data.

This paper investigates the consensus problem of the second-order nonlinear multi-agent systems (MAS) with unknown periodic time-varying parameters (UPTVP) and imprecise communication topology structure (ICTS). T–S fuzzy models is used to express the ICTS, and the concept of fuzzy union connected (FUC) is defined. Under the condition that the ICTS is FUC, an adaptive repetitive learning control protocol is presented such that all the followers can track the leader asymptotically. With the information of leader agent is known to a small portion of following agents, an auxiliary control term was presented for each follower agent to handle leaders dynamics. The distributed MAS consensus is analyzed based on the Lyapunov stability theory. Furthermore, the proposed protocol is further promoted to solve the formation control problem. Finally, simulation results are shown to demonstrate the validity of the proposed design methods in this paper.

Redundant design can greatly improve the flexibility of robot manipulators, but may suffer from potential limitations such as system complicity, model uncertainties, physical limitations, which make it challenging to achieve accurate tracking. In this paper, we propose a novel kinematic controller based on a recurrent neural network(RNN) which is competent in model adaption. An identifier which is related to joint velocity and tracking error is designed to learn the kinematic parameters online. In the inner loop, the redundancy resolution is formulated as a quadratic optimization problem, and a RNN is built to obtain the optimal solution recurrently, and the minimum norm of joint velocity is derived as the secondary task. Theoretical analysis demonstrates the global convergence of tracking error. Compared with existing methods, uncertain kinematic model of the robot is allowed in this paper, and pseudo-inverse of Jacobian matrix is avoided, with the consideration of physical limitations in a joint framework. Numerical and actual experiments based on a serial robot Kinova JACO2 show the effectiveness of the proposed controller.

The action-dependent heuristic approximate dynamic (ADHDP) for nonlinear multi-input multi-output (MIMO) system needs different forms to adapt to variable practical objects. Due to some inappropriate network structure or training algorithm, unsuccessful designs or undesirable control effect is common in reality. Thus, at first, this paper addresses the chain rule problem of the compound derivative in training the nonlinear MIMO ADHDP. Then, this paper researches and proposes four actor-critic algorithms systematically according to four typical nonlinear systems. That is, the action-network extension, the sub-network, the cascaded action-network and the combined method. To illustrate the four methods, their detailed structures, derivation procedures and training algorithms are derived. The Lyapunov stability for the nonlinear MIMO ADHDP is proved as well. Through examples of an idling engine and aircraft controlling, the simulation results show the effectiveness of these methods. Besides, the property, advantages, disadvantages and the applicability of these methods are compared and highlighted. The four methods can be used to meet the design requirement of almost all the nonlinear MIMO ADHDP control systems. For incoming scholars in search of a nonlinear MIMO ADHDP to achieve the best control effect, the four actor-critic structures and algorithms can be a reference.

In this paper, a neurodynamic approach concerning complex Zhang neural networks (ZNNs) is presented to solve a complex-variable dynamic quadratic programming (QP). The proposed complex ZNNs are activated by various complex activation functions respectively, such as linear function, sign function and Li function. It is proved that ZNNs with different activation functions have different convergence rates involving super exponential convergency and finite time convergency. Furthermore, by introducing an integral term, a noise-tolerated ZNN is proposed to ensure the convergency under constant noise. Compared with existing works, the presented complex ZNNs in this paper have less amount of neurons and superior convergence rate. Some numerical examples are provided to illustrate the effectiveness and validity of the results. In addition, the complex ZNNs also apply well to LCMP beamforming problem.

In this paper, we focus on approaching fast and accurate target detection in high-resolution remote sensing images (RSIs). Recently, machine-learning based target detection systems have drawn increasing attention and some excellent target detection frameworks have been proposed for RSIs. However, huge storage and time consumption are still key flaws of those methods in practical applications. Especially, the employment of transfer learning would produce tremendous growth in parameters. A new target detection framework named target heat-map network (THNet) is proposed in this paper to address these problems. This framework consists of three parts: shallow features-extracting network, decoder network, and the location method. Firstly, we introduce the transfer-compression learning to train a shallow network under the supervision of a deep pre-trained network. Secondly, a decoder network is constructed to predict heat-map layers. Finally, a location method based on thresholding is proposed to identify positions of target instances. Compared with existing state-of-the-art methods including Faster-R-CNN, YOLOv2 and SSD, THNet with transfer learning has better performance, and THNet with transfer-compression learning also has superior performance in quantitative evaluation as well as significantly reducing time and saving storage, which makes it a great choice for applications with critical requirements for storage cost and running time.

This paper focuses on studying the distributed leader-following consensus problem of heterogeneous second-order time-varying nonlinear multi-agent systems (HSTN_MASs) under directed switching topology. Firstly, based on the relative position and relative velocity measurements among neighborhood agents, a class of distributed tracking protocols are proposed. Then the state transformations are provided to solve the time-varying nonlinearity and translate the leader-following problem into the stabilization control problem. By constructing the topology-dependent Lyapunov function and choosing appropriate time-varying regulation factor and coupling strength, it is shown that the exponential tracking of HSTN_MASs can be achieved. Moreover, this paper discusses the distributed leaderless consensus problem of HSTN_MASs. Finally, several simulation examples are given to illustrate the validity of theoretical results.

This paper focuses on the problem of finite-time consensus of linear multi-agent systems with input time-varying delay by employing a non-fragile control scheme, where the network topology of the communication process among agents is subject to a switching directed graph. The main intention of this paper is to propose a non-fragile controller design protocol that guarantees the finite-time stability of the resulting closed-loop system even in the presence of time-varying delay in the input channel. Sufficient conditions for the solvability of such a problem are obtained by using the Lyapunov–Krasovskii stability theory, the algebraic graph theory and some integral inequalities. Based on the obtained conditions, an explicit expression of the desired non-fragile control gain matrix is then presented. Finally, an illustrative example is given to exhibit the applicability and effectiveness of the proposed finite-time consensus control design method.

Unpredictable failures and unscheduled maintenance of physical systems increases production resources, produces more harmful waste for the environment, and increases system life cycle costs. Efficient remaining useful life (RUL) estimation can alleviate such an issue. The RUL is predicted by making use of the data collected from several types of sensors that continuously record different indicators about a working asset, such as vibration intensity or exerted pressure. This type of continuous monitoring data is sequential in time, as it is collected at a certain rate from the sensors during the asset's work. Long Short-Term Memory (LSTM) neural network models have been demonstrated to be efficient throughout the literature when dealing with sequential data because of their ability to retain a lot of information over time about previous states of the system. This paper proposes using a new LSTM architecture for predicting the RUL when given short sequences of monitored observations with random initial wear. By using LSTM, this paper proposes a new objective function that is suitable for the RUL estimation problem, as well as a new target generation approach for training LSTM networks, which requires making lesser assumptions about the actual degradation of the system.

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and weight sharing, can reduce the number of parameters and increase processing speed during training and inference. However, as the dimension of data becomes higher and the CNN architecture becomes more complicated, the end-to-end approach or the combined manner of CNN is computationally intensive, which becomes limitation to CNN’s further implementation. Therefore, it is necessary and urgent to implement CNN in a faster way. In this paper, we first summarize the acceleration methods that contribute to but not limited to CNN by reviewing a broad variety of research papers. We propose a taxonomy in terms of three levels, i.e. structure level, algorithm level, and implementation level, for acceleration methods. We also analyze the acceleration methods in terms of CNN architecture compression, algorithm optimization, and hardware-based improvement. At last, we give a discussion on different perspectives of these acceleration and optimization methods within each level. The discussion shows that the methods in each level still have large exploration space. By incorporating such a wide range of disciplines, we expect to provide a comprehensive reference for researchers who are interested in CNN acceleration.

Feature (gene) selection of microarray data is a very important and challenging task. This paper proposes a new feature selection method, partial maximum correlation information (PMCI), for microarray data classification. The proposed method extracts several orthogonal components from the feature space to evaluate the importance of each feature. In PMCI method, the component extraction is based on the correlation between feature space and class coding space. Meanwhile, this paper also provides a new generic class-encoding scheme for multi-class problems. This type of encoding scheme can ensure the smooth implementation for PMCI and has a clear geometric meaning. To validate the performance of the proposed method, this paper selects six state-of-the-art algorithms for comparison based on ten widely used microarray benchmark datasets for experiment. The experimental results show that PMCI is an effective feature selection method with good performance and lower time complexity.

This paper is concerned with the H ∞ state estimation problem for time-varying artificial neural networks with component-based distributed delays and stochastic protocol scheduling. A shared communication channel is adopted for data transmissions between the sensors and the estimator. For the purpose of avoiding data collisions, the stochastic protocol is used to schedule the transmission opportunities of sensors. A finite-horizon H ∞ index is introduced to reflect the performance specification of the estimation. The aim of this paper is to design a time-varying H ∞ estimator over a given finite-horizon such that the dynamics of the estimation error satisfy the given H ∞ performance requirement. Sufficient conditions are established for the existence of the desired estimator and the explicit expressions of the desired estimator parameters are then given in terms of the solutions to a set of recursive linear matrix inequalities. Finally, a numerical example is given to demonstrate the effectiveness of the developed state estimation scheme.

Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new filter based supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The filter based supervised feature selection method is a classifier-independent method while the TRC-LDA criterion is a recently developed criterion for dimensionality reduction that can well preserve discriminative information of dataset. However, there are seldom methods by utilizing TRC-LDA criterion for feature selection. On the other hand, imposing the l2,0-norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l2,0-nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we develop a new method, namely, Trace Ratio Criterion Discriminative Feature Selection (TRC-DFS), for feature selection. The proposed TRC-DFS has imposed l2,1-norm, i.e. an approximation of l2,0-norm, to the projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed TRC-DFS can both achieve feature selection as well as capture the discriminative structure of data. We also extend the proposed method with l2,p-norm (0

Twin extreme learning machine (TELM) is an efficient and effective method for pattern classification, based on widely known extreme learning machine (ELM). However, TELM is mainly used to deal with supervised learning problems. In this paper, we extend TELM to handle semi-supervised learning problems and propose a novel Laplacian twin extreme learning machine (LapTELM), which simultaneously trains two related and paired semi-supervised ELMs with two nonparallel separating planes for the final classification. The proposed method exploits the geometry structure property of the unlabeled samples and incorporates it as a manifold regularization term. This allows LapTELM to reap the benefits of fully exploring the plentiful unlabeled samples while retaining the learning ability and efficiency of TELM. Moreover, the paper shows that semi-supervised and supervised TELM can form an unified learning framework. Compared with several mainstream semi-supervised learning methods, the experimental results on the synthetic and several real-world data sets verify the effectiveness and efficiency of LapTELM.

The target of this paper is to design a model-free robust controller for uncertain systems. The uncertainties of the control system mainly consists of model uncertainty and external disturbance, which widely exist in the practical utilization. These uncertainties will negatively influence the system performance and this motivates us to train a model-free controller to solve this problem. Reinforcement learning is an important branch of machine learning and is able to achieve well performed control results by optimizing a policy without the knowledge of mathematical plant model. In this paper, we construct a reward function module to describe the specific environment of the concerned system, taking uncertainties into account. Then we utilize a new policy gradient method to optimize the policy and implement this algorithm with the actor-critic structure neuro networks. These two networks are our reinforcement learning controllers. Finally, we illustrate the applicability and efficiency of the proposed method by applying it on an experimental helicopter platform model, which includes model uncertainties and external disturbances.

This paper studies the consensus of multi-agent systems with faults and mismatches under switched topologies using a delta operator method. Since faults and mismatches can result in failure of the consensus even for a fixed topology with a spanning tree, how to reach a consensus is a complicated and challenging problem under such circumstances especially when part topologies have no spanning tree. Although some works studied the influence of faults and mismatches on the consensus, there is little work on reaching a consensus for the multi-agent systems with faults and mismatches. In this paper, we introduce the delta operator to unify the consensus analysis for continuous, discrete, or sampled systems under one framework. We develop the theories on the delta operator systems first and then apply theories of the delta operator systems to the consensus problems. By converting the consensus problems into stability problems, we investigate and prove consensus and the associated conditions for systems 1) without any fault, 2) with a known fault, and 3) with unknown faults, under switching topologies with matching or mismatching coefficients. Numerical examples are provided and validate the effectiveness of the theoretical results.

Detection of incipient faults is an important and challenging issue in the area of fault diagnosis and prognosis. This paper presents a new incipient fault detection approach for nonlinear dynamical systems via deterministic learning. Through defining and establishing the banks of health, sub-health and incipient fault modes, the incipient fault detectability condition is derived with the fault mismatch function. The system dynamics underlying three kinds of system modes are accurately approximated via deterministic learning firstly. Secondly, a bank of estimators is constructed using the learned modes. A set of residuals is achieved by comparing the bank of estimators with the monitored system. According to the smallest residual principle and the fault mismatch function, if the average L1 norm of the residual, which is associated with one of the incipient fault modes, is smaller than the others at a time instant td, the system incipient fault is detected. Finally, the incipient fault detectability is analyzed rigorously. Numerical simulation is investigated to demonstrate the effectiveness of the approach. The significance of the paper lies in that the learned system modes are utilized to achieve rapid detection in the initial stage of incipient fault for nonlinear dynamical systems.