Search

Deep learning has made substantial progress in crowd density estimation, but in practical application, due to the interference factors such as uneven distribution of crowd and changes in illumination, the existing methods still have large errors in counting. To solve the above problems, a crowd density estimation method based on multichannel dense grouping network is proposed. To solve the problem that it is difficult to extract the multiscale information of the crowd due to the uneven distribution of the crowd, multichannel dense grouping module (McDGM) is designed. In the module, Improved grouping convolution block (IGCB) is dense connected with other layers to obtain different levels of crowd characteristics, so as to avoid the loss of multi-scale information. In addition, the parameters of IGCB are reduced by group convolution. Then, to overcome the change of illumination, a crowd image enhancement algorithm is designed, which makes the image clear by the average pixel value and adjusting the contrast. Finally, to enhance the sensitivity of the network to crowd counting, a new loss function is proposed, which adds counting loss to the previous pixel space loss to improve the accuracy of crowd counting. The algorithm in this paper has been tested on ShanghaiTech and UCF_CC_50 datasets. The test results show that the algorithm in this paper has better statistical accuracy.

From the perspective of large-scale system, a directed complex dynamic network (DCDN) may be considered as a coupling system of the node subsystem (NS) and the link subsystem (LS). In this paper, by using the outgoing link vector and incoming link vector for DCDN, the dynamics of LS is described by employing the vector differential equation instead of the matrix differential equation. Since the outgoing and incoming link vectors have the stronger geometric intuition, the results in this paper show that this kind model of links can not only reflect the direction of links but also find the dynamic tracking goal of links more easily when the state synchronization of NS emerges. Furthermore, by employing the simple mathematical conditions, the nonlinear controller of NS and the coupling term of LS are proposed to ensure achieving the asymptotical state synchronization for DCDN. Finally, the numerical simulations are given to demonstrate the validity of the results in this paper.

In this paper, a neural synchronization of the optimal structure-based group of neural networks is proposed. Asymmetric cryptography is widely used to generate a key amongst two parties and to exchange the key through an insecure channel. However, since the methods that used this strategy, like RSA, have been compromised, new methods for producing a key that can offer security must be discovered. A new group of cryptography known as neural cryptography was created to solve this issue. The primary aim of neural cryptography is to produce a secret key over an unreliable medium. The optimal neural network architecture for creating and defining a secret key between the two authorized individuals is examined in this article. Furthermore, studies into the coordination of a group of neural networks are uncommon. For the design of the public key exchange protocol, synchronization of a cluster of neural networks with Three Layer Tree Parity Machine (TLTPM) is proposed. To calculate the synchronization time, steps taken, and the number of times the attacking neural network could replicate the actions of the two accepted networks, more than 15 million simulations were run. Various parametric experiments have been conducted on the proposed methodology. Simulations of the approach show that it is correct, according to the results of the paper.

This paper tackles the event-triggering filtering problem for a class of nonlinear networked control systems subject to the prescribed disturbance attenuation level. To alleviate the data transmission burden of communication channels, an event-triggering mechanism is adopted, where the data can be transmitted only if certain condition is fulfilled. The T-S fuzzy model approach is utilized to approximate the nonlinear dynamics. It is the purpose of this paper to design a filter for the addressed nonlinear system ensuring that the filtering error system is asymptotically stable whereas the prespecified H ∞ criterion is guaranteed. By using Lyapunov approach in combination with matrix analysis method, the sufficient conditions are established for the existence of desired filtering parameters in terms of the solvability of certain linear matrix inequalities. Finally, a numerical simulation example is put forward to demonstrate the validity and effectiveness of the developed theoretical results.

This paper investigates the motion control of a Stewart parallel robot with uncertain load. The external disturbances caused by load can trigger dynamic coupling among six degree-of-freedoms and have a significant impact on control precision. In practice, the dynamic parameters of load is difficult to be precedent identified and the robot with load is a multi-input multi-output(MIMO) system in physical space, which makes the dynamic coupling problem difficult to solve effectively and the high-performance control is hard to achieve. In this paper, the dynamic effect of the load is first analyzed. Then a novel motion control method, modal space neural network compensation control, is designed with the aim of reducing the effect of the load disturbances. This controller implements the neural network to compensate for the load disturbances. In addition, the modal space control theory is introduced to realize the independent control of each control channel. The feasibility of the proposed controller is evaluated in numerical simulations. Results reveal that the proposed control method gives exceptional tracking performance and dynamic coupling suppression capability.

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

This paper considers the leader-following consensus problem for delayed neural networks with multi-layer signed digraph topologies. Both structurally balanced and unbalanced topologies are considered. By using the structural balance theory and the pinning control strategy, some sufficient conditions are given to guarantee that the bipartite leader-following consensus can be reached in structurally balanced multi-layer signed neural networks, which depend on the node dynamics and the topology of the multi-layer signed network. In addition, a scheme is designed for selecting the pinning nodes and gains of each layer to achieve bipartite leader-following consensus. When the multi-layer signed network is structurally unbalanced, the asymptotic stability of the neural network is investigated by decomposing the structurally unbalanced network into a structurally balanced subgraph and some special edges. The conditions proposed in this paper are also suitable for addressing the consensus problem of traditional single layer signed neural networks. Numerical examples are given to illustrate the correctness and effectiveness of the proposed theorems and algorithms.

This paper considers the stochastic consensus problem of high-order linear discrete-time multiagent systems (MASs) with uncertain topologies and external disturbances. The randomly switching interaction signed topologies are governed by a time-homogenous Markov chain with incomplete knowledge of the transition probabilities. Based on Lyapunov function theory and adaptive control strategies, we employ a homogeneous parameter-dependent Lyapunov function to propose sufficient conditions under a set of linear matrix inequality (LMI) constraints. A free-weighting matrix method is introduced to give the proposed LMIs extra freedom to search feasible solutions. Furthermore, the adaptive controllers applied in this paper are independent of topologies and fully distributed; i.e., the system achieves robust consensus without continuously updating the controller and dealing with global data. Additionally, we extend the theoretical framework to analyze the disturbance rejection performance of MASs under the stochastic process. Two numerical examples are provided to show the effectiveness of the algorithms.

This paper presents a novel nonlinear adaptive sensor fusion method for integrated navigation systems with varying noise parameters. The innovation is utilizing deep neural networks to mine the noise-related patterns of specific sensors and combining it with conventional nonlinear filters. This hybrid approach improves the feasibility and robustness of adaptive filtering by achieving an effective estimation of the originally weakly observable noise parameters. The specific sensors are defined as α -type sensors whose errors are entirely generated by themselves. The mathematical model for analyzing α -type sensors output sequence and the deep neural network for mining the patterns of interest are established. All adaptive filtering systems using α -type sensors can benefit from this paper. Specifically, it is applied to inertial and satellite integrated navigation system. The numerical experiments indicate that the proposed filter achieves promising accuracy and robustness improvement as compared to conventional nonlinear filters. And the comparisons between different nonlinear approximation algorithms indicate that the first-order approximation is accurate enough for our application.

This paper presents the neural dynamical network to compute the generalized and restricted singular value decompositions (GSVD/RSVD) in the regularization methods for ill-posed problems. The neural network model is defined by ordinary differential equations (ODE) which can be solved by many state-of-the-art techniques. The main purpose of the paper is to develop two neural network models for finding approximations of the GSVD and the RSVD. The globally asymptotic stability analyses are provided and numerical experiments illustrate our theory and methods. For small scale problems, the estimation can be as accurate as O ( 10 - 15 ) . For ill-posed problems, the truncation regularization method implementing the GSVD/RSVD algorithms also produces accurate results.

Time-varying linear complementarity problem (TLCP) has received a great deal of attention due to its broad variety of scientific and engineering applications. Several efficient Zhang neural networks are introduced for solving TLCP in this paper. Theoretical analysis shows that the related error function of the model proposed in this paper eventually tends to zero. The state convergence time periods of those Zhang neural networks with three types of activation functions are proved to be finite and can be quantitatively estimated by using some given parameters. Further, it is shown that the proposed neural network is of noise-tolerance, which means the neural network is more appropriate for a wider application. Moreover, in order to implement neural network numerically, a related discrete-time version is also studied. Finally, numerical simulations confirm the analysis of the proposed models concretely.

Irregular time series data is becoming increasingly prevalent with the growth of multi-sensor systems as well as the continued use of unstructured manual data recording mechanisms. Irregular data and the resulting missing values severely limit the data's ability to be analysed and modelled for classification and forecasting tasks. Often, conventional methods used for handling time series data introduce bias and make strong assumptions on the underlying data generation process, which can lead to poor model predictions. Traditional machine learning and deep learning methods, although at the forefront of data modelling, are at best compromised by irregular time series data sets and fail to model the temporal irregularity of incomplete time series. Gated recurrent neural networks (RNN), such as LSTM and GRU, have had outstanding success in sequential modelling, and have been applied in many application fields, including natural language processing. These models have become an obvious choice for time series modelling and a promising tool for handling irregular time series data. RNNs have a unique ability to be adapted to make effective use of missing value patterns, time intervals and complex temporal dependencies in irregular univariate and multivariate time series data. In this paper, we provide a systematic review of recent studies in which gated recurrent neural networks have been successfully applied to irregular time series data for prediction tasks within several fields, including medical, human activity recognition, traffic monitoring and environmental monitoring. The review highlights the two common approaches for handling irregular time series data: missing value imputation at the data pre-processing stage and modification of algorithms to directly handle missing values in the learning process. Reviewed models are confined to those that can address issues with irregular time series data and does not cover the broader range of models that deal more generally with sequences and regular time series. This paper aims to present the most effective techniques emerging within this branch of research as well as to identify remaining challenges, so that researchers may build upon this platform of work towards further novel techniques for handling irregular time series data.

This paper solves the fault-tolerant control (FTC) problem of uncertain nonlinear systems in nonstrict-feedback form. The controlled plants considered in this paper are more complicated than existing ones, which is composed of the unknown nonlinearities, unmeasured states, and sensor faults and unknown dead zone input nonlinearity. Fuzzy logic systems and a fault-estimation-based state observer are used for identifying the unknown nonlinearities and obtaining the unmeasured states, respectively. By designing the sensor faults compensation and the dead zone inverse compensation methods, an observer-based fault-tolerant compensation control scheme is developed under backstepping recursive design frame. And it is testified that the closed-loop signals are bounded, and the tracking performance is satisfied even when the controlled systems are not free of sensor faults and unknown dead zone input nonlinearity. An electromechanical system is used to test the effectiveness of the developed control strategy.

This paper proposes a solution to translating the visible images into infrared images, which is challenging in computer vision. Our solution belongs to unsupervised learning, which has recently become popular in image-to-image translation. However, existing methods do not produce satisfactory results because (1) most existing methods are mainly used in entertainment scenarios with single scenes and low complexity. The problem solved by this article is more diverse and more complicated. (2) The infrared response of objects depends not only on itself but also on the current environment, and existing methods cannot correlate long-range dependent objects. In this paper, We propose Global Spatial Attention (GSA), which enhances dependence between long-range objects and improves the synthesized image quality. Compared with other methods, GSA can save more space and time. Moreover, we introduce the idea of subspace learning into the neural network to make training more stable. Our method takes unpaired visible images and infrared images for training, which are easy to collect. Experimental results show that our method can generate high-quality infrared images from visible images and outperforms state-of-the-art methods.

Road landmark inventory is becoming an important data product for the maintenance of transport infrastructures. Several commercial sensors are available which include synchronized optical cameras that allowto build 360° panoramic images of the surroundings of the vehicle used for road inspection. This paper is devoted to the analysis of such panorama images,specifically the area that contains themost relevant information. Road lane landmark detection is posed as a two class classification problem that may be solved bymachine learningapproaches, such as Random Forest (RF) and ensembles of Extreme Learning Machines (V-ELM). Besides model parameter selection, a central problem is the construction of a labeled training and validation datasetto cope with the highly uncontrolled conditions of image capture. Besides, human labor cost makes image data labeling a very expensive process. This paper proposes an open ended Active Learning (AL) approach involving a human oraclein the loop who provides the data labeling and can trigger the AL process when detection quality is degraded by the change in imaging conditions. The paper reports encouraging results over a collection of sample images selected from an industrial road landmark inventory operation. As an additional contribution, this paper assesses the ability of AL to overcomesome of the issues raised by highly class imbalanced datasets.

Multi-view clustering has gained soar attention in the field of data mining in recent years. Most multi-view clustering methods are considered to be complete for each view. However, in many applications, some views may contain missing instances, resulting in the incomplete multi-view learning problem. Incomplete views cannot be directly dealt with traditional multi-view clustering methods. In this paper, we propose a One-step multi-view subspace clustering with incomplete views (OMVSC-IV) method based on low-rank matrix factorization to overcome this problem. Specifically, the proposed algorithm uses low-rank matrix factorization to learn a consensus representation matrix, and then combined with the objective function of non-negative embedding and spectral embedding subspace clustering proposed in this paper. The whole process is established into the same objective function for joint optimization and requires no post-processing (e.g., K-means), thus avoiding the defect of being sensitive to initial values. Furthermore, the accuracy, parameter, sensitivity and convergence of the algorithm are studied by experiments. Experimental results on nine data sets show that the proposed algorithm performs superior to some of the latest algorithms.

Depth estimation is a classic task in computer vision, which is of great significance for many applications such as augmented reality, target tracking and autonomous driving. Traditional monocular depth estimation methods are based on depth cues for depth prediction with strict requirements, e.g. shape-from-focus/ defocus methods require low depth of field on the scenes and images. Recently, a large body of deep learning methods have been proposed and has shown great promise in handling the traditional ill-posed problem. This paper aims to review the state-of-the-art development in deep learning-based monocular depth estimation. We give an overview of published papers between 2014 and 2020 in terms of training manners and task types. We firstly summarize the deep learning models for monocular depth estimation. Secondly, we categorize various deep learning-based methods in monocular depth estimation. Thirdly, we introduce the publicly available dataset and the evaluation metrics. And we also analysis the properties of these methods and compare their performance. Finally, we highlight the challenges in order to inform the future research directions.

With the advent of smart phones, the popularity of free Android applications has risen rapidly. This has led to malicious Android apps being involuntarily installed, which violate the user privacy or conduct attack. Malware detection on Android platforms therefore is a growing concern because of the undesirable similarity between malicious behavior and benign behavior, which can lead to slow detection, and allow compromises to persist for comparatively long periods of time in infected phones. The contributions of this paper are first a multiple dimensional, kernel feature-based framework and feature weight-based detection (WBD) designed to categorize and comprehend the characteristics of Android malware and benign apps. Furthermore, our software agent is orchestrated and implemented for the data collection and storage to scan thousands of benign and malicious apps automatically. We examine 112 kernel attributes of executing the task data structure in the Android system and evaluate the detection accuracy with a number of datasets of various dimensions. We find that memory- and signal-related features contribute to more precise classification than schedule-related and other descriptors of task states listed in our paper. Particularly, memory-related features provide fine-grain classification policies for preserving higher classification precision than the signal-related and others. Furthermore, we study and evaluate 80 newly infected attributes of the Android kernel task structure, prioritizing the 70 features of most significance based on dimensional reduction to optimize the efficiency of high-dimensional classification. Our second contribution is that our experiments demonstrate that, as compared to existing techniques with a short list of task structure features (16 or 32 features), our method can achieve 94%-98% accuracy and 2%–7% false positive rate, while detecting malware apps with reduced-dimensional features that adequately abbreviate online malware detections and advance offline malware inspections.

Recently, due to its widespread applications in public safety, anomaly detection in crowd scenes has become a hot topic. Some deep-learning-based methods attain significant achievements in this field. Nevertheless, most of them suffer from over-fitting to some extent because of scarce data, which are usually abrupt and low-frequency in the real world. To remedy the above problem, this paper firstly develops a synthetic anomaly event generating system, which could simulate typical specific abnormal events. By utilizing this system, a large synthetic, diverse anomaly event dataset is built, which contains 2,149 video sequences. After getting the dataset, a 3D CNN is designed to detect the abnormal types at the video level. However, we find that there are obvious domain differences (also named as “domain gap/shifts”) between synthetic videos and real-world data, which results in performance degradation when applying the model to the real world. Thus, this paper further proposes a cyclic 3D GAN for domain adaption to reduce the domain gap, which translates the synthetic data to the photorealistic video sequences. Then the detection model is trained on the translated data and it can perform well in the real data. Experimental results illustrate that the proposed method outperforms these baselines for the domain adaptation anomaly detection.

This paper discusses the containment control (CC) problem of general linear multi-agent systems (MAS)s by means of two kinds of distributed event-triggered mechanisms. Two types of event-triggered control protocols, namely, the state feedback control law and the dynamic output feedback control protocol, are designed for each follower. Under the proposed control protocols and triggering mechanisms, the containment control problem can be solved by proving that the containment error converges to zero based on the assumption and algorithms. At last, we verify the rationality of the theoretical results. Through two numerical simulation, we can see that the trajectory of each follower converges to the convex hull formed by all leaders. In addition, in order to verify the advantages of the obtained results, we give a simulation example and compare the methods designed in this paper with the methods in other literature.

While meta learning approaches have achieved remarkable success, obtaining a stable and unbiased meta-learner remains a significant challenge, since the initial model of a meta-learner could be too biased towards existing tasks to adapt to new tasks. In order to avoid a biased meta-learner and improve its generalizability, this paper proposes a generic meta learning method that aims to learn an unbiased meta-learner towards a variety of tasks before its initial model is adapted to unseen tasks. Specifically, this paper presents a meta weight learning method for minimizing the inequality of performance across different training tasks. An end-to-end training approach is introduced for the proposed algorithm that allows for effectively learning weight and initializing the network model. Alternatively, a variety of measurement methods of weight is also designed to test the effectiveness of different weight learning methods on the improvement of model-agnostic meta-learning algorithm. The simulation results show that the proposed meta weight learning method not only outperforms state-of-the-art meta learning algorithms, but also is superior to other manually designed measurement methods of weight on discrete and continuous control problems.

Human intention prediction is of great significance in many applications, such as human-robot interaction, intelligent rehabilitation robots. This paper studies the problem of short-term next-active-object prediction in egocentric images. The short-term next-active-object refers to the object that a human is going to interact with in the short-term future, which is an embodiment of human intention. Most current methods usually use object-centered cues, such as the deviation of object appearance change and the unique shape of the egocentric object trajectory, to predict the next-active-object. In this paper, inspired by the fact that human intention is also revealed by human-centered cues, we propose a deep neural network model that integrates the cues from visual attention and hand positions to predict the next-active-object. Firstly, the probability maps of visual attention and hand positions are constructed, and then the probability distribution of next-active-object is generated. We experimentally compare our method with several baseline methods using two datasets and confirm its effectiveness. In addition, ablation experiments are conducted, and crucial points concerning the next-active-object are discussed.

Peer reviews form an essential part of scientific communications. Research papers and proposals are reviewed by several peers before they are finally accepted or rejected for publication and funding, respectively. With the steady increase in the number of research domains, scholarly venues (journal and/or conference), researchers, and papers, managing the peer review process is becoming a daunting task. Application of recommender systems to assist peer reviewing is, therefore, being explored and becoming an emerging research area. In this paper, we present a deep learning network based Meta-Review Generation considering peer review prediction of the scholarly article (MRGen). MRGen is able to provide solutions for: (i) Peer review prediction (Task 1) and (ii) Meta-review generation (Task 2). First, the system takes the peer reviews as input and produces a draft meta-review. Then it employs an integrated framework of convolution layer, long short-term memory (LSTM) model, Bi-LSTM model, and attention mechanism to predict the final decision (accept/reject) of the scholarly article. Based on the final decision, the proposed model MRGen incorporates Pointer Generator Network-based abstractive summarization to generate the final meta-review. The focus of our approach is to give a concise meta-review that maximizes information coverage, coherence, readability and also reduces redundancy. Extensive experiments conducted on the PeerRead dataset demonstrate good consistency between the recommended decisions and original decisions. We also compare the performance of MRGen with some of the existing state-ofthe- art multi-document summarization methods. The system also outperforms a few existing models based on accuracy, Rouge scores, readability, non-redundancy, and cohesion.

The problem of distributed fixed-time leader-following consensus tracking control for nonholonomic chained-form dynamic systems with nonlinear disturbances is considered in this paper. Compared with the existing works, this paper considers a class of more general nonholonomic multi-agent systems where both the leader and the followers have nonlinear uncertainties. For undirected graphs, a distributed observer for each follower is investigated such that both the leader state and input can be estimated by the followers in a fixed time. Based on the distributed observers, a consensus tracking controller is proposed to ensure that the tracking errors convergence to zero within a fixed time. Lyapunov analysis proves the stability of the closed-loop system. A simulation example of wheeled mobile robots is given to demonstrate the effectiveness of the proposed controllers.

This paper investigates the problem of neural network (NN) prescribed performance tracking control for a class of switched stochastic nonlinear pure-feedback systems which contain unknown nonlinear functions, unmeasured state variables, and unknown hysteresis input. It provides an adaptive NN controller which is not restricted to a particular type of hysteresis input. A general mathematical model is introduced to describe two kinds of hysteresis nonlinearities and to utilize in the control design producer. Other focus of this paper is on the performance constraint problem to avoid performance degradation and system damage in practical control systems. Prescribed performance control (PPC) and backstepping technique are thus synthesized to develop an adaptive NN output feedback tracking control scheme under deterministic switching signal. Regarding this concern, to cope with the cause of non-differentiable difficulties and complex deductions in traditional PPC, a new asymmetry error transformation is employed. Moreover, radial basis function NNs (RBFNNs) are applied to approximate the unknown nonlinear functions and to construct a NN nonlinear observer to estimate the immeasurable state variables. Based on Lyapunov stability theory, it is demonstrated that the proposed controller can guarantee that all signals in the closed-loop system are semiglobally uniformly ultimately bounded in probability and the tracking error converges to a small neighborhood of the origin with the prescribed performance bounds. Finally, two simulation examples are provided to confirm the advantages of the presented control design approach.

With the booming development of smart phones and location-based services, spatial keyword queries of semantic trajectories can take location information and multiple keywords as parameters and return semantic trajectories related to these parameters in space and text. With the development of the Location Based Social Network (LBSN) in recent years, social data can be used to improve query results. Generally, the semantic trajectory of having more supporters (users with positive comments about the location) who are more closely related to the query user is more likely to be the recommended result. Considering social information in the query can recommend more satisfactory results for users, this paper studies a new problem of semantic trajectory space keyword query: Social space Keyword Query based on semantic Trajectory (SKQT). This paper proposes a hybrid index structure that introduces social factors called Social Inverted Linear Quadtree (SIL-Quadtree) to improve the efficiency of SKQT query. The experiments use two datasets, Gowalla and Brightkite. The results show that the algorithm and index structure proposed in this paper perform better on running time and I/O.

The choice of activation functions in Convolutional Neural Networks (CNNs) is very important. Rectified Linear Unit (ReLU) has been widely-used in most CNNs. Recently, a series of non-monotonic activation functions gradually become the new standard to enhance performance of CNNs. Inspired by them, this paper firstly proposes a novel non-monotonic activation function called Power Function Linear Unit (PFLU). The negative part of PFLU is non-monotonic and closer to zero with the negative input decreasing, which can maintain sparsity of the negative part while introducing negative activation values and non-zero derivative values for the negative part. The positive part of PFLU does not use identity mapping but is closer to identity mapping with the positive input increasing, which can bring non-linearity property for the positive part. Next, this paper proposes faster PFLU (FPFLU). A wide range of classification experiments show that PFLU tends to work better than current state-of-the-art non-monotonic activation functions, and FPFLU can run faster than most non-monotonic activation functions.

Bayesian networks have many applications. Learning the optimal structure of a Bayesian network has always been important in this respect. In this paper, a bidirectional heuristic search algorithm is proposed for the order graph space commonly used in a Bayesian network. At the same time, heuristic functions that are admissible and consistent in terms of both forward and backward search are proposed to ensure convergence of the algorithm to the optimal solution. The experimental results show that, compared with traditional unidirectional heuristic search, in most cases, the bidirectional heuristic search proposed in this paper needs to expand fewer states, the convergence efficiency is higher, and less running time is needed.

Local Differential Privacy (LDP) can provide each user with strong privacy guarantees under untrusted data curators while ensuring accurate statistics derived from privatized data. Due to its powerfulness, LDP has been widely adopted to protect privacy in various tasks (e.g., heavy hitters discovery, probability estimation) and systems (e.g., Google Chrome, Apple iOS). In particular, ( ∊ , δ ) -LDP has been studied in related statistical tasks like private learning and hypothesis testing, but is mainly achieved by using Gaussian mechanism, leading to the limited data utility. In this paper, we investigate several novel mechanisms that achieve ( ∊ , δ ) -LDP with higher data utility in collecting and analyzing users’ data. Specifically, we first design two ( ∊ , δ ) -LDP algorithms for mean estimations on multi-dimensional numeric data, which can ensure higher accuracy than the optimal Gaussian mechanism. Then, we investigate different local protocols for frequency estimations on categorical attributes under ( ∊ , δ ) -LDP. Based on the proposed mechanisms, we further study on ( ∊ , δ ) -LDP-compliant stochastic gradient descent algorithms for machine learning models. Besides, the theoretical analysis of the error bound and the variance of the proposed algorithms are also presented in the paper. We have conducted extensive experiments on both real-world and synthetic datasets and demonstrated the high data utility of our proposed algorithms in the perspectives of simple data statistics tasks and complex machine learning tasks. The experimental results have shown that our proposed algorithms can effectively improve the data utility in different tasks while alleviating the privacy concerns of each individual.

This paper investigates the formation containment control under sampling and with time delays for multi-agent systems (MASs). First, a novel sampling control protocol for the MASs to achieve formation containment is proposed, where the communication among agents is only occurred at the sampling instance. Then the communication load and the waste of energy can be decreased significantly. In addition, time delays are considered in this paper. Second, by establishing an appropriate Lyapunov function, sufficient conditions for MASs with sampled data and time delays reaching formation containment are obtained by handling the established Lyapunov function. Finally, the obtained results are demonstrated by a simulation example.

The natural concept ontology structure of clothes has enabled easy management of large quantities of fashion images for online retailers and it is meaningful to study how to automatically recognize fashion images for both commercial promotion and academic research. In this paper, a new hierarchical approach is developed for large-scale fashion recognition. We first embed concept ontology into deep convolutional neural network (CNN) by adopting multiple deep CNN branches to learn node-specific features and classifiers explicitly. Then, we introduce a hierarchical knowledge distillation method to further improve the performance of fashion recognition. Finally, we employ the proposed approach for fashion recommendation. To deal with hierarchical deep learning constrains, we leverage back propagation to simultaneously refine the shared deep CNNs and the diverse CNN branches for relevant node features and classifiers by using our joint objective function. The main advantages of this paper lie in (1) providing an effective way for recognizing rich semantic explanations of fashion images without training large or multiple networks, and (2) saving the storage&time costs by learning personalized features and classifiers for each tree node. The experimental results on both our organized fashion dataset and the public DeepFashion dataset have verified the effectiveness and efficiency of the proposed approach on both hierarchical fashion recognition and within category fine-grained fashion recommendation.

For time series prediction, hybrid systems that combine linear and nonlinear models can provide more accurate performance than a single model. However, the irregularity of the error series and the unknown nature of combinations of different forecasters may strongly impact the performance of hybrid systems. Therefore, in this paper, we propose a novel method for time series prediction, in which error decomposition and a nonlinear combination of forecasters are introduced. The proposed method performs the following: (i) linear modeling to obtain the error series, (ii) error decomposition by using variational mode decomposition (VMD), (iii) nonlinear modeling and a phase fix procedure for the error subseries, and (iv) a combination of forecasters through an appropriate combination function generated by a nonlinear model. By using the proposed method, this paper constructs two hybrid systems, in which the autoregressive integrated moving average (ARIMA) is used for linear modeling, and two artificial intelligence (AI) models, namely, the multilayer perceptron (MLP) and support vector regression (SVR), are used for nonlinear modeling and combination, respectively. Finally, four time series data sets, six evaluation metrics, two single models and thirteen hybrid systems are used to assess the effectiveness of the proposed method. The empirical results show that hybrid systems based on error decomposition and a nonlinear combination of forecasters can achieve better performance than some existing systems and models.

Graph Neural Networks(GNNs) are powerful methods to analyze the non-Euclidean data. As a dominant type of GNN, Graph Convolutional Networks(GCNs) have wide applications. However, the analysis of the generalization error for GCNs with multilayer is limited. Based on the review of single-layer GCNs, this paper analyzes the generalization error of two-layers GCNs and extends the conclusion to the general GCNs models. Firstly, this paper examines two-layers GCNs and obtains the stability of the GCNs algorithm. And then, based on this algorithmic stability, the generalization stability of multilayer GCNs is obtained. This paper shows that the algorithmic stability of GCNs depends upon the graph filters and its product with node features as well as the training procedure. Furthermore, the generalization error gap of GCNs tends to be enlarged with more layers, which can interpret why GCNs with deeper layers have relatively poorer performance in test datasets.

Real-time pricing (RTP) is the pivotal component of demand response (DR) that promotes the power utilization of users in a cost-efficient way. This paper presents a reward fairness-based distributed RTP (RFbDRTP) framework for electric users to reduce the electricity consumption cost as well as experience the fairness from participating in DR. In RFbDRTP, a behavior welfare model (BWM) is developed to exploit the reward fairness strategy that returns the cost savings as a reward to the users in proportion to their load increase or decrease in DR. The model can motivate users to react to the price signal through adjusting their electricity consumption patterns, to in turn enable optimal distributed RTP for matching supply–demand. In this paper, users are assumed to interact with each other because of pricing based on the imbalance between load demand and electricity supply. We formulate the interactions among the users into a noncooperative game and give a sufficient condition to ensure the unique equilibrium in the game. After that, we develop a distributed algorithm and give a sufficient convergence condition of the algorithm. The simulation results show that the proposed RFbDRTP is effective in motivating users to participate in DR and matching demand with supply, and the distributed algorithm can converge to the equilibrium with a significant convergence rate.

Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering. These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue, we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different architectures as different "views" of the same data. This approach is based on the assumption that information contained in the different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets, and produces state-of-the-art results for IC., Comment: 21 pages, 16 figures, 10 tables, preprint of our paper published in Neurocomputing

Analyzing the runtime behaviors of Android apps is crucial for malware detection. In this paper, we attempt to learn the behavior level features of an app from function calls. The challenges of this task are twofold. First, the absence of function attributes hinders the understanding of app behaviors. Second, the graphical representation of function calls cannot be directly processed by classical machine learning algorithms. In this paper, we develop two methods to overcome these challenges. Based on function embedding, we first propose the concept of enhanced function call graphs (E-FCGs) to characterize app runtime behaviors. We then develop a Graph Convolutional Network (GCN) based algorithm to obtain vector representations of E-FCGs. Extensive experiments show that the features learned by our method can achieve surprisingly high detection performance on a variety of classifiers (e.g., LR, DT, SVM, KNN, RF, MLP and CNN), significantly outperforming the traditional static features.

Inspired by the recurrent highway networks, we study the coupled networks as a modular behavior of the networks, which can better improve the performance of the system. For a neural network, the non-zero continuous attractors play an important role to decode the information. The emphasis of this paper is to study the conditions for the generation of non-zero continuous attractors in the coupled new system theoretically. By exploring the relationship between network parameters, the conditions for generating non-zero continuous attractors in the new model are successfully determined. In order to verify the correctness and validity of the proposed conclusions, the results of this paper are further verified by simulation.

In this paper, we propose a method for obtaining sentence-level embeddings. While the problem of securing word-level embeddings is very well studied, we propose a novel method for obtaining sentence-level embeddings. This is obtained by a simple method in the context of solving the paraphrase generation task. If we use a sequential encoder-decoder model for generating paraphrase, we would like the generated paraphrase to be semantically close to the original sentence. One way to ensure this is by adding constraints for true paraphrase embeddings to be close and unrelated paraphrase candidate sentence embeddings to be far. This is ensured by using a sequential pair-wise discriminator that shares weights with the encoder that is trained with a suitable loss function. Our loss function penalizes paraphrase sentence embedding distances from being too large. This loss is used in combination with a sequential encoder-decoder network. We also validated our method by evaluating the obtained embeddings for a sentiment analysis task. The proposed method results in semantic embeddings and outperforms the state-of-the-art on the paraphrase generation and sentiment analysis task on standard datasets. These results are also shown to be statistically significant., Comment: This work is an extension of our COLING-2018 paper arXiv:1806.00807

This paper studies the blind super-resolution of real low-quality and low-resolution (LR) images. Existing convolutional network (CNN) based approaches usually learn a single image super-resolution (SISR) model for a specific downsampler (e.g., bicubic downsampling, blurring followed by downsampling). The learned model, however, is tailored to the specific downsampler and fails to super-resolve real LR images which are degraded in more sophisticated and diverse manners. Moreover, the ground-truth high-resolution (HR) of real LR images are generally unavailable. Instead of learning from unpaired real LR-HR images or a specific downsampler, this paper learns blind SR network from a realistic, parametric degradation model by considering blurring, noise, downsampling, and even JPEG compression. In contrast to direct blind reconstruction of HR image, the proposed model adopts a cascaded architecture for noise estimation, blurring estimation, and non-blind SR, which can be jointly end-to-end learned from training data and benefit generalization ability. By taking the bicubicly upscaled LR image as input to non-blind SR, the proposed method can present a single unified model for blind SR with any upscaling factors and varying degradation parameters. Experimental results show that the proposed method performs favorably on synthetic and real LR images.

Vehicle re-identification (re-ID) has numerous applications in real life, such as video surveillance, information retrieval, and public security. However, it suffers from the misalignment of vehicles, which is a critical problem caused by inaccurate detection and different views. This paper proposes a novel network named Part Alignment Network (PAN) which can properly handle the misalignment of vehicles. It is therefore adapted to the vehicle re-ID task, avoiding the use of any additional pre-processing steps such as the annotation of vehicle key points and part segmentation by hand. In PAN, cross-correlation is adopted to the alignment of vehicle parts. Then, an effective network architecture is designed to extract the discriminative aligned features. By combining complementary aligned features and original features, more robust feature representations are learned. To show the effectiveness of PAN, this paper conducts experiments on three vehicle re-ID databases (VD1, VD2, and VehicleID), on which it improves the current state-of-the-art performance.

In this study, the novel fully distributed asynchronous event-triggered protocols that use sampled state information are developed to address the leader-follower and leaderless consensus issues for the general linear multi-agent systems with physically interconnected network. By designing the state estimate of each follower agent, the protocols avoid the requirements for the continuous communication. Meanwhile, this paper proves that the designed protocols don’t exhibit the Zeno behavior in any finite time by giving a strictly positive lower-bound of the minimum inter-event interval. In contrast to the existing related papers, the salient features of this study are that both physically interconnected network and communication network of the multi-agent systems are considered and the developed protocols don’t require global network information, which reduce the conservativeness of previous systems modeling and protocols. Finally, two simulation results are provided to display the performance and effectiveness of the protocols.

The time-varying Lyapunov equation is an important problem that has been extensively employed in the engineering field and the Zeroing Neural Network (ZNN) is a powerful tool for solving such problem. However, unpredictable noises can potentially harm ZNN’s accuracy in practical situations. Thus, the comprehensive performance of the ZNN model requires both fast convergence rate and strong robustness, which are not easy to accomplish. In this paper, based on a new neural dynamic, a novel Noise-Tolerance Finite-time convergent ZNN (NTFZNN) model for solving the time-varying Lyapunov equations has been proposed. The NTFZNN model simultaneously converges in finite time and have stable residual error even under unbounded time-varying noises. Furthermore, the Simplified Finite-te convergent Activation Function (SFAF) with simpler structure is used in the NTFZNN model to reduce model complexity while retaining finite convergence time. Theoretical proofs and numerical simulations are provided in this paper to substantiate the NTFZNN model’s convergence and robustness performances, which are better than performances of the ordinary ZNN model and the Noise-Tolerance ZNN (NTZNN) model. Finally, simulation experiment of using the NTFZNN model to control a wheeled robot manipulator under perturbation validates the superior applicability of the NTFZNN model.

This paper focuses on the quasi-fixed time synchronization of memristive Cohen-Grossberg neural networks with reaction diffusion. First, a new definition of finite/fixed-time attraction set and the concept of quasi finite/fixed-time synchronization are proposed. Then, a quasi fixed-time synchronized lemma is given and an example is used to illustrate the correctness of the lemma. At the same time, the new results given in this article are compared with the existing ones under certain conditions, and the conclusions of this paper are less conservative in remark. In addition, quasi fixed-time synchronization theorem and full synchronization theorem are derived by designing different controllers, finally two examples are given to illustrate the validity of these theorems.

This paper studies the problem of adaptive output-feedback formation tracking control for networked uncertain nonholonomic mobile robots (NMRs) with different limited communication distances, while achieving connectivity preservation and collision avoidance. In robot control, equipping all the speed sensors not only increases the cost of the system but also risks losing accuracy and reliability. In this paper, it is assumed that the velocities of the nonholonomic mobile robots in this paper are unknown, they are estimated according to the output of the systems by an adaptive observer via a neural network. The designed dynamic surface based on nonlinear transformation errors can achieve the above three control objectives at the same time, avoiding the situation of using multiple potential functions to solve such problems, which will lead to conflicts in the selection of the design parameters. Compared with the existing references about formation tracking to maintain connectivity and avoid collisions, the main contribution of this paper is to design an observer of robot speed estimation and use only a neural network to estimate the unknown nonlinear term of the system itself. The unknown nonlinearity produced in the process of controller design does not need an additional neural network. Furthermore, the effectiveness of the proposed strategy is verified by simulation examples.

In this paper, five evaluation indexes were determined for the evaluation of the node importance of the eco-industrial system, and a comprehensive evaluation method based on the network load was established by introducing the cascade effect model of complex network. Then, by combining the process of industrial production and the operation status of each raw material and production point, the paper analyzes the important sites by using the model principle of cascade effect in the actual production, and designs the optimization scheme on the basis of saving the cost of the enterprise.

In the past few decades, numerous studies have demonstrated the effectiveness of diet modification in preventing and controlling different chronic diseases, bringing increased attention to the creation of nutritional recipes. Currently, the development of recipes is primarily based on personal experience rather than nutritional specifications. To this end, this paper first uses kernel canonical correlation analysis to demonstrate the relationship between nutrients in food and recipes. Based on this relationship, a new recipe expression method is put forward, which objectively reflects differing importance of an ingredient in differing recipes. Then, recipes are composed using an auto-encoder in deep learning, and a fusion model of two auto-encoders is proposed for better concocting recipes. This paper uses two machine learning methods, namely, non-negative matrix factorization and two-step regularized least squares, to form recipes. To tackle overfitting and instability in non-negative matrix factorization during the training of recipe model, we introduce the Frobenius norm to redefine the objective function and add non-smooth sparse matrices. Similar food has similar taste, but their nutrients might differ. This paper also considers nutrients as a kernel matrix of the two-step regularized least squares, which can effectively avoid the occurrence of different food combinations with similar taste. Experimental results show that developing recipes based on nutrients in food is feasible and effective in the context of machine learning.

This paper pays attention to the synchronization issue of complex-valued memristive neural networks (CVMNNs) that contain reaction–diffusion terms and Markovian jump parameters. To better meet the needs of some practical projects, the problem of synchronization investigated in this paper is defined in finite time and fixed time; namely, by designing a suitable controller, the synchronization error system can converge to zero in a finite/fixed time. Additionally, by combining Lyapunov–Krasovskii functional theory and algebraic inequality methods, a novel criterion of finite/fixed-time synchronization for proposed drive and response systems is derived. Note that the finite-time and fixed-time synchronization criteria are integrated into a unified theorem. Finally, two examples are provided, one is a simple numerical example to account for the feasibility of main results, the other realizes the application of this paper in establishing a spatiotemporal chaotic cryptosystem for image encryption, and the proposed cryptosystem has been illustrated that it has obvious advantages of large key space and high security by simulation results.

The prediction of short term power load owns a great influence on performance of the whole electric system. Raising the result of power load forecasting is always a research spot. This paper proposes a method combined Elman neural network (ENN) and the particle swarm optimization (PSO) for the short-term power load forecasting. Firstly, this paper introduces the principle of ENN and PSO algorithm respectively and analyzes performance of network influenced by parameters of ENN. Then the particle swarm optimization algorithm is applied to searching the optimal learning rate of ENN. To study the capability of the method proposed in this paper, the method is utilized for the short-term power load forecasting, which is suitable to be solved by ENN. Besides, a comparison experiment on this method (PSO–ENN) with general regression neural network (GRNN), the original ENN and the traditional back-propagation neural network (BPNN) is given out to illustrate effectiveness of PSO–ENN.

Accurate prediction of remaining useful life (RUL) for lithium-ion battery (LIB) plays a key role in increasing the reliability and safety of battery related industries and facilities. In this paper, RUL prediction of LIB is investigated by employing a hybrid data-driven method based support vector regression (SVR) and error compensation (EC). Firstly, two health indicators (HIs) are established by using capacity and discharging voltage difference of equal time interval (DVD), respectively. Secondly, the ensemble empirical mode decomposition (EEMD) is adopted to preprocess the obtained HIs, which is used to reduce the influence of capacity regeneration and noise. Especially, phase space reconstruction (PSR) with C–C technique is introduced to achieve optimal input sequence selection pattern, it has an important influence on the accuracy of SVR prediction. As an important innovation of the paper, the idea of EC is implemented by combining the predictions of both forecast error and RUL prediction with PSR-SVR. Last but not least, the genetic algorithm (GA) is utilized to optimize the key parameters of SVR so as to achieve more accurate RUL prediction. To verify the effectiveness of the proposed approach, the real data set of LIBs from National Aeronautics and Space Administration (NASA) is carried out, and the dominant is emphasized by comparison with other important methods.

In order to address such negative issues as GAN’s mediocre image quality, high-demand for training samples and computation resources, this paper proposes the collaborative Generative Adversarial Network with Visual perception and memory reasoning (ESA-CGAN). This not only makes use of the vision self-attention mechanism and objects salience model to analyze the global information and detailed features of objects, but also designs cross-correlation self-attention module so as to make a balance between the computation efficiency and computational on the one hand and the statistical efficiency and the ability to simulate remote dependencies on the other hand. Based on convolutional long-term and short-term memory network, this paper optimizes the attention feature map so as to highlight the features of objects themselves and improve their generative abilities. Meanwhile, a cooperative learning mechanism between generators is designed, which combines self-constructed generation model and pre-training generation model to form a generation model group. It not only improves effectively the generation ability of the model, improve the computing efficiency, but also restrains the collapse of the model from another angle. In fact, the model proposed here has completed numerical experiments on multiple common standard datasets and self-configuring datasets, and has made comparisons with several mainstream generation antagonistic network models in terms of the performance of image data augmentation. The experimental results demonstrate that the model has excellent simulation ability to enable itself to effectively realize the augmentation of data, thus making it highly applicable in the future.