Search

Rain removal from a single image is a challenging and significant task of image pre-processing. In this paper, we learn the multi-scale streaks from rainy images using feature pyramid, and to improve the effectiveness of the learning, we focus on the feature propagation re-usage and propagation in the extremely deep de-raining network. Specifically, we design a de-raining2 unit and propose a novel deep de-raining network, respectively, called Pyramid Fully Residual Unit and Network (PFR-Unit and PFR-Net). The PFR-Unit employs fully residual learning in each level of feature pyramid and the PFR-Net connects PFR-Units by a compact dense architecture. The fully residual learning encourages the feature re-usage in PFR-Unit by performing identity mapping for all available shortcuts. The compact dense connection strengthens the feature propagation between the PFR-Units and ensures the unicity of the learning space for the PFR-Units. Along with negative SSIM loss, the PFR-Net presents a good performance in single image de-raining. Comprehensive experimental results show that the PFR-Net outperforms the state-of-the-art single de-raining methods with a big margin on Rain100H, Rain100L and Rain1200 datasets.

This paper focuses on the recognition problem of dynamical patterns consisting of sampling sequences. Specifically, based on the concept of structural stability, a novel similarity measure for dynamical patterns is first given. Then, a specific realization is provided, which consists of: (1) an approximation scheme for computation of partial derivative information by utilizing the knowledge learned through deterministic learning; (2) a similarity comparison scheme using the recognition errors generated from the discrete-time dynamical estimators; and (3) performance analysis of the recognition scheme with general recognition conditions. Compared with the existing methods, in which misrecognition may occur when the differences of dynamics between adjacent training patterns are very small, the proposed method is more appealing in the sense that, the partial derivatives of dynamics are introduced to complement the similarity measures, such that the recognition performance is much improved. Simulation studies are conducted to verify the proposed method in a relatively large data set.

Video dehazing is of crucial importance to the outdoor surveillance, and its main challenges mainly come from the spatial-temporal coherence and computational efficiency. This paper presents an efficient real-time video dehazing approach via incremental transmission learning and spatial-temporally coherent regularization, while explicitly suppressing the possible visual artifacts. First, we propose to initialize the transmission map frame-by-frame by a boundary constrained open dark channel model. Then, supposing a scene point yields highly correlated transmission values between adjacent frames, we impose a temporally coherent term to maintain the temporal consistency of consecutive transmission values, and simultaneously derive an incremental transmission adjusting term to adapt the abrupt scene depth changes between the adjacent frames. Accordingly, the highly correlated and spatial-temporally coherent transmission maps can be optimized for each video frame dehazing, whereby the flickering artifacts can be well reduced. Finally, we group each haze/haze-free images in pair and further utilize the guided joint bilateral filter to suppress the visual artifacts that possibly amplified during the dehazing process. The experimental results show that the proposed video dehazing approach is able to well preserve the spatial-temporal coherence, runs sufficiently fast, and also performs favorably compared to the state-of-the-art methods.

In this paper, we generalize the usual multiplicative ∊ -indicator I ∊ and additive ∊ -indicator I ∊ + to the forms with L p -norm, i.e. I ∊ p -indicator and I ∊ + p -indicator, and investigate their properties under different p. From the analysis, we find that both I ∊ p -indicator and I ∊ + p -indicator have a good performance on measuring the population convergence and diversity, especially when p is set to be infinity. In order to show their differences more intuitively, we implement a series of comparative experiments by the proposed I ∊ p -indicator and I ∊ + p -indicator based evolutionary algorithm. The experimental results confirm that all these ∊ -indicators have good performances, particulary as p is set to be infinity. We also compare the proposed indicator-based evolutionary algorithm with five existing algorithms. The experimental results demonstrate that our algorithm is effective. In addition, considering that the usual IGD metric is not accurate in measuring the convergence of a population, we adopt the measure based on I ∊ + p -indicator and propose a more effective performance metric, which is denoted as IGD ∊ + .

Metallic pieces are typically machined by continuous tool passes, which usually causes regular patterns in the form of straight edges in the surface of the pieces. An irregular pattern on the surface of the piece implies a decrease of the quality of the machined piece. In this paper, we propose an acquisition system and a machine-vision based method to describe the texture of the inner and outer surfaces of machined pieces with cylindrical holes. In order to capture images of the hole surface, we used a microscope camera connected to a rigid industrial boroscope. Considering the extracted texture descriptors, a significant correlation is shown. Consequently, the feature vector is reduced and then classified by several algorithms using an exhaustive grid search strategy with 10-fold cross validation. Best results are achieved with the Extremely Randomized Trees classifier with a mean test score on the hold out set of 92.98%, what improves previous research and meets the requirements of the field.

Face recognition systems are widely used for target recognition and identity authentication, such as automated teller machines, mobile phones, and entrance guard systems. However, face recognition systems are vulnerable to presentation attacks, such as photo, replay, and 3D mask attacks. In particular, silicone mask attacks pose a greater threat to face recognition systems because high-quality silicone masks do living properties. To promote the development of face anti-spoofing detection algorithms for silicone mask attacks, this paper constructs a Silicone Mask Face Motion Video Dataset (SMFMVD) containing 200 real face videos and 200 silicone mask face videos. These videos include different facial motions collected from 20 subjects. Moreover, inspired by the observation that the silicone mask face’s facial movement is not so natural as the real face, we propose a novel silicone mask face anti-spoofing detection method based on visual saliency and facial motion characteristics. Specifically, we compute the visual saliency map of a given face image by simulating two kinds of eye movement behaviors, namely “gaze” and “saccade”. Then, we propose a saliency-weighted histogram of local binary pattern operator to extract facial texture features in spatial domain and a saliency-guided histogram of oriented optical flow operator to extract facial motion features in temporal domain. Finally, the support vector machine is used to fuse two groups of facial features to distinguish real and spoof faces. Extensive experiments on public and self-built datasets show its superiority over the state-of-the-art methods.

The traditional robust principal component analysis (RPCA) via decomposition into low-rank plus sparse matrices offers a powerful framework for a large variety of applications in computer vision. However, the reconstructed image experiences serious interference by Gaussian noise, resulting in the degradation of image quality during the denoising process. Thus, a novel manifold constrained joint sparse learning (MCJSL) via non-convex regularization approach is proposed in this paper. Morelly, the manifold constraint is adopted to preserve the local geometric structures and the non-convex joint sparsity is introduced to capture the global row-wise sparse structures. To solve MCJSL, an efficient optimization algorithm using the manifold alternating direction method of multipliers (MADMM) is designed with closed-form solutions and it achieves a fast and convergent procedure. Moreover, the convergence is analyzed mathematically and numerically. Comparisons among the proposed MCJSL and some state-of-the-art solvers, on several accessible datasets, are presented to demonstrate its superiority in image denoising and background subtraction. The results indicate the importance to incorporate the manifold learning and non-convex joint sparse regularization into a general RPCA framework.

In this paper, we develop a novel Joint Model Compression (referred to as JMC) method by combining structured pruning and dense knowledge distillation techniques to significantly compress original large language model into a deep compressed shallow network. In particular, a new Direct Importance-aware Structured Pruning (referred as DISP) approach is proposed to structurally prune the redundant structures in the Transformer networks directly based on the corresponding parameter matrices in the model. Besides, a Dense Knowledge Distillation (referred to as DKD) method is developed with a many-to-one layer mapping strategy to leverage more comprehensive layer-wise linguistic knowledge for the distillation. Further, the proposed structured pruning and dense knowledge distillation are integrated together to perform the joint compression, which enables us to achieve a significant compression without sacrificing model accuracy. The extensive experimental results across four NLP tasks on seven datasets demonstrate its effectiveness and superiority to the baselines, while maintaining similar performance to original large model with further remarkable benefits for inference-time speedup and memory efficiency.

The fitness landscape is a theory applied to the evolutionary dynamics of biological evolution to explain the behavior of evolutionary algorithms in the solution of optimization problems. With the continuous advancement of evolutionary algorithm optimization, a fitness landscape can present more abundant feature information, such as the local fitness, fitness distance correlation, and landscape roughness. These landscape features reflect the optimal solution distribution, quantity, and local unimodal topology of the optimization problem from various angles. This paper expresses the adaptability landscape features of typical optimization problems, engages in a quantitative analysis of the fitness distance correlation information, evaluates the difficulty of solving the problem within the search space, and obtains the correlation degree classification result. The search strategy adapts the mixed mutation and the fitness distance correlation for differential evolution. Empirical studies show that, the fitness distance correlation search strategy for the differential evolution algorithm can avoid falling into the local optimum, improve accuracy and convergence, and solve the single-objective optimization problem in a more comprehensive manner.

Illumination normalization is one of the most challenging issues for facial analysis. To be specific, the variation of environmental illumination influences the visual quality of an image and drastically degrades the performance of face recognition, detection, landmark and other related applications. Retinex theory provides an important concept for processing illumination variation, which supposes that a face image can be decomposed into [an invariant reflectance component and a variant illumination component. Enlighten by this theory, in this paper, we put forward a novel deep learning approach which combines self-supervised learning and adversarial training for face illumination normalization (FIN-GAN). The proposed FIN-GAN framework can be implemented by two steps. Firstly, self-supervised learning is employed to decompose the original face image into the illumination component and the illumination-invariant component with Retinex constraint. Then, we employ the conditional generative adversarial network for face image reconstruction. For network optimization, we design the combined loss to ensure visual quality and preserve identity information. Experiments are performed on Extended-YaleB, CAS-PEAL, CMU-PIE and Multi-PIE datasets. Through multiple quantitative criteria, we demonstrate that the proposed FIN-GAN obtains promising performance in face illumination normalization.

The outbreak and rapid spread of coronavirus disease 2019 (COVID-19) has had a huge impact on the lives and safety of people around the world. Chest CT is considered an effective tool for the diagnosis and follow-up of COVID-19. For faster examination, automatic COVID-19 diagnostic techniques using deep learning on CT images have received increasing attention. However, the number and category of existing datasets for COVID-19 diagnosis that can be used for training are limited, and the number of initial COVID-19 samples is much smaller than the normal’s, which leads to the problem of class imbalance. It makes the classification algorithms difficult to learn the discriminative boundaries since the data of some classes are rich while others are scarce. Therefore, training robust deep neural networks with imbalanced data is a fundamental challenging but important task in the diagnosis of COVID-19. In this paper, we create a challenging clinical dataset (named COVID19-Diag) with category diversity and propose a novel imbalanced data classification method using deep supervised learning with a self-adaptive auxiliary loss (DSN-SAAL) for COVID-19 diagnosis. The loss function considers both the effects of data overlap between CT slices and possible noisy labels in clinical datasets on a multi-scale, deep supervised network framework by integrating the effective number of samples and a weighting regularization item. The learning process jointly and automatically optimizes all parameters over the deep supervised network, making our model generally applicable to a wide range of datasets. Extensive experiments are conducted on COVID19-Diag and three public COVID-19 diagnosis datasets. The results show that our DSN-SAAL outperforms the state-of-the-art methods and is effective for the diagnosis of COVID-19 in varying degrees of data imbalance.

Recommender system uses user-item historical interactions to portray user preferences. Due to the problem of data sparseness, auxiliary information is introduced to describe user preferences, such as user/item attribute information. However, some of these methods only consider user(item) attributes when modeling user preferences. Although another part of the methods interact with user attributes and item attributes, this interaction doesn’t consider the potential preference of a certain attribute of the user to a certain attribute of the item. This makes the model unable to capture fine-grained user preferences. In fact, capturing fine-grained user preferences in terms of attributes is very effective in improving the recommendation effect. Therefore, in this paper we propose a recommendation method to capture user fine-grained preferences through attribute information. Specifically, there are two elaborately designed modules, interactive preference module and attribute fine-grained preference module. The former uses user-item historical interactions to model user’s interaction preferences. In order to obtain the user’s fine-grained preferences, the latter interacts with each attribute of the user and all the attributes of the item and uses the attention mechanism to mine the user’s more fine-grained preferences on attributes. Extensive experiments on three publicly available datasets demonstrate the effectiveness of the proposed method.

Multi-modal abstractive summarization for videos is an emerging task, aiming to integrate multi-modal and multi-source inputs (video, audio transcript) into a compressed textual summary. Although recent multi-encoder-decoder models on this task have shown promising performance, they did not explicitly model interactions of multi-source inputs. While some strategies like co-attention are utilized for modeling this interaction, considering ultra-long sequences and additional decoder in this task, the coupling of multi-modal data from multi-encoders and decoder needs complicated structure and additional parameters. In this paper, we propose a concise Decoder-only Multi-modal Transformer (D-MmT) based on the above observations. Specifically, we cut the encoder structure, and introduce an in-out shared multi-modal decoder to make the multi-source and target fully interact and couple in the shared feature space, reducing the model parameter redundancy. Also, we design a concise cascaded cross-modal interaction (CXMI) module in the multi-modal decoder that generates joint fusion representations and spontaneously establishes a fine-grained intra- and inter- association between multi-modalities. In addition, to make full use of the ultra-long sequence information, we introduce a joint in-out loss to make the input transcript also participate in backpropagation to enhance the contextual feature representation. The experimental results on the How2 dataset show that the proposed model outperforms the current state-of-the-art approach with fewer model parameters. Further analysis and visualization show the effectiveness of our proposed framework.

Aspect level sentiment classification aims to extract fine-grained sentiment expressed towards specific aspects from a sentence. The key to this task lies in connecting aspects and their respective sentiment contexts. Existing methods measure the dependency weights between aspects and context words via either the semantic similarity between words captured by attention mechanism or the structural proximity between words in syntactic structures. However, methods in both groups fail to fully exploit explicit syntactic dependency, which we argue should be critical to identify sentiment contexts. In this paper, we propose a novel syntactic-dependency-based attention network (SDATT) to incorporate explicit syntactic dependency for aspect level sentiment classification. SDATT first models the dependency path between each word and the aspect to characterize aspect-oriented syntactic representation of each word. The generated syntactic representations are later fed into the attention layer to help infer the dependency weights for sentiment prediction. Experimental results on five benchmark datasets show the superior performance of the proposed model over state-of-the-art baselines.

Light Field (LF) images have the unique advantage of recording scenes from multiple viewpoints, which provides many applications, such as refocusing and depth estimation. However, low-light conditions can severely influence these applications. In this paper, we propose a two-stage deep learning framework for the LF restoration under low-light imaging. First, there is a multi-to-one (MTO) network, which restores each view separately by utilizing multiple auxiliary views. All the views share the same feature extractor, with an efficient spatial-channel attention mechanism to extract more informative features. A channel-attention feature fusion (CAFF) module is designed to selectively fuse more useful complementary information from the auxiliary views, with a learnable global scalar to adjust the importance of the auxiliary features. Then, the outputs of the MTO network are further enhanced by an (all-to-all) ATA network, which uses spatial and angular residual blocks to process all the views synchronously for fully encoding the spatial-angular information. Extensive experiments have been conducted to demonstrate the superior performance and robustness of our method, i.e., it can restore the luminance, spatial details and angular geometries of the LF images under various light levels effectively.

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features.

In recent years, deep learning techniques have significantly improved the performance of single image super-resolution (SISR). However, this improvement is often achieved at the cost of introducing a large amount of parameters, which limits the real-world applications for SISR. In this paper, we propose a lightweight SISR network called Multi-scale Channel Attention Network for Image Super-Resolution (MCSN). Our contributions are threefold. First of all, the multi-scale feature fusion block (MSFFB) can extract multi-scale features by filters with different receptive fields. Secondly, the channel shuffle attention mechanism (CSAM) encourages the flow of the information across feature channels and enhances the ability of feature selection. Thirdly, the global feature fusion connection (GFFC) can effectively improve feature utilization. Extensive experiments demonstrate that the parameter amount of our method is reduced by 3/4 compared with the current state-of-the-art MSRN method, while both the subjective visual effect and objective quality of the reconstructed high-resolution images are significantly better.

A Reinforcement Learning (RL) agent interacts with the environment to learn a policy with high accumulated rewards through attempts and failures. However, RL suffers from its own trial-and-error learning nature, which results in an unstable learning process. In this paper, we investigate a common phenomenon called rewards drop at the late-stage RL training session, where the rewards trajectory oscillates dramatically. In order to solve such a problem, we propose a novel rewards shaping technique named Variance Aware Rewards Smoothing (VAR). We show that the proposed method reduces the variance of rewards and mitigates the rewards drop problem without changing the formulation of the value function. Furthermore, the theoretical analysis of convergence of VAR is provided, which is derived from the γ -contraction operator and the fixed point attribute of the value function. Finally, the theoretical results are illustrated by extensive results on various benchmarks and advanced algorithms across different random seeds to demonstrate the effectiveness and the compatibility of VAR.

In the paper, data-driven finite-horizon robust formation-containment control scheme is developed based on integral reinforcement learning and zero-sum game for perturbed multi-agent networks with completely unknown nonlinear dynamics. At first, distributed finite-time sliding mode estimators are designed to obtain the desired states of leaders and followers, respectively. Then finite-horizon robust leader formation control and follower containment control are achieved based on proposed model-free integral reinforcement learning algorithms implemented by critic-actor-disturbance structure, in the framework of multi-player zero-sum game where the non-quadratic performance index for each agent considers the influence of saturated inputs and disturbances of local neighbors thoroughly. Furthermore, it is proved that the whole network has bounded L 2 gain robust stability and Nash equilibrium of zero-sum game exists. Simulation results are provided to demonstrate the effectiveness of the proposed scheme.

Differential privacy, a strict privacy notion, prevents the presence or absence of a single data from being identified. This paper proposes a novel average consensus algorithm to achieve differentially private average consensus on multi-agent systems (MASs) with general directed graphs. For MASs under the proposed algorithm, necessary and sufficient conditions are established on achieving average consensus and preserving differential privacy of the states of the agents at any time instant. Finally, numerical examples are provided to illustrate these results.

Both image decomposition and data completion are not only ubiquitous but also challenging tasks in the study of computer vision. In this paper, different from existing approaches, we propose a novel regularization model for image decomposition and data completion, which integrates relative total variation (RTV) with Schatten- 1 / 2 or Schatten- 2 / 3 norm, respectively. RTV is shown to be able to extract the fundamental structure effectively from the complicated texture patterns and largely to avoid the drawback of oil painting artifacts. Schatten quasi-norm is used to capture texture patterns in a completely-separated manner. The proposed model is in essence divided into “RTV+ double nuclear norm” and “RTV+ Frobenius/nuclear hybrid norm”, which can be solved by splitting variables and then by using the alternating direction method of multiplier (ADMM). Convergence of the algorithm is discussed in detail. The proposed approach is applied to several benchmark low-level vision problems: gray-scale image decomposition and reconstruction, text removal, color natural scene image completion, and visual data completion, demonstrating the distinguishable effectiveness of the new model, comparing to the latest developments in literature.

In this paper, a novel intermittent noise-diffusion layer is proposed to synchronize a network system. Different from most of the traditional noise-induced synchronization or noise stabilization (noise exists in nodes of the network), we give an intermittent noise-diffusion layer (the noise exists in the edges of the network) to study the almost sure exponential synchronization. The theoretical results show that the general network system can be synchronized in the almost sure sense as long as the topological structure of the network in the intermittent noise-diffusion layer is an undirected connected graph. Two examples are proposed to illustrate the theory obtained.

To ensure the reliability and security of the data, the large-scale distributed storage system usually adopts the data redundancy mechanism to repair the data on the faulty nodes. Comparing with the replication-based method, the data redundancy mechanism of erasure code can effectively improve the use of storage space, but it may result in the large network overhead when recovering the data. The regenerating code is an improved erasure code, which can reduce the quantity of data transmission compared to that of the erasure code. Adopting the regenerating code to repair the data on a faulty node requires constructing an optimal repair tree to maximum the bandwidth of the bottleneck link, which is an NP-hard problem. To construct the optimal repair tree, a hybrid genetic algorithm is proposed in this paper. In particular, our proposal comprehensively considers the network topology and link bandwidth of storage nodes and designs a problem-specific cross-correlation operator, mutation operator and local search operator. In addition, we provided the mathematical proof of the global convergence with probability one with respect to the proposed hybrid genetic algorithm. Through a series of simulation experiments, the results show that our proposal is able to determine the optimal repair tree, which effectively reduces the delay of faulty node repair in distributed storage systems, and improve the repairing efficiency.

The ever-increasing complexity of robots causes failures of them as a side effect. Successful detection of anomalies in robotic systems is a key issue in order to improve their maintenance and consequently reducing economic costs and downtime. Going one step further in the detection of anomalies in robots, different mechanisms to deal with data irregularities are proposed and validated in present paper in order to increase detection rates. More precisely, strategies to overcome missing values and class imbalance are considered as complementary tools to get better one-class classification results. The effect of such strategies is evaluated through cross-validation when applying a standard supervised learning model, the Support Vector Machine. Experiments are run on an up-to-date and public dataset that contains some examples of different software anomalies that the middleware of the robot under analysis may experience.

K-nearest neighbor (KNN) based weighted multi-class twin support vector machines (KWMTSVM) is a novel multi-class classification method. In this paper, we propose a novel least squares version of KWMTSVM called LS-KWMTSVM by replacing the inequality constraints with equality constraints and minimized the slack variables using squares of 2-norm instead of conventional 1-norm. This simple modification leads to a very fast algorithm with much better results. The modified primal problems in the proposed LS-KWMTSVM solves only two systems of linear equations whereas two quadratic programming problems (QPPs) need to solve in KWMTSVM. The proposed LS-KWMTSVM, same as KWMTSVM, employed the weight matrix in the objective function to exploit the local information of the training samples. To exploit the inter class information, we use weight vectors in the constraints of the proposed LS-KWMTSVM. If any component of vectors is zero then the corresponding constraint is redundant and thus we can avoid it. Elimination of redundant constraints and solving a system of linear equations instead of QPPs makes the proposed LS-KWMTSVM more robust and faster than KWMTSVM. The proposed LS-KWMTSVM, commensurate as the KWMTSVM, all the training data points into a “1-versus-1-versus-rest” structure, and thus our LS-KWMTSVM generate ternary output {-1,0,+1} which helps to deal with imbalance datasets. Numerical experiments on several UCI and KEEL imbalance datasets(with high imbalance ratio) clearly indicate that the proposed LS-KWMTSVM has better classification accuracy compared with other baseline methods but with remarkably less computational time. This work was supported by Science and Engineering Research Board (SERB) under Early Career Research Award Grant No. ECR/2017/000053 and Ramanujan fellowship Grant No. SB/S2/ RJN-001/2016. This work was also supported by Council of Scientific & Industrial Research (CSIR), New Delhi, INDIA under Extra Mural Research (EMR) Scheme Grant No. 22(0751)/17/ EMR-II.

Interpersonal relation plays an essential role to gain understandings on how people interact with each other. In computer vision, interpersonal relations provide vital information to interpret people’s behaviors. However, the existing research has either omitted the interaction information between subjects or the structural information in the images. In this paper, we propose a new architecture to reason interpersonal relations based on higher-order graph networks and multi-scale features. First, we extract features of the whole images, the facial features, and the union region of face pairs. Apart from the pixel-wise features, we also consider the positional features of face-to-face pairs and the spatial scene cues. Higher-order Graph Neural Networks (GNNs) were employed to map out the interpersonal relations based on the feature extracted. Experimental results show that the proposed Higher-order Graph Neural Networks with multi-scale features can effectively recognize the social relations in images with over 5% improvement in absolute balanced accuracy compared with the state-of-the-art work.

The use of hierarchical networks in motor pattern generation in hexapod robots and in insect movement provides ways to understand and control both systems. In this paper we consider the Central Pattern Generator (CPG) of insect movement consisting of six coupled neurons developed by Ghigliazza and Holmes (2004) that produces the global leg coordination pattern. We provide a detailed study of the possible gaits generated by the CPG through different numerical techniques recently developed for the study of small networks, which allow us to consider the complete model without any simplification. We combine the analysis of isolated neurons (using several three dimensional parameter spaces) to give a roadmap of the dynamics of the neurons involved, lateral phase lag plots to show the convergence and transitions towards particular patterns and a quasi-Monte Carlo sweeping method to describe different pattern routes in the parametric phase space. In all of our studies we have found the same final result: most of the observed patterns follow routes that lead to the stable tripod gait. We obtain several routes made of symmetrical patterns, but despite considering mainly a symmetric leg CPG we detect non-symmetrical patterns that provide (minor) routes present in the model. The bifurcations of the main pattern routes detected in the parameter space are studied in detail using continuation techniques. This study reveals the bifurcations that create and destroy the different routes and how for large values of some parameters only the tripod gait is present. Based on the symmetric case, a preliminary study of asymmetric configurations is done revealing the robustness of the already located patterns and routes. Due to the relevance of the tripod gait, and since more parameters are included in this study, we introduce an algorithm to locate the limit of the tripod gait in the parameter space, which shows that there is a large three parametric region where the tripod pattern is ubiquitous and highly dominant in rapidly moving insect regimes.

This paper proposes a class of neural dynamics models based on a neural dynamic approach to realize attitude tracking control of a flapping wing micro aerial vehicle (FWMAV), different from the existing conventional controllers. Firstly, from the perspective of FWMAV attitude tracking control, an attitude tracking model is presented. Further theoretical analyses reveal that there exist theoretical errors on such a controller when solving the attitude tracking control problem. To remedy its deficiency, we propose a class of adaptive neural dynamics models with activation functions. Different from the existing controller, the adaptive neural dynamics model adopts different activation functions to accelerate the convergence speed of attitude tracking errors and makes errors converge to zero in a short time. Finally, simulation results and theoretical analyses are provided, which verify that the proposed controllers possess the superiority and effectiveness.

The graph model is a reliable propagation mechanism in saliency detection, saliency value propagation diffusion results between any two nodes are determined merely by defining an effect affinity matrix. Most existing methods generally calculamatrix using mean values of single or multiple feature vectors, not fully exploit the diversity and consistency of multi-view features, may produce poor foreground uniformity and completeness in the complex scene. Multi-views should share an affinity matrix as well as complement each other. In this paper, we propose a graph-based saliency detection with a learning joint affinity matrix. First, we capture multiple appearance features from the image and generate a learning joint affinity matrix based on low-rank representation. Then, for computing an effect affinity matrix, we linearly integrate the traditional affinity matrix and learning joint affinity matrix, helping to construct an affinity graph for diffusion-based compactness. Finally, to effectively optimize the initial saliency map, we diffuse the learning joint affinity matrix and traditional impact factor matrix via cross-view diffusion processing, which begets an approving advantage for single-layer cellular automata. Results on three benchmark datasets demonstrate that our proposed method shows the best performance against nine state-of-the-art models.

Recently, many scenarios, such as affective disorders treatment, have sparked rising needs for establishment of personalized emotion recognition (PER) models. Unfortunately, the data sparsity issue violates the basic i.i.d. assumption of supervised learning (i.e., training data and test data are independently and identically distributed). In this paper, we present a semi-supervised joint domain adaption (SSJDA) solution, aiming to inject the hidden domain knowledge from ample labeled data of multiple source individuals into the target subject’s customized model. Specifically, we put forward a novel Progressive Low-Rank Subspace Alignment (PLRSA) approach, which unifies a semi-supervised instance-transfer paradigm and an unsupervised mapping-transfer learning paradigm in a single optimization framework. We leverage the boosting-based TrAdaBoost algorithm and the Transfer Component Analysis (TCA) algorithm for the implementation of instance reweighting and feature matching, respectively. Then we introduce the l 2 , 1 - norm to pass feedback and make the joint learning feasible. The central idea is to progressively minimize the cross-domain distribution discrepancies to finally construct the optimal domain-invariant features. We systematically compare the PLRSA method with five state-of-the-art techniques using two public EEG datasets (DEAP and SEED). Both many-to-one and one-to-one evaluations are performed. The experimental results have confirmed the efficacy of the proposed method.

Current annotation for plant disease images depends on manual sorting and handcrafted features by agricultural experts, which is time-consuming and labour-intensive. In this paper, we propose a self-supervised clustering framework for grouping plant disease images based on the vulnerability of Kernel K-means. The main idea is to establish a cross iterative under-clustering algorithm based on Kernel K-means to produce the pseudo-labeled training set and a chaotic cluster to be further classified by a deep learning module. In order to verify the effectiveness of our proposed framework, we conduct extensive experiments on three different plant disease datatsets with five plants and 17 plant diseases. The experimental results show the high superiority of our method to do image-based plant disease classification over balanced and unbalanced datasets by comparing with five state-of-the-art existing works in terms of different metrics.

This paper proposes a novel prescribed finite-time output feedback control scheme for quadrotor trajectory tracking control. The proposed control scheme considers the quadrotor modeling containing uncertain nonlinearities and strong coupling. The neural observer is set up to estimate the unknown state variables. The main difficulty caused by the time-vary coefficient matrix to design a state observer is overcame by the convex combination technique. To get a prescribed finite-time tracking performance, Barrier Lyapunov Function (BLF) approach is presented for control design. The constructed controllers guarantee that the tracking errors meet the prescribed accuracy in a pre-specified finite settling time. Finally, a simulation example verifies the effectiveness of the control scheme.

This paper designs a novel online event-based near-optimal control scheme for multi-player nonzero-sum (NZS) games with partially unknown system dynamics. With the introduction of event-triggered mechanism (ETM), the repetitive computing burdens and communication loads of the signals are largely alleviated. Under the event-triggered mechanism, the framework of identifier-critic is structured to solve the coupled Hamilton–Jacobi equations (CHJEs) for NZS games. A feedforward neural network (FNN) is employed to construct an identifier to learn the unknown dynamics of the nonlinear system. The critic network for each player utilizes a modified tuning law, which is composed of three terms, to seek for the approximated optimal control schemes. Besides the conventional term derived from the gradient descent method, the second term derived from the experience replay (ER) technique utilizes the historical state data to update the critic network weight vector so as to remove the persistence of excitation (PE) condition. In addition, a novel term is added to stabilize the closed-loop systems. Owing to the stabilizing term, there is no need for the initial stabilizing control. In theory, the uniform ultimate boundedness (UUB) properties of the system states and the critic network weight errors are proved by utilizing Lyapunov theorem. Furthermore, the minimal intersample time is proved to be lower bounded with a positive constant which means that the Zeno behavior is excluded during the learning phases. Finally, we show the validity of the designed algorithm via simulating two examples.

Nonlinear and nonconvex optimization problem (NNOP) is a challenging problem in control theory and applications. In this paper, a novel mixture varying-gain dynamic learning network (MVG-DLN) is proposed to solve NNOP with inequality constraints. To do so, first, this NNOP is transformed into some equations through Karush–Kuhn–Tucker (KKT) conditions and projection theorem, and the neuro-dynamics function can be obtained. Second, the time varying convergence parameter is utilized to obtain a faster convergence speed. Third, an integral term is used to strengthen the robustness. Theoretical analysis proves that the proposed MVG-DLN has global convergence and good robustness. Three numerical simulation comparisons between FT-FP-CDNN and MVG-DLN substantiate the faster convergence performance and greater robustness of the MVG-DLN in solving the nonlinear and nonconvex optimization problems.

In this paper, an ideal current source forgetting memristor model is proposed. Based on the model, three kinds of synapses with long- and short-term memory are designed: the series forgetting memristor synapse, the forgetting memristor bridge synapse written independently and the forgetting memristor bridge synapse written in batches. Combined with the forgetting property, the long- and short-term weight of the forgetting synapse can be controlled by the long- and short-term resistance of memristors. Compared the three forgetting synapses, the series forgetting memristor synapse has the lowest requirement for memristors, the forgetting memristor bridge synapse written independently is the most flexible, and the forgetting memristor bridge synapse written in batches is the most convenient. Compared with traditional synapses, forgetting synapses with long- and short-term memory have multi-weight storage. When forgetting synapses are applied to associative memory, it can be found more patterns are stored in the neural network and different patterns are recalled at different time due to the forgetting effect.

Multi-view clustering, which utilizes the ample information provided by multiple sources to obtain better performance, has attracted much attention. However, existing clustering algorithms either have no ability to offer confidence for each assignment or suffer from the disturbance of outliers. To address these problems, in this paper, we propose a novel multi-view fuzzy clustering method via transferring softmin to fuzzy models. To obtain fuzzy assignments, we utilize the softmin with temperature and further develop an efficient algorithm to solve the non-convex problem approximately. We also show another explanation for the algorithm from the aspect of the prior distribution of various views. Besides, we design a scalable robust loss function, which interpolates between l2-norm and the squared l2-norm, to enhance the robustness to outliers. Extensive experiments show the superiority of our model under different clustering metrics.

The localization problem of unknown nodes in wireless sensor networks (WSN) has drawn increasing scholarly attention along together the popularity of meta-heuristic algorithms. To overcome the shortcomings of low accuracy that traditional least square method (LSM) inevitably produces, this paper introduces an improved adaptive genetic algorithm (IAGA) to handle the aforementioned problem and uses a modified evaluation function to reduce the error of distance measurement in a topological structure. The experimental results prove that the IAGA algorithm based on DV-Hop has superior performance in comparison of original DV-Hop and other meta-heuristic algorithms. The conclusion can be drawn that meta-heuristic algorithms have an better superiority over DV-Hop in locating nodes in WSN and the IAGA is more promising than other meta-heuristic algorithms.

The recent years have witnessed significant advances in deep video action recognition. However, the performances of deep learning-based video action recognition methods are limited in the case of relatively small samples or no labeled samples. Therefore, trying to use unlabeled video data to generate clustering label is essential for small sample learning and zero sample learning. In this paper, we propose a novel deep video action clustering network, which aims to learn the similarity relationship among the unlabeled video samples, and generate the clustering label for each video sample. Specifically, the proposed method simultaneously learns the spatio-temporal features and subspace representations under a jointly optimized framework. It consists of a 3D U-Net self-representation generator, a video-clip reconstruction discriminator, and a confidence-based feedback mechanism. The 3D U-Net self-representation generator learns the spatio-temporal features of the video clips and produces subspace representation matrix. Then, the similarity graph is constructed based on this subspace representation matrix, and the clustering result is obtained. In the learning procedure, a confidence-based feedback mechanism is designed to feed the high-confidence labels of partial samples back to further guide the subspace structure learning, so that the optimal result can be obtained. During training, the video-clip reconstruction discriminator is introduced to evaluate the reconstructed video clips, which is beneficial for capturing the discriminative spatio-temporal features. Experimental results on a video benchmark dataset demonstrate the effectiveness of the proposed method.

Multi-installment scheduling (MIS) has made great strides in minimizing the makespan of large-scale workloads on distributed systems. By the makespan is meant the total time it takes for workload distribution, computation and result retrieval. However, existing studies have hardly taken result retrieval time into consideration due to an idealistic assumption that the amount of result generated after workload computation is so small that the retrieval time could be neglected. This unrealistic assumption may have a seriously negative effect on task-scheduling strategies especially for big-data-related applications nowadays. In view of this, this paper studies the MIS problem with result retrieval on heterogeneous distributed systems. We propose a new MIS model referred to as MIS-RR and solve three crucial issues: (1) we obtain, for the first time, a closed-form solution to an optimal load partition by strict mathematical derivation for this problem; (2) we get an optimal number of installments by designing a heuristic algorithm; (3) we obtain an optimal scheduling sequence of servers involved in computation by proposing an evolutionary algorithm. Experimental results clearly show that our proposed strategy can achieve the shortest makespan as well as the highest average CPU utilization and system utilization compared to existing scheduling strategies.

Interpretability is a significant aspect of the distributed word representation learning model. Although the most advanced pretrained models have achieved the best results till date, the interpretability of a pretrained model is difficult to explain clearly. For this reason, based on the interpretability of distributed word embeddings, this paper presents a method of learning word representation using joint context. At present, the existing distributed word representation models for learning word representations usually focus on either neighbor or syntactic context. We argue that it is necessary to simultaneously model both contexts. In particular, the point mutual information obtained by combining the two types of contexts can efficiently express the correlation between the words. We propose two alternative distribution models for learning word representations by employing the neighbor and syntactic contexts via a simple and effective joint learning framework. Furthermore, the proposed models are trained on a public corpus, and the learned representations are evaluated in word analogy, word similarity, and sentence classification tasks. The experimental results demonstrate the potential of the proposed method.

Deep convolutional neural networks have shown great potential in image recognition tasks. However, the fact that the mechanism of deep learning is difficult to explain hinders its development. It involves a large amount of parameter learning, which results in high computational complexity. Moreover, deep convolutional neural networks are often limited by overfitting in regimes in which the number of training samples is limited. Conversely, kernel learning methods have a clear mathematical theory, fewer parameters, and can contend with small sample sizes; however, they are not able to handle high-dimensional data, e.g., images. It is important to achieve a performance and complexity trade-off in complicated tasks. In this paper, we propose a novel scalable deep convolutional random kernel learning in Gaussian process architecture called SDCRKL-GP, which is characterized by excellent performance and low complexity. First, we successfully incorporated the deep convolutional architecture into kernel learning by implementing the random Fourier feature transform for Gaussian processes, which can effectively capture hierarchical and local image-level features. This approach enabled the kernel method to effectively handle image processing problems. Second, we optimized the parameters of deep convolutional filters and Gaussian kernels by stochastic variational inference. Then, we derived the lower variational bound of the marginal likelihood. Finally, we explored the model architecture design space selection method to determine the appropriate network architecture for different datasets. The design space consists of the number of layers, the channels per layer, and so on. Different design space selections improved the scalability of the SDCRKL-GP architecture. We evaluated SDCRKL-GP on the MNIST, FMNIST, CIFAR10, and CALTECH4 benchmark datasets. Taking MNIST as an example, the error rate of classification is 0.60%, and the number of parameters, number of computations and memory access cost of the architecture are 19.088k, 0.984M, and 1.057M, respectively. The experimental results verified that the proposed SDCRKL-GP method outperforms several state-of-the-art algorithms in both accuracy and speed in image recognition tasks. The code is available at https://github.com/w-tingting/deep-rff-pytorch.

Today, artificial intelligence is everywhere in people’s daily lives. Visual tracking, which is used to identify and continuously track specific targets, is an important research domain in the study of artificial intelligence. However, current visual tracking methods are not accurate enough for object tracking with background clutter, which can easily lead to tracking failures. Therefore, in this paper, in order to solve the problem of tracking failure in clutter background, we propose a template update mechanism to improve the accuracy of visual tracking. First, an original template is saved when the background clutter is detected. During background clutter, we use both the original template and the current template at the location estimated by the optical flow and choose better one. Next, the original template is reused after the background clutter is ended. Finally, the proposed mechanism is used both in the KCF and BACF algorithm to verify the effectiveness of the mechanism. With experiments on the OTB2015 dataset, results show that the proposed mechanism has improved accuracy and success rate of the two baseline algorithms. Meanwhile, in state-of-the-art algorithms, the algorithm using the proposed mechanism also has excellent tracking performance. In addition, this method also has strong tracking robustness and adaptation capability to sequential learning for video data.

Magnetic Resonance Imaging (MRI) has been widely used in clinical application and pathology research to help doctors provide better diagnoses. However, accurate diagnosis by MRI remains a great challenge, as images obtained via current MRI techniques usually have low resolutions. Improving MRI image quality and resolution has thus become a critically important task. This paper presents an innovative Coupled-Projection Residual Network (CPRN) for MRI super-resolution. CPRN consists of two complementary sub-networks: a shallow network and a deep one, which maintain content consistency while learning high frequency differences between low-resolution and high-resolution images. The shallow sub-network employs coupled-projection to better retain the MR image details, where a novel feedback mechanism is introduced to guide the reconstruction of high-resolution images. The deep sub-network learns from the residuals of the high-frequency image information, where multiple residual blocks are cascaded to magnify the MR images at the last network layer. Finally, the features from the shallow and deep sub-networks are fused for the reconstruction of high-resolution MR images. For effective feature fusion between the deep and shallow sub-networks, a step-wise connection (CPRN_S) is designed, inspired by the human cognitive process (from simple to complex). Experiments over three public MRI datasets show that our proposed CPRN achieves superior MRI super-resolution performance compared with the state-of-the-art.

In multi-objective optimization, the Pareto-optimal solutions must be widely and uniformly distributed on the Pareto-optimal front. When this condition is satisfied, decision makers can choose satisfactory solutions. In this paper, gaps in the evolving nondominated front are filled by a multi-objective evolutionary algorithm using a uniformly evolving scheme (MOEA/UE), thus generating a uniform and wide Pareto-optimal front. The MOEA/UE first calculates the distance between the points on the nondominated front and identifies those with relatively wide gaps. At the widely spaced points, the objective and constraint functions are approximated by linear functions expressed as Taylor series in auxiliary linear programs. Finally, the MOEA/UE solves these auxiliary linear programs and retains the optimal or intermediate solutions as the potentially superior individuals that will be selected in the next generation of the population. Convergence toward potentially superior solutions is accelerated by a mutation operator (a basic differential evolution operator). In constrained multi-objective optimization problems, the MOEA/UE implements a new constraint-handling strategy that pulls infeasible solutions into or nearby the feasible domain. Experimental results illustrate that MOEA/UE obtains wider and more uniform Pareto-optimal fronts and better Pareto-optimal solutions.

Moire patterns, mainly due to the aliasing between the grids of display devices and camera sensors, often occur in recaptured screen images (videos). The spatial variety in densities and scales makes it difficult to remove moire patterns with hand-crafted priors. Prevalent supervised-learning based methods require huge amount of paired images, which are difficult to capture and align. This paper proposes an unsupervised Generative Adversarial Network for Moire Removal (MR-GAN), which is the first attempt at unsupervised learning based moire removal. The recaptured images are corrected against vignetting artifacts to make MR-GAN focus on correcting brightness and contrast distortions while removing moire patterns. We propose two complementary discriminator groups to effectively distinguish moire patterns and image features at both large scales and small scales. A group of self-supervised loss functions, including cycle consistent loss, identity loss, cosine similarity loss, and content leakage loss, are designed to train effective generators. Experimental results demonstrate that our method outperforms state-of-the-art demoireing methods on a large set of test images. Our code and dataset are released in the link: https://github.com/JerryLeolfl/pytorch-MRGAN-master .

The dual hesitant fuzzy set(DHFS) is a useful tool to deal with situations in which people are hesitant about providing their satisfaction degree and dissatisfaction degree. In this paper, we introduce the concepts of weighted dual hesitant fuzzy set(WDHFS) and weighted dual hesitant fuzzy element(WDHFE). Furthermore, we introduce some basic operations such as union, intersection, complement, multiplication and power operation of weighted dual hesitant fuzzy elements, investigate their operation properties, propose the score function and the accuracy function of WDHFE to compare two weighted dual hesitant fuzzy elements, and present two kinds of aggregation operators such as WDHFWA operator and WDHFWG operator to fuse weighted dual hesitant fuzzy information. Besides, we introduce the concept of hesitance degree of WDHFE, and propose a distance measure between weighted dual hesitant fuzzy elements based on the feature vector of WDHFE, which satisfies the four properties including triangle inequality. In addition, we develop an approach to group decision making based on the weighted dual hesitant fuzzy environment. Finally, two numerical examples are used to illustrate the effectiveness and practicality of our proposed approach.

Alzheimer’s disease (AD), characterized by cognitive impairment, mainly affects middle-aged and elderly people. As the aging process of the world continues to intensify, AD harms people’s life, economy and society more and more seriously. Therefore, it has become an urgent problem to study the pathogenesis of AD and seek treatment on this basis. Hybrid synapse, autapse and spatial correlated noise in diverse neural activities have been investigated separately, however, theoretically understanding combination of them still has not been fully studied. Here in this paper, a neural network with multiple associative memory abilities is established from the perspective of the degeneration of associative memory ability in AD patients under the conditions of hybrid synapse, autapse and spatial correlated noise. In order to explore the pathogenesis, a synaptic loss and synaptic compensation model are established to analyze the associative memory ability of AD in different degrees of disease. The simulation results demonstrate the effectiveness of the proposed models and pave a way in the study of dynamic mechanism with higher bio-interpretability in neural networks.

Relation extraction has been an active research interest in the field of Natural Language Processing (NLP). The past works primarily focused on a corpus of formal text which is inherently non-dialogic. Recently, the dialogue-based relation extraction task, which detects relations among speaker-aware entities scattering in dialogues, has been gradually arousing people’s attention. Some sequence-based neural methods have been carried out to obtain the relevant information. However, identifying cross-sentence relations remains unsolved, especially in the context of a specific-domain dialogue system. In this paper, we propose a Relational Attention Enhanced Graph Convolutional Network (RAEGCN), which constructs the whole dialogue as a semantic interactive graph by emphasizing the speaker-related information and leveraging various inter-sentence dependencies. A dense connectivity mechanism is also introduced to empower the multi-hop relational reasoning across sentences, which can capture both local and non-local features simultaneously. Experiments show the significant superiority and robustness of our model on a real-world dataset DialogRE, as compared with previous approaches.

The most existing deep neural networks (DNN)-based methods for fault diagnosis only focus on prediction accuracy without considering the limitation of labeled sample size. In practical applications of DNN-based methods, it is time-consuming and costly to collect massive labeled samples. In this paper a task named few-shot fault diagnosis is defined as training model given small labeled samples in source domain and testing given small samples in target domain. We develop a novel intelligent fault diagnosis model for few-shot fault diagnosis which is using similarities of sample pairs to classify samples, rather than end-to-end classification. The proposed model contains modules of feature learning and metric learning. The module of feature learning has twin neural networks aiming to extract features from the sample pair. The module of metric learning is to predict similarity of the sample pair. The similarities of sample pairs combined the test sample with each labelled sample are utilized to complete the classification task. Label smoothing is utilized to further improve performance of classification. The performance of the proposed model is verified by two fault diagnosis cases which are bearing fault diagnosis cross different working conditions and cross bearing locations. The comparison studies with other models demonstrate the superiority of the proposed model.

Taking advantage of labeled auxiliary training data whose distribution is similar to the distribution of the gallery, single sample face recognition (SSFR) has achieved encouraging performance. However, in many real-world applications, it is difficult to collect such an auxiliary training dataset, while it may be easier to collect an unlabeled target training dataset whose distribution is similar to the distribution of the gallery and a labeled source training dataset whose distribution may be different to the distribution of the gallery. How can these three datasets be effectively leveraged to handle SSFR? To address this issue, this paper proposes a new method of G allery- S ensitive Single Sample Face Recognition based on D omain A daptation (GS-DA). First, GS-DA employs the method of TSD (targetize the source domain) to construct a common subspace and a targetized source domain. Secondly, it projects each gallery image into the common subspace and obtains the sparse representation of each gallery image in the common subspace. Thirdly, it reconstructs each gallery image from the targetized source domain to estimate the within-class scatter matrix and the between-class scatter matrix of the gallery. Lastly, it learns a discriminant model by maximizing the sum of the traces of the between-class scatter matrix of the gallery and the between-class scatter matrix of the targetized source domain as well as minimizing the sum of the traces of the total scatter matrix of the gallery and the total scatter matrix of the target training data. The experimental results on five datasets illustrate the superiority of GS-DA in leveraging these three datasets for SSFR.