Search

As a special type of neurodynamic methodology dedicated to finding zeros of equations, zeroing neurodynamics has shown a powerful ability in solving challenging online time-varying problems. Multi-linear systems, on the other hand, are a type of tensor equations with a wide range of applications. In this paper, a zeroing neurodynamics approach for solving multi-linear systems with M -tensors is proposed along with three specific accelerated finite-time convergent zeroing neurodynamics models, which breaks the limitations of a recently presented neural network approach termed continuous time neural network. Activation functions available in constructing zeroing neurodynamics models for solving multi-linear systems with M -tensors are also investigated in the rest of the paper. Theoretical analyses prove that the proposed zeroing neurodynamics approach is stable in the sense of Lyapunov stability theory and the proposed models converge to the theoretical solution in finite time. Finally, computer simulations are provided to substantiate the efficacy and superiority of the proposed zeroing neurodynamics models for solving multi-linear systems with M -tensors.

The complex and varied urban road conditions have always been a difficult and a pivotal component in the study of driverless technology especially at intersections. In China, it is necessary for driverless cars to understand the gestures of traffic police. To identify the traffic police gesture at the intersection, the key step is to detect the traffic police at the intersection. At present, the research on traffic police detection is still in its infancy, there exists common problems such as slow detection speed and other real time problems in this method, and there is not a standardized traffic police data set ether. For the real time problems, this paper introduces the attention mechanism, and proposes a new real-time detection method of traffic police based on attention mechanism. The method proposed in this paper has strong robustness and can quickly complete the target detection task. For the data set problem, this paper analyzes and discloses similar data sets published through research. In the meanwhile, this paper collects 24,530 video data on the actual road, and it extracts and saves 12,000 pictures containing traffic police at frame rate of 30FPS as traffic police detection training and validated data set.

In this paper, adaptive neural tracking control problem is considered for non-strict-feedback high-order nonlinear systems with quantized input signal. Compared with the logarithmic quantizer, the quantizer introduced in this paper can avoid chattering problem. The dynamic surface control (DSC) technique is introduced to solve the problem of ‘explosion of complexity’, which is appeared in the classic adaptive backstepping control of high-order nonlinear systems. The structural properties of radial basis function neural networks (RBF NNs) are used to simplify the design difficulty from the functions of whole state variables. According to the classic adaptive backstepping technique and neural network algorithm, an output tracking controller is designed, which can guarantee that all the signals of the closed-loop system are semiglobally uniformly bounded and the output of the system can track the reference signal. Finally, a numerical example is presented to verify the effectiveness of the proposed method.

Person re-identification is to recognize a pedestrian who has been observed at different places in monitoring system, which is done by computer automatically. Important practical significance can be found in this paper on improving the intelligence of monitoring system. There are two challenging problems needed to be solved for person re-identification, which is feature representation and matching. Designing a suitable feature representation method is both challenging and crucial. Ideally, the extracted features should be robust enough to cope with the various situations of the monitoring environment, such as posture difference, illumination changing, and shooting angle difference, etc. There are a lot of challenges for feature matching too, such as small sample size, inter-class confusion, and intra-class variation, etc. In order to meet these challenges, a new person re-identification scheme was put forward in this paper. The following three aspects would be used to describe it. First, an improved BOF feature extraction algorithm based on SURF was put forwarded, SURF algorithm extracts the preliminary feature descriptor and generates visual dictionary, so the influence factors such as illumination and scale invariants can be deal with. And then, covariance descriptor was adopted by the algorithm. It has high robustness, and matching accuracy can be improved too for the case of small sample. Finally, an effective classifier was designed by LIBSVM based on improved BOF algorithm, so the efficiency of the person re-identification algorithm can be improved. The proposed method was compared with the current mainstream algorithm through experiment, and it can be found through the experimental results that it is effective to solve the difficult problems for person re-identification.

Many methods achieve the visual object tracking task with deep learning technologies. As the deep features of different levels contain various semantic information and functions, this paper presents a multi-hierarchy feature aggregation approach to tackle the specific issues in the tracking task, which consists of two aspects. On one hand, this paper integrates the features captured by the offline and online classifiers at the score level, which constructs complementary roles of these classifiers to enhance the stability of classification. Besides, the proposed offline classifier is continuously optimized with different levels of features to reinforce classification constraints. On the other hand, we design a butterfly attention module to promote the capacity of multi-hierarchy feature aggregation in the regression network, which aims to fuse and strengthen the multi-scale features by attending to their spatial information. It can capture more spatial contexts by utilizing the self-attention mechanism during the fusion procedure, and preserve the hierarchy of the features during the strengthening process. Extensive experiments on four public datasets, i.e., VOT2018, OTB100, NFS and LaSOT datasets, demonstrate the effectiveness of the proposed methods.

Context and rationale Intrusion Detection, the ability to detect malware and other attacks, is a crucial aspect to ensure cybersecurity. So is the ability to identify this myriad of attacks. Objective Artificial Neural Networks (as well as other machine learning bio-inspired approaches) are an established and proven method of accurate classification. ANNs are extremely versatile – a wide range of setups can achieve significantly different classification results. The main objective and contribution of this paper is the evaluation of the way the hyperparameters can influence the final classification result. Method and results In this paper, a wide range of ANN setups is put to comparison. We have performed our experiments on two benchmark datasets, namely NSL-KDD and CICIDS2017. Conclusions The most effective arrangement achieves the multi-class classification accuracy of 99.909% on an established benchmark dataset.

Single-object tracking is regarded as a challenging task in computer vision, especially in complex spatio-temporal contexts. The changes in the environment and object deformation make it difficult to track. In the last 10 years, the application of correlation filters and deep learning enhances the performance of trackers to a large extent. This paper summarizes single-object tracking algorithms based on correlation filters and deep learning. Firstly, we explain the definition of single-object tracking and analyze the components of general object tracking algorithms. Secondly, the single-object tracking algorithms proposed in the past decade are summarized according to different categories. Finally, this paper summarizes the achievements and problems of existing algorithms by analyzing experimental results and discusses the development trends.

In the process of Human-robot skill transfer, we require the robot to reproduce the trajectory of teacher and expect that the robot can generalize the learned trajectory. For the trajectory after generalization, we expect that the robot arm can accurately track. However, because the model of the robot can not be accurately obtained, some researchers have proposed using a neural network to approximate the unknown term. The parameters of the traditional RBF neural network are usually selected through the empirical and trial-and-error method, which maybe biased and inefficient. In addition, due to the end-effector of the mechanical arm trajectory will be constantly changing according to the needs of the task, when the neural network of compact set cannot contain the whole input vector, the neural network cannot achieve the ideal approximation effect. In this paper, the broad neural network is used to approximate the unknown terms of the robot. This method can reuse the motion controller that has been learned and complete other motions in the robot operating space without relearning its weight parameters. In this paper, the effectiveness of the proposed method is proved by the ultrasound scanning task.

The problem of pursuing the data items of a large database whose distances to a query item are the least is known as Similarity Search (Nearest Neighbor Search) problem. There exist various algorithms to address this problem. Some of the well known algorithms are i) exact algorithms ii) approximation algorithms and iii) randomized algorithms. This paper has made study only on exact and approximation algorithms because randomized algorithm produces approximate results with some probability. Recently, there are several approximation algorithms are proposed by the researchers because this type of algorithms minimizes the problem of Curse of Dimensionality. This paper mainly has two major sections. In first section, various methods under exact and approximation algorithms are discussed with regard to storage, preprocessing and query time. In the second section, efficient algorithms for similarity search suitable for certain physiological characteristics based biometric systems are considered. Biometric system has five main steps viz acquisition of Image, pre-processing, extraction of features, matching and making final decision. In this paper, indexing algorithms for similarity search suitable for iris trait based on different features are discussed in detail. Since the nature of features are distinct and different in biometric traits, there does not exist a universal (one unique) solution which can apply to all traits of biometric systems. Various performance measures like Penetration Rate and Hit Rate are used to determine the correct recognition rate with top best match (rank-1 accuracy).

This paper proposes a hybrid architecture based on neural networks, fuzzy systems, and n-uninorms for solving pattern classification problems, termed as ENFS-Uni0 (short for evolving neuro-fuzzy system based on uni-nullneurons). The model can produce knowledge in an on-line (single-pass) and evolving learning context in a particular form of neuro-fuzzy rules representing the dependencies among input features through IF-THEN type relations. The rules antecedents are thereby realized through uni-nullneurons, which are constructed from n-uninorms, leading to the possibility to express both, AND- and OR-connections (and a mixture of these) among the single antecedent parts of a rule (and thus achieving an advanced interpretability aspect of the rules). The neurons’ evolution is done through an extended version of an autonomous data partition method (ADPA). On-line interpretation of the timely evolution of rules is addressed by (i) a concept for tracking the degree of changes of the rules over data stream samples, which may indicate experts/operators how much dynamics is in the process and may be used as a structural active learning component to request operator’s feedback in the case of significant changes and (ii) a concept for updating feature weights incrementally. These weights express the (possibly changing) impact degrees of features on the classification problem: features with low weights can be seen as unimportant and masked out when showing rules to an expert ( → rule length reduction). The rules’ consequents are represented by certainty vectors and are recursively updated by an indicator-based recursive weighted least squares (I-RWLS) approach (one RWLS estimator per class) where the weights are given through the neuron activation levels in order to gain stable local learning. The model proposed in this paper was successfully compared to related hybrid and evolving approaches in the literature for classifying binary and multi-class patterns. The results obtained by the proposed model show an outperformance of the related works in terms of higher accuracy trend lines over time, while offering a high degree of interpretability through coherent neuro-fuzzy rules to solve the classification problems.

Subtle facial expression recognition (SFER) aims to classify facial expressions with very low intensity into corresponding human emotions. Subtle facial expression can be regarded as a special kind of facial expression, whose facial muscle movements are more difficult to capture. In the last decade, various methods have been developed for common facial expression recognition (FER). However, most of them failed to automatically find the most discriminative parts of facial expression and the correlation of muscle movements when human makes facial expression, which makes them unsuitable for SFER. To better solve SFER problem, an attention mechanism based model focusing on salient local regions and their correlations is proposed in this paper. The proposed method: 1) utilizes multiple attention blocks to attend to distinct discriminative regions and extract corresponding local features automatically, 2) a correlation attention module is integrated in the model to extract global correlation feature over the salient regions, and finally 3) fuses the correlation feature and local features in an efficient way for the final facial expression classification. By this way, the useful but subtle local information can be utilized in more detail, and the correlation of different local regions is also extracted. Extensive experiment on the LSEMSW and CK+ datasets shows that the method proposed in this paper achieves superior results, which demonstrates its effectiveness.

The purpose of this paper is to give an overview of various well known medical image registration techniques with a special focus on registration between anatomical and functional medical images. Examples of anatomical medical images (AMI) are Computer Tomography (CT), Magnetic Resonance Images (MRI), X-ray radiographs and ultrasound etc. whereas Positron Emission Tomography (PET), Single Photon Emission Tomography (SPECT) and fMRI are examples of functional medical images (FMI). Irrespective of such types (AMI or FMI), every case can be considered as a medical imaging modality. All these modalities are widely used by clinicians to study the structure/functionality of human body parts for diagnosis and treatment. It is frequently required to combine PET/SPECT with CT/MRI to simultaneous study the metabolic and molecular information received through PET/SPECT with fine anatomical details observed by CT/MRI. This concurrent study will help in the diagnosis and localization of many diseases like cancer, blockage in coronary arteries and brain-related diseases like Parkinson, Alzheimer etc. Further, in many cases, clinicians are required to co-register one or more anatomical images with functional images, for example, ultrasound-guided biopsy fused with PET and MRI. This registration can be done either at the hardware level or at the software level. The introduction of integrated PET-CT machine increases the acceptability of hardware-based registration systems as compared to software-based methods among the medics. One possible reason for this is the lack of validation of results achieved through software-based registration methods. On the other hand, software-based registration methods also have many advantages over the hardware-based registration systems like lesser exposure to radiation and no need for new investment on hardware etc. In this paper, both the methods with their merits and demerits are discussed in detail.

Machine Learning has emerged in the last years as the main solution to many of nowadays’ data-based decision problems. However, while new and more powerful algorithms and the increasing availability of computational resources contributed to a widespread use of Machine Learning, significant challenges still remain. Two of the most significant nowadays are the need to explain a model’s predictions, and the significant costs of training and re-training models, especially with large datasets or in streaming scenarios. In this paper we address both issues by proposing an approach we deem predictive and user-centric. It is predictive in the sense that it estimates the benefit of re-training a model with new data, and it is user-centric in the sense that it implements an explainable interface that produces interpretable explanations that accompany predictions. The former allows to reduce necessary resources (e.g. time, costs) spent on retraining models when no improvements are expected, while the latter allows for human users to have additional information to support decision-making. We validate the proposed approach with a group of public datasets and present a real application scenario.

With intrinsic non-uniform illumination variations in laparoscopic images, such images often suffer from insufficient lighting and low visibility. This in turn may cause incorrect targeting, surgical risk, and extended operating time during laparoscopic surgery. Although various methods for nature image enhancement have been proposed in past decades, surgical laparoscopic image enhancement still needs improving due to issues including naturalness, blurred texture, color cast, and extensive computational time. To address these problems, this paper proposes a distributed Retinex optimization method for laparoscopic image enhancement using refined information fusion. The proposed method minimizes a distributed optimization model with l 2 -norm regularization terms, resulting in a low computational complexity. By integrating the refined image information into the optimization process, our method significantly enhances dark regions while preserving naturalness and texture structures. Experimental results show that our optimization algorithm is more effective than conventional enhancement algorithms in terms of vision augmentation, performance index, and computation time.

The prognostics and health management (PHM) aims to provide a comprehensive solution for equipment health care. Classifying the operation mode of excavator, one of the tasks in the PHM, is important to evaluate the remaining useful lifetime. Several studies have been conducted to classify the operations with either video or sensor data, but they have several limitations to use only one type of data. A model trained with sensor data cannot classify the similar operations such as “digging” and “ditch digging”, whereas a model with video data is vulnerable to surrounding condition like weather. In this paper, to overcome these shortcomings, we propose a deep neural network ensemble called FusionNet that classifies the operations of excavator. Two models are trained with sensor data and video frames respectively, where the feature extractors are transferred to the FusionNet. The proposed network ensemble performs a flexible and well-optimized classification by automatically calculating weights according to the extracted feature vectors and combining them. To verify the proposed model, several experiments are conducted with the real-world data. The proposed model achieves the accuracy of 99.17% which outperforms the conventional methods. We also confirm that the proposed model can address the shortcomings of using only one type of data and maximize the benefits through the automatic weighting of extracted features.

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches.

Automatic International Classification of Diseases (ICD) coding is an important task in the future of artificial intelligence healthcare. In recent years, a lot of traditional machine learning-based methods have been proposed, and they achieved good results on this task. However, these traditional machine learning-based methods for automatic ICD coding only focus on the semantic features of clinical notes and ignore the feature extraction of ICD titles that are the descriptions of ICD codes. In this paper, we propose a knowledge attention-based deep learning framework called KAICD for automatic ICD coding. KAICD makes full use of the clinic notes and the ICD titles. The semantic features of clinic notes are extracted by a multi-scale convolutional neural network. For ICD titles, we use attention-based Bidirectional Gated Recurrent Unit (Bi-GRU) to build a knowledge database, which can offer additional information. Depending on input clinic notes, we can use the attention mechanism to obtain different knowledge vectors from the knowledge database where some ICD titles are more relevant to the input clinic notes. Last, we concatenate the knowledge vectors and the semantic features of clinic notes, and use them for the final prediction. KAICD is tested on a public dataset Medical Information Mart for Intensive Care III (MIMIC III); it achieves micro-precision of 0.502, micro-recall of 0.428, and micro-f1 of 0.462, which outperforms other competing methods. Furthermore, the results of the ablation study show that the knowledge database of ICD titles learned by the attention-based Bi-GRU enhances the feature expression and improves the prediction performance.

Meta-learning aims to leverage experience from previous tasks to achieve an effective and fast adaptation ability when encountering new tasks. However, it is unclear how the generalization property applies to new tasks. Probably approximately correct (PAC) Bayes bound theory provides a theoretical framework to analyze the generalization performance for meta-learning with an explicit numerical generalization error upper bound. A tighter upper bound may achieve better generalization performance. However, for the PAC-Bayes meta-learning bound, the prior distribution is selected randomly which results in poor generalization performance. In this paper, we derive three novel generalization error upper bounds for meta-learning based on the PAC-Bayes relative entropy bound. Furthermore, in order to avoid randomly prior distribution, based on the empirical risk minimization (ERM) method, a data-dependent prior for the PAC-Bayes meta-learning bound algorithm is developed and the sample complexity and computational complexity are analyzed. The experiments illustrate that the proposed three PAC-Bayes bounds for meta-learning achieve a competitive generalization guarantee, and the extended PAC-Bayes bound with a data-dependent prior can achieve rapid convergence ability.

This paper empirically analyses the impact of changes to the set of training examples on the neural network error surface. Specific quantitative characteristics of the error surface related to properties such as ruggedness, modality and structure are measured and visualized as the set of training examples changes. Both the case of random subsampling and active learning from a fixed dataset are examined, producing ten different training scenarios. For each scenario eleven error surface characteristics are calculated for five common benchmark problems. The results demonstrate that the error surface characteristics calculated using only a subsample of the available data commonly do not generalize to that of the full dataset. The observed error surface characteristics are significantly impacted by the particular set of examples used to calculate error. Some error surface characteristics are significantly altered by small changes in the set of examples used to calculate error. The main finding from this study is that when the set of training examples may change during training the training of a neural network is in essence a dynamic optimization problem, suggesting that optimization algorithms developed specifically to solve dynamic optimization problems may be more efficient at training neural networks under such conditions.

Methods based on Graph Convolutional Networks (GCN) for skeleton-based action recognition have achieved great success due to their ability to exploit graph structural information from skeleton data. Recently, the bone information has attracted considerable attention as an effective modality which complements the more conventional joint information for action recognition. However, most existing GCN-based methods extract the joint and bone features with two separate GCN networks, ignoring the dependencies between them. In this paper, a novel GCN model is proposed to exploit the information across joints, bones and their relationship collaboratively on a single undirected graph instead of two separate networks. We call the proposed model Vertex-Edge Graph Convolutional Network (VE-GCN) since it conducts the graph convolution operation on the sampling area containing the designated vertexes from joints and edges from bones, respectively. In addition to conducting the Vertex-Edge graph convolution based on the physical connections of the skeleton, we further apply the Vertex-Edge graph convolution to the non-physical joint-joint and joint-bone connections to capture the distal dependencies, and then the convolution results on the non-physical connections are incorporated into the VE-GCN. Moreover, the Conditional Random Field (CRF) is adopted as the loss function to achieve the task of action recognition. Experimental results on four challenging benchmarks (NTU RGB+D, NTU RGB+D 120, N-UCLA, SYSU) show that the proposed model achieves state-of-the-art performance.

Recursive least mean p-power extreme learning machine (RLMP-ELM) is a newly proposed online machine learning algorithm and is able to provide a robust online prediction of the datasets with noises of different statistics. To further explore the proposed RLMP-ELM to be used in real-world embedded systems, a generic serial FPGA-based hardware architecture of RLMP-ELM is presented in this paper. The entire hardware architecture of RLMP-ELM includes three serial processing modules, which are implemented parameterizably and can be adapted for different application requirements. The hardware framework is in a serial fashion, but parallelization efforts are focused on the processes with high computing complexity by analysis of potential inter-task dependency. To overcome the limitation of memory bandwidth, the block RAM and ping-pong on-chip buffer are applied to improve the computational throughput. The validation experiments are performed through five datasets with different p values. Accuracy results show that our implementation on FPGA could achieve similar accuracy compared to 64-bit floating-point software implementation. We also report and compare hardware performance of our proposed architecture with other existing implementations. The results show that our hardware architecture offers the excellent balance among accuracy, logic occupation and hardware performance.

Fraud detection is essential for e-commerce platforms to maintain a fair business environment. Many existing works propose manually designed methods such as label propagation and dense block mining rules on built user-item graphs to detect fraud behaviours, but they are always heuristic and thus have limited performance. Other learning-based methods can either handle only the fraud detection problem well in the transductive scenario when there is only structural information or require rich content features to obtain a good inductive ability. Considering that content features are not always available in practice and there are usually many fraudulent behaviours that belong to newly emerging users and items, how to learn effective inductive rules with structures only is still underexplored. In this paper, we propose a subgraph-based method named SubGNN for collective fraud detection. In SubGNN, first, we extract the subgraphs around the given edges (user behaviours) to be tested. Then, we remove nodes’ global IDs so that SubGNN is entity-independent. Finally, by learning knowledge reasoning rules on extracted heterogeneous subgraphs using our proposed relational graph isomorphism network (R-GIN), a powerful graph neural network (GNN) model, SubGNN can achieve precise fraud detection. Experiments are conducted on publicly available Amazon and Yelp datasets and a newly collected Taobao dataset. The results clearly show the advantages and prospects of our method. When using SubGNN to detect fraudulent transactions on Taobao, the precision is higher than 0.99 and more than 90 % of fraud samples are recalled.

Rain removal from a single image is a challenging and significant task of image pre-processing. In this paper, we learn the multi-scale streaks from rainy images using feature pyramid, and to improve the effectiveness of the learning, we focus on the feature propagation re-usage and propagation in the extremely deep de-raining network. Specifically, we design a de-raining2 unit and propose a novel deep de-raining network, respectively, called Pyramid Fully Residual Unit and Network (PFR-Unit and PFR-Net). The PFR-Unit employs fully residual learning in each level of feature pyramid and the PFR-Net connects PFR-Units by a compact dense architecture. The fully residual learning encourages the feature re-usage in PFR-Unit by performing identity mapping for all available shortcuts. The compact dense connection strengthens the feature propagation between the PFR-Units and ensures the unicity of the learning space for the PFR-Units. Along with negative SSIM loss, the PFR-Net presents a good performance in single image de-raining. Comprehensive experimental results show that the PFR-Net outperforms the state-of-the-art single de-raining methods with a big margin on Rain100H, Rain100L and Rain1200 datasets.

This paper focuses on the recognition problem of dynamical patterns consisting of sampling sequences. Specifically, based on the concept of structural stability, a novel similarity measure for dynamical patterns is first given. Then, a specific realization is provided, which consists of: (1) an approximation scheme for computation of partial derivative information by utilizing the knowledge learned through deterministic learning; (2) a similarity comparison scheme using the recognition errors generated from the discrete-time dynamical estimators; and (3) performance analysis of the recognition scheme with general recognition conditions. Compared with the existing methods, in which misrecognition may occur when the differences of dynamics between adjacent training patterns are very small, the proposed method is more appealing in the sense that, the partial derivatives of dynamics are introduced to complement the similarity measures, such that the recognition performance is much improved. Simulation studies are conducted to verify the proposed method in a relatively large data set.

Video dehazing is of crucial importance to the outdoor surveillance, and its main challenges mainly come from the spatial-temporal coherence and computational efficiency. This paper presents an efficient real-time video dehazing approach via incremental transmission learning and spatial-temporally coherent regularization, while explicitly suppressing the possible visual artifacts. First, we propose to initialize the transmission map frame-by-frame by a boundary constrained open dark channel model. Then, supposing a scene point yields highly correlated transmission values between adjacent frames, we impose a temporally coherent term to maintain the temporal consistency of consecutive transmission values, and simultaneously derive an incremental transmission adjusting term to adapt the abrupt scene depth changes between the adjacent frames. Accordingly, the highly correlated and spatial-temporally coherent transmission maps can be optimized for each video frame dehazing, whereby the flickering artifacts can be well reduced. Finally, we group each haze/haze-free images in pair and further utilize the guided joint bilateral filter to suppress the visual artifacts that possibly amplified during the dehazing process. The experimental results show that the proposed video dehazing approach is able to well preserve the spatial-temporal coherence, runs sufficiently fast, and also performs favorably compared to the state-of-the-art methods.

Metallic pieces are typically machined by continuous tool passes, which usually causes regular patterns in the form of straight edges in the surface of the pieces. An irregular pattern on the surface of the piece implies a decrease of the quality of the machined piece. In this paper, we propose an acquisition system and a machine-vision based method to describe the texture of the inner and outer surfaces of machined pieces with cylindrical holes. In order to capture images of the hole surface, we used a microscope camera connected to a rigid industrial boroscope. Considering the extracted texture descriptors, a significant correlation is shown. Consequently, the feature vector is reduced and then classified by several algorithms using an exhaustive grid search strategy with 10-fold cross validation. Best results are achieved with the Extremely Randomized Trees classifier with a mean test score on the hold out set of 92.98%, what improves previous research and meets the requirements of the field.

Face recognition systems are widely used for target recognition and identity authentication, such as automated teller machines, mobile phones, and entrance guard systems. However, face recognition systems are vulnerable to presentation attacks, such as photo, replay, and 3D mask attacks. In particular, silicone mask attacks pose a greater threat to face recognition systems because high-quality silicone masks do living properties. To promote the development of face anti-spoofing detection algorithms for silicone mask attacks, this paper constructs a Silicone Mask Face Motion Video Dataset (SMFMVD) containing 200 real face videos and 200 silicone mask face videos. These videos include different facial motions collected from 20 subjects. Moreover, inspired by the observation that the silicone mask face’s facial movement is not so natural as the real face, we propose a novel silicone mask face anti-spoofing detection method based on visual saliency and facial motion characteristics. Specifically, we compute the visual saliency map of a given face image by simulating two kinds of eye movement behaviors, namely “gaze” and “saccade”. Then, we propose a saliency-weighted histogram of local binary pattern operator to extract facial texture features in spatial domain and a saliency-guided histogram of oriented optical flow operator to extract facial motion features in temporal domain. Finally, the support vector machine is used to fuse two groups of facial features to distinguish real and spoof faces. Extensive experiments on public and self-built datasets show its superiority over the state-of-the-art methods.

The traditional robust principal component analysis (RPCA) via decomposition into low-rank plus sparse matrices offers a powerful framework for a large variety of applications in computer vision. However, the reconstructed image experiences serious interference by Gaussian noise, resulting in the degradation of image quality during the denoising process. Thus, a novel manifold constrained joint sparse learning (MCJSL) via non-convex regularization approach is proposed in this paper. Morelly, the manifold constraint is adopted to preserve the local geometric structures and the non-convex joint sparsity is introduced to capture the global row-wise sparse structures. To solve MCJSL, an efficient optimization algorithm using the manifold alternating direction method of multipliers (MADMM) is designed with closed-form solutions and it achieves a fast and convergent procedure. Moreover, the convergence is analyzed mathematically and numerically. Comparisons among the proposed MCJSL and some state-of-the-art solvers, on several accessible datasets, are presented to demonstrate its superiority in image denoising and background subtraction. The results indicate the importance to incorporate the manifold learning and non-convex joint sparse regularization into a general RPCA framework.

In this paper, we develop a novel Joint Model Compression (referred to as JMC) method by combining structured pruning and dense knowledge distillation techniques to significantly compress original large language model into a deep compressed shallow network. In particular, a new Direct Importance-aware Structured Pruning (referred as DISP) approach is proposed to structurally prune the redundant structures in the Transformer networks directly based on the corresponding parameter matrices in the model. Besides, a Dense Knowledge Distillation (referred to as DKD) method is developed with a many-to-one layer mapping strategy to leverage more comprehensive layer-wise linguistic knowledge for the distillation. Further, the proposed structured pruning and dense knowledge distillation are integrated together to perform the joint compression, which enables us to achieve a significant compression without sacrificing model accuracy. The extensive experimental results across four NLP tasks on seven datasets demonstrate its effectiveness and superiority to the baselines, while maintaining similar performance to original large model with further remarkable benefits for inference-time speedup and memory efficiency.

The fitness landscape is a theory applied to the evolutionary dynamics of biological evolution to explain the behavior of evolutionary algorithms in the solution of optimization problems. With the continuous advancement of evolutionary algorithm optimization, a fitness landscape can present more abundant feature information, such as the local fitness, fitness distance correlation, and landscape roughness. These landscape features reflect the optimal solution distribution, quantity, and local unimodal topology of the optimization problem from various angles. This paper expresses the adaptability landscape features of typical optimization problems, engages in a quantitative analysis of the fitness distance correlation information, evaluates the difficulty of solving the problem within the search space, and obtains the correlation degree classification result. The search strategy adapts the mixed mutation and the fitness distance correlation for differential evolution. Empirical studies show that, the fitness distance correlation search strategy for the differential evolution algorithm can avoid falling into the local optimum, improve accuracy and convergence, and solve the single-objective optimization problem in a more comprehensive manner.

Illumination normalization is one of the most challenging issues for facial analysis. To be specific, the variation of environmental illumination influences the visual quality of an image and drastically degrades the performance of face recognition, detection, landmark and other related applications. Retinex theory provides an important concept for processing illumination variation, which supposes that a face image can be decomposed into [an invariant reflectance component and a variant illumination component. Enlighten by this theory, in this paper, we put forward a novel deep learning approach which combines self-supervised learning and adversarial training for face illumination normalization (FIN-GAN). The proposed FIN-GAN framework can be implemented by two steps. Firstly, self-supervised learning is employed to decompose the original face image into the illumination component and the illumination-invariant component with Retinex constraint. Then, we employ the conditional generative adversarial network for face image reconstruction. For network optimization, we design the combined loss to ensure visual quality and preserve identity information. Experiments are performed on Extended-YaleB, CAS-PEAL, CMU-PIE and Multi-PIE datasets. Through multiple quantitative criteria, we demonstrate that the proposed FIN-GAN obtains promising performance in face illumination normalization.

The outbreak and rapid spread of coronavirus disease 2019 (COVID-19) has had a huge impact on the lives and safety of people around the world. Chest CT is considered an effective tool for the diagnosis and follow-up of COVID-19. For faster examination, automatic COVID-19 diagnostic techniques using deep learning on CT images have received increasing attention. However, the number and category of existing datasets for COVID-19 diagnosis that can be used for training are limited, and the number of initial COVID-19 samples is much smaller than the normal’s, which leads to the problem of class imbalance. It makes the classification algorithms difficult to learn the discriminative boundaries since the data of some classes are rich while others are scarce. Therefore, training robust deep neural networks with imbalanced data is a fundamental challenging but important task in the diagnosis of COVID-19. In this paper, we create a challenging clinical dataset (named COVID19-Diag) with category diversity and propose a novel imbalanced data classification method using deep supervised learning with a self-adaptive auxiliary loss (DSN-SAAL) for COVID-19 diagnosis. The loss function considers both the effects of data overlap between CT slices and possible noisy labels in clinical datasets on a multi-scale, deep supervised network framework by integrating the effective number of samples and a weighting regularization item. The learning process jointly and automatically optimizes all parameters over the deep supervised network, making our model generally applicable to a wide range of datasets. Extensive experiments are conducted on COVID19-Diag and three public COVID-19 diagnosis datasets. The results show that our DSN-SAAL outperforms the state-of-the-art methods and is effective for the diagnosis of COVID-19 in varying degrees of data imbalance.

Recommender system uses user-item historical interactions to portray user preferences. Due to the problem of data sparseness, auxiliary information is introduced to describe user preferences, such as user/item attribute information. However, some of these methods only consider user(item) attributes when modeling user preferences. Although another part of the methods interact with user attributes and item attributes, this interaction doesn’t consider the potential preference of a certain attribute of the user to a certain attribute of the item. This makes the model unable to capture fine-grained user preferences. In fact, capturing fine-grained user preferences in terms of attributes is very effective in improving the recommendation effect. Therefore, in this paper we propose a recommendation method to capture user fine-grained preferences through attribute information. Specifically, there are two elaborately designed modules, interactive preference module and attribute fine-grained preference module. The former uses user-item historical interactions to model user’s interaction preferences. In order to obtain the user’s fine-grained preferences, the latter interacts with each attribute of the user and all the attributes of the item and uses the attention mechanism to mine the user’s more fine-grained preferences on attributes. Extensive experiments on three publicly available datasets demonstrate the effectiveness of the proposed method.

Multi-modal abstractive summarization for videos is an emerging task, aiming to integrate multi-modal and multi-source inputs (video, audio transcript) into a compressed textual summary. Although recent multi-encoder-decoder models on this task have shown promising performance, they did not explicitly model interactions of multi-source inputs. While some strategies like co-attention are utilized for modeling this interaction, considering ultra-long sequences and additional decoder in this task, the coupling of multi-modal data from multi-encoders and decoder needs complicated structure and additional parameters. In this paper, we propose a concise Decoder-only Multi-modal Transformer (D-MmT) based on the above observations. Specifically, we cut the encoder structure, and introduce an in-out shared multi-modal decoder to make the multi-source and target fully interact and couple in the shared feature space, reducing the model parameter redundancy. Also, we design a concise cascaded cross-modal interaction (CXMI) module in the multi-modal decoder that generates joint fusion representations and spontaneously establishes a fine-grained intra- and inter- association between multi-modalities. In addition, to make full use of the ultra-long sequence information, we introduce a joint in-out loss to make the input transcript also participate in backpropagation to enhance the contextual feature representation. The experimental results on the How2 dataset show that the proposed model outperforms the current state-of-the-art approach with fewer model parameters. Further analysis and visualization show the effectiveness of our proposed framework.

Aspect level sentiment classification aims to extract fine-grained sentiment expressed towards specific aspects from a sentence. The key to this task lies in connecting aspects and their respective sentiment contexts. Existing methods measure the dependency weights between aspects and context words via either the semantic similarity between words captured by attention mechanism or the structural proximity between words in syntactic structures. However, methods in both groups fail to fully exploit explicit syntactic dependency, which we argue should be critical to identify sentiment contexts. In this paper, we propose a novel syntactic-dependency-based attention network (SDATT) to incorporate explicit syntactic dependency for aspect level sentiment classification. SDATT first models the dependency path between each word and the aspect to characterize aspect-oriented syntactic representation of each word. The generated syntactic representations are later fed into the attention layer to help infer the dependency weights for sentiment prediction. Experimental results on five benchmark datasets show the superior performance of the proposed model over state-of-the-art baselines.

Light Field (LF) images have the unique advantage of recording scenes from multiple viewpoints, which provides many applications, such as refocusing and depth estimation. However, low-light conditions can severely influence these applications. In this paper, we propose a two-stage deep learning framework for the LF restoration under low-light imaging. First, there is a multi-to-one (MTO) network, which restores each view separately by utilizing multiple auxiliary views. All the views share the same feature extractor, with an efficient spatial-channel attention mechanism to extract more informative features. A channel-attention feature fusion (CAFF) module is designed to selectively fuse more useful complementary information from the auxiliary views, with a learnable global scalar to adjust the importance of the auxiliary features. Then, the outputs of the MTO network are further enhanced by an (all-to-all) ATA network, which uses spatial and angular residual blocks to process all the views synchronously for fully encoding the spatial-angular information. Extensive experiments have been conducted to demonstrate the superior performance and robustness of our method, i.e., it can restore the luminance, spatial details and angular geometries of the LF images under various light levels effectively.

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features.

In recent years, deep learning techniques have significantly improved the performance of single image super-resolution (SISR). However, this improvement is often achieved at the cost of introducing a large amount of parameters, which limits the real-world applications for SISR. In this paper, we propose a lightweight SISR network called Multi-scale Channel Attention Network for Image Super-Resolution (MCSN). Our contributions are threefold. First of all, the multi-scale feature fusion block (MSFFB) can extract multi-scale features by filters with different receptive fields. Secondly, the channel shuffle attention mechanism (CSAM) encourages the flow of the information across feature channels and enhances the ability of feature selection. Thirdly, the global feature fusion connection (GFFC) can effectively improve feature utilization. Extensive experiments demonstrate that the parameter amount of our method is reduced by 3/4 compared with the current state-of-the-art MSRN method, while both the subjective visual effect and objective quality of the reconstructed high-resolution images are significantly better.

A Reinforcement Learning (RL) agent interacts with the environment to learn a policy with high accumulated rewards through attempts and failures. However, RL suffers from its own trial-and-error learning nature, which results in an unstable learning process. In this paper, we investigate a common phenomenon called rewards drop at the late-stage RL training session, where the rewards trajectory oscillates dramatically. In order to solve such a problem, we propose a novel rewards shaping technique named Variance Aware Rewards Smoothing (VAR). We show that the proposed method reduces the variance of rewards and mitigates the rewards drop problem without changing the formulation of the value function. Furthermore, the theoretical analysis of convergence of VAR is provided, which is derived from the γ -contraction operator and the fixed point attribute of the value function. Finally, the theoretical results are illustrated by extensive results on various benchmarks and advanced algorithms across different random seeds to demonstrate the effectiveness and the compatibility of VAR.

In the paper, data-driven finite-horizon robust formation-containment control scheme is developed based on integral reinforcement learning and zero-sum game for perturbed multi-agent networks with completely unknown nonlinear dynamics. At first, distributed finite-time sliding mode estimators are designed to obtain the desired states of leaders and followers, respectively. Then finite-horizon robust leader formation control and follower containment control are achieved based on proposed model-free integral reinforcement learning algorithms implemented by critic-actor-disturbance structure, in the framework of multi-player zero-sum game where the non-quadratic performance index for each agent considers the influence of saturated inputs and disturbances of local neighbors thoroughly. Furthermore, it is proved that the whole network has bounded L 2 gain robust stability and Nash equilibrium of zero-sum game exists. Simulation results are provided to demonstrate the effectiveness of the proposed scheme.

Differential privacy, a strict privacy notion, prevents the presence or absence of a single data from being identified. This paper proposes a novel average consensus algorithm to achieve differentially private average consensus on multi-agent systems (MASs) with general directed graphs. For MASs under the proposed algorithm, necessary and sufficient conditions are established on achieving average consensus and preserving differential privacy of the states of the agents at any time instant. Finally, numerical examples are provided to illustrate these results.

Both image decomposition and data completion are not only ubiquitous but also challenging tasks in the study of computer vision. In this paper, different from existing approaches, we propose a novel regularization model for image decomposition and data completion, which integrates relative total variation (RTV) with Schatten- 1 / 2 or Schatten- 2 / 3 norm, respectively. RTV is shown to be able to extract the fundamental structure effectively from the complicated texture patterns and largely to avoid the drawback of oil painting artifacts. Schatten quasi-norm is used to capture texture patterns in a completely-separated manner. The proposed model is in essence divided into “RTV+ double nuclear norm” and “RTV+ Frobenius/nuclear hybrid norm”, which can be solved by splitting variables and then by using the alternating direction method of multiplier (ADMM). Convergence of the algorithm is discussed in detail. The proposed approach is applied to several benchmark low-level vision problems: gray-scale image decomposition and reconstruction, text removal, color natural scene image completion, and visual data completion, demonstrating the distinguishable effectiveness of the new model, comparing to the latest developments in literature.

In this paper, a novel intermittent noise-diffusion layer is proposed to synchronize a network system. Different from most of the traditional noise-induced synchronization or noise stabilization (noise exists in nodes of the network), we give an intermittent noise-diffusion layer (the noise exists in the edges of the network) to study the almost sure exponential synchronization. The theoretical results show that the general network system can be synchronized in the almost sure sense as long as the topological structure of the network in the intermittent noise-diffusion layer is an undirected connected graph. Two examples are proposed to illustrate the theory obtained.

To ensure the reliability and security of the data, the large-scale distributed storage system usually adopts the data redundancy mechanism to repair the data on the faulty nodes. Comparing with the replication-based method, the data redundancy mechanism of erasure code can effectively improve the use of storage space, but it may result in the large network overhead when recovering the data. The regenerating code is an improved erasure code, which can reduce the quantity of data transmission compared to that of the erasure code. Adopting the regenerating code to repair the data on a faulty node requires constructing an optimal repair tree to maximum the bandwidth of the bottleneck link, which is an NP-hard problem. To construct the optimal repair tree, a hybrid genetic algorithm is proposed in this paper. In particular, our proposal comprehensively considers the network topology and link bandwidth of storage nodes and designs a problem-specific cross-correlation operator, mutation operator and local search operator. In addition, we provided the mathematical proof of the global convergence with probability one with respect to the proposed hybrid genetic algorithm. Through a series of simulation experiments, the results show that our proposal is able to determine the optimal repair tree, which effectively reduces the delay of faulty node repair in distributed storage systems, and improve the repairing efficiency.

The ever-increasing complexity of robots causes failures of them as a side effect. Successful detection of anomalies in robotic systems is a key issue in order to improve their maintenance and consequently reducing economic costs and downtime. Going one step further in the detection of anomalies in robots, different mechanisms to deal with data irregularities are proposed and validated in present paper in order to increase detection rates. More precisely, strategies to overcome missing values and class imbalance are considered as complementary tools to get better one-class classification results. The effect of such strategies is evaluated through cross-validation when applying a standard supervised learning model, the Support Vector Machine. Experiments are run on an up-to-date and public dataset that contains some examples of different software anomalies that the middleware of the robot under analysis may experience.

K-nearest neighbor (KNN) based weighted multi-class twin support vector machines (KWMTSVM) is a novel multi-class classification method. In this paper, we propose a novel least squares version of KWMTSVM called LS-KWMTSVM by replacing the inequality constraints with equality constraints and minimized the slack variables using squares of 2-norm instead of conventional 1-norm. This simple modification leads to a very fast algorithm with much better results. The modified primal problems in the proposed LS-KWMTSVM solves only two systems of linear equations whereas two quadratic programming problems (QPPs) need to solve in KWMTSVM. The proposed LS-KWMTSVM, same as KWMTSVM, employed the weight matrix in the objective function to exploit the local information of the training samples. To exploit the inter class information, we use weight vectors in the constraints of the proposed LS-KWMTSVM. If any component of vectors is zero then the corresponding constraint is redundant and thus we can avoid it. Elimination of redundant constraints and solving a system of linear equations instead of QPPs makes the proposed LS-KWMTSVM more robust and faster than KWMTSVM. The proposed LS-KWMTSVM, commensurate as the KWMTSVM, all the training data points into a “1-versus-1-versus-rest” structure, and thus our LS-KWMTSVM generate ternary output {-1,0,+1} which helps to deal with imbalance datasets. Numerical experiments on several UCI and KEEL imbalance datasets(with high imbalance ratio) clearly indicate that the proposed LS-KWMTSVM has better classification accuracy compared with other baseline methods but with remarkably less computational time. This work was supported by Science and Engineering Research Board (SERB) under Early Career Research Award Grant No. ECR/2017/000053 and Ramanujan fellowship Grant No. SB/S2/ RJN-001/2016. This work was also supported by Council of Scientific & Industrial Research (CSIR), New Delhi, INDIA under Extra Mural Research (EMR) Scheme Grant No. 22(0751)/17/ EMR-II.

Interpersonal relation plays an essential role to gain understandings on how people interact with each other. In computer vision, interpersonal relations provide vital information to interpret people’s behaviors. However, the existing research has either omitted the interaction information between subjects or the structural information in the images. In this paper, we propose a new architecture to reason interpersonal relations based on higher-order graph networks and multi-scale features. First, we extract features of the whole images, the facial features, and the union region of face pairs. Apart from the pixel-wise features, we also consider the positional features of face-to-face pairs and the spatial scene cues. Higher-order Graph Neural Networks (GNNs) were employed to map out the interpersonal relations based on the feature extracted. Experimental results show that the proposed Higher-order Graph Neural Networks with multi-scale features can effectively recognize the social relations in images with over 5% improvement in absolute balanced accuracy compared with the state-of-the-art work.

The use of hierarchical networks in motor pattern generation in hexapod robots and in insect movement provides ways to understand and control both systems. In this paper we consider the Central Pattern Generator (CPG) of insect movement consisting of six coupled neurons developed by Ghigliazza and Holmes (2004) that produces the global leg coordination pattern. We provide a detailed study of the possible gaits generated by the CPG through different numerical techniques recently developed for the study of small networks, which allow us to consider the complete model without any simplification. We combine the analysis of isolated neurons (using several three dimensional parameter spaces) to give a roadmap of the dynamics of the neurons involved, lateral phase lag plots to show the convergence and transitions towards particular patterns and a quasi-Monte Carlo sweeping method to describe different pattern routes in the parametric phase space. In all of our studies we have found the same final result: most of the observed patterns follow routes that lead to the stable tripod gait. We obtain several routes made of symmetrical patterns, but despite considering mainly a symmetric leg CPG we detect non-symmetrical patterns that provide (minor) routes present in the model. The bifurcations of the main pattern routes detected in the parameter space are studied in detail using continuation techniques. This study reveals the bifurcations that create and destroy the different routes and how for large values of some parameters only the tripod gait is present. Based on the symmetric case, a preliminary study of asymmetric configurations is done revealing the robustness of the already located patterns and routes. Due to the relevance of the tripod gait, and since more parameters are included in this study, we introduce an algorithm to locate the limit of the tripod gait in the parameter space, which shows that there is a large three parametric region where the tripod pattern is ubiquitous and highly dominant in rapidly moving insect regimes.

This paper proposes a class of neural dynamics models based on a neural dynamic approach to realize attitude tracking control of a flapping wing micro aerial vehicle (FWMAV), different from the existing conventional controllers. Firstly, from the perspective of FWMAV attitude tracking control, an attitude tracking model is presented. Further theoretical analyses reveal that there exist theoretical errors on such a controller when solving the attitude tracking control problem. To remedy its deficiency, we propose a class of adaptive neural dynamics models with activation functions. Different from the existing controller, the adaptive neural dynamics model adopts different activation functions to accelerate the convergence speed of attitude tracking errors and makes errors converge to zero in a short time. Finally, simulation results and theoretical analyses are provided, which verify that the proposed controllers possess the superiority and effectiveness.

The graph model is a reliable propagation mechanism in saliency detection, saliency value propagation diffusion results between any two nodes are determined merely by defining an effect affinity matrix. Most existing methods generally calculamatrix using mean values of single or multiple feature vectors, not fully exploit the diversity and consistency of multi-view features, may produce poor foreground uniformity and completeness in the complex scene. Multi-views should share an affinity matrix as well as complement each other. In this paper, we propose a graph-based saliency detection with a learning joint affinity matrix. First, we capture multiple appearance features from the image and generate a learning joint affinity matrix based on low-rank representation. Then, for computing an effect affinity matrix, we linearly integrate the traditional affinity matrix and learning joint affinity matrix, helping to construct an affinity graph for diffusion-based compactness. Finally, to effectively optimize the initial saliency map, we diffuse the learning joint affinity matrix and traditional impact factor matrix via cross-view diffusion processing, which begets an approving advantage for single-layer cellular automata. Results on three benchmark datasets demonstrate that our proposed method shows the best performance against nine state-of-the-art models.