Search

Person re-identification is to recognize a pedestrian who has been observed at different places in monitoring system, which is done by computer automatically. Important practical significance can be found in this paper on improving the intelligence of monitoring system. There are two challenging problems needed to be solved for person re-identification, which is feature representation and matching. Designing a suitable feature representation method is both challenging and crucial. Ideally, the extracted features should be robust enough to cope with the various situations of the monitoring environment, such as posture difference, illumination changing, and shooting angle difference, etc. There are a lot of challenges for feature matching too, such as small sample size, inter-class confusion, and intra-class variation, etc. In order to meet these challenges, a new person re-identification scheme was put forward in this paper. The following three aspects would be used to describe it. First, an improved BOF feature extraction algorithm based on SURF was put forwarded, SURF algorithm extracts the preliminary feature descriptor and generates visual dictionary, so the influence factors such as illumination and scale invariants can be deal with. And then, covariance descriptor was adopted by the algorithm. It has high robustness, and matching accuracy can be improved too for the case of small sample. Finally, an effective classifier was designed by LIBSVM based on improved BOF algorithm, so the efficiency of the person re-identification algorithm can be improved. The proposed method was compared with the current mainstream algorithm through experiment, and it can be found through the experimental results that it is effective to solve the difficult problems for person re-identification.

Many methods achieve the visual object tracking task with deep learning technologies. As the deep features of different levels contain various semantic information and functions, this paper presents a multi-hierarchy feature aggregation approach to tackle the specific issues in the tracking task, which consists of two aspects. On one hand, this paper integrates the features captured by the offline and online classifiers at the score level, which constructs complementary roles of these classifiers to enhance the stability of classification. Besides, the proposed offline classifier is continuously optimized with different levels of features to reinforce classification constraints. On the other hand, we design a butterfly attention module to promote the capacity of multi-hierarchy feature aggregation in the regression network, which aims to fuse and strengthen the multi-scale features by attending to their spatial information. It can capture more spatial contexts by utilizing the self-attention mechanism during the fusion procedure, and preserve the hierarchy of the features during the strengthening process. Extensive experiments on four public datasets, i.e., VOT2018, OTB100, NFS and LaSOT datasets, demonstrate the effectiveness of the proposed methods.

Context and rationale Intrusion Detection, the ability to detect malware and other attacks, is a crucial aspect to ensure cybersecurity. So is the ability to identify this myriad of attacks. Objective Artificial Neural Networks (as well as other machine learning bio-inspired approaches) are an established and proven method of accurate classification. ANNs are extremely versatile – a wide range of setups can achieve significantly different classification results. The main objective and contribution of this paper is the evaluation of the way the hyperparameters can influence the final classification result. Method and results In this paper, a wide range of ANN setups is put to comparison. We have performed our experiments on two benchmark datasets, namely NSL-KDD and CICIDS2017. Conclusions The most effective arrangement achieves the multi-class classification accuracy of 99.909% on an established benchmark dataset.

Single-object tracking is regarded as a challenging task in computer vision, especially in complex spatio-temporal contexts. The changes in the environment and object deformation make it difficult to track. In the last 10 years, the application of correlation filters and deep learning enhances the performance of trackers to a large extent. This paper summarizes single-object tracking algorithms based on correlation filters and deep learning. Firstly, we explain the definition of single-object tracking and analyze the components of general object tracking algorithms. Secondly, the single-object tracking algorithms proposed in the past decade are summarized according to different categories. Finally, this paper summarizes the achievements and problems of existing algorithms by analyzing experimental results and discusses the development trends.

In the process of Human-robot skill transfer, we require the robot to reproduce the trajectory of teacher and expect that the robot can generalize the learned trajectory. For the trajectory after generalization, we expect that the robot arm can accurately track. However, because the model of the robot can not be accurately obtained, some researchers have proposed using a neural network to approximate the unknown term. The parameters of the traditional RBF neural network are usually selected through the empirical and trial-and-error method, which maybe biased and inefficient. In addition, due to the end-effector of the mechanical arm trajectory will be constantly changing according to the needs of the task, when the neural network of compact set cannot contain the whole input vector, the neural network cannot achieve the ideal approximation effect. In this paper, the broad neural network is used to approximate the unknown terms of the robot. This method can reuse the motion controller that has been learned and complete other motions in the robot operating space without relearning its weight parameters. In this paper, the effectiveness of the proposed method is proved by the ultrasound scanning task.

This paper proposes a hybrid architecture based on neural networks, fuzzy systems, and n-uninorms for solving pattern classification problems, termed as ENFS-Uni0 (short for evolving neuro-fuzzy system based on uni-nullneurons). The model can produce knowledge in an on-line (single-pass) and evolving learning context in a particular form of neuro-fuzzy rules representing the dependencies among input features through IF-THEN type relations. The rules antecedents are thereby realized through uni-nullneurons, which are constructed from n-uninorms, leading to the possibility to express both, AND- and OR-connections (and a mixture of these) among the single antecedent parts of a rule (and thus achieving an advanced interpretability aspect of the rules). The neurons’ evolution is done through an extended version of an autonomous data partition method (ADPA). On-line interpretation of the timely evolution of rules is addressed by (i) a concept for tracking the degree of changes of the rules over data stream samples, which may indicate experts/operators how much dynamics is in the process and may be used as a structural active learning component to request operator’s feedback in the case of significant changes and (ii) a concept for updating feature weights incrementally. These weights express the (possibly changing) impact degrees of features on the classification problem: features with low weights can be seen as unimportant and masked out when showing rules to an expert ( → rule length reduction). The rules’ consequents are represented by certainty vectors and are recursively updated by an indicator-based recursive weighted least squares (I-RWLS) approach (one RWLS estimator per class) where the weights are given through the neuron activation levels in order to gain stable local learning. The model proposed in this paper was successfully compared to related hybrid and evolving approaches in the literature for classifying binary and multi-class patterns. The results obtained by the proposed model show an outperformance of the related works in terms of higher accuracy trend lines over time, while offering a high degree of interpretability through coherent neuro-fuzzy rules to solve the classification problems.

Subtle facial expression recognition (SFER) aims to classify facial expressions with very low intensity into corresponding human emotions. Subtle facial expression can be regarded as a special kind of facial expression, whose facial muscle movements are more difficult to capture. In the last decade, various methods have been developed for common facial expression recognition (FER). However, most of them failed to automatically find the most discriminative parts of facial expression and the correlation of muscle movements when human makes facial expression, which makes them unsuitable for SFER. To better solve SFER problem, an attention mechanism based model focusing on salient local regions and their correlations is proposed in this paper. The proposed method: 1) utilizes multiple attention blocks to attend to distinct discriminative regions and extract corresponding local features automatically, 2) a correlation attention module is integrated in the model to extract global correlation feature over the salient regions, and finally 3) fuses the correlation feature and local features in an efficient way for the final facial expression classification. By this way, the useful but subtle local information can be utilized in more detail, and the correlation of different local regions is also extracted. Extensive experiment on the LSEMSW and CK+ datasets shows that the method proposed in this paper achieves superior results, which demonstrates its effectiveness.

The purpose of this paper is to give an overview of various well known medical image registration techniques with a special focus on registration between anatomical and functional medical images. Examples of anatomical medical images (AMI) are Computer Tomography (CT), Magnetic Resonance Images (MRI), X-ray radiographs and ultrasound etc. whereas Positron Emission Tomography (PET), Single Photon Emission Tomography (SPECT) and fMRI are examples of functional medical images (FMI). Irrespective of such types (AMI or FMI), every case can be considered as a medical imaging modality. All these modalities are widely used by clinicians to study the structure/functionality of human body parts for diagnosis and treatment. It is frequently required to combine PET/SPECT with CT/MRI to simultaneous study the metabolic and molecular information received through PET/SPECT with fine anatomical details observed by CT/MRI. This concurrent study will help in the diagnosis and localization of many diseases like cancer, blockage in coronary arteries and brain-related diseases like Parkinson, Alzheimer etc. Further, in many cases, clinicians are required to co-register one or more anatomical images with functional images, for example, ultrasound-guided biopsy fused with PET and MRI. This registration can be done either at the hardware level or at the software level. The introduction of integrated PET-CT machine increases the acceptability of hardware-based registration systems as compared to software-based methods among the medics. One possible reason for this is the lack of validation of results achieved through software-based registration methods. On the other hand, software-based registration methods also have many advantages over the hardware-based registration systems like lesser exposure to radiation and no need for new investment on hardware etc. In this paper, both the methods with their merits and demerits are discussed in detail.

Extreme-learning-machine based auto-encoder (ELM-AE) is regarded as a useful architecture with fast learning speed and general approximation ability, and stacked ELM is used to develop efficient and effective deep learning networks. However, considering features learned from conventional ELM-AEs have issues of weak nonlinear representation ability and random factors in feature projection, this paper proposes an improved ELM-AE architecture which utilize low-rank matrix factorization to learn optimal low-dimensional features. Two superiorities can be obtained compared to conventional ELM-AEs. One is the dimensionality of the hidden layer in ELM-AE could be set arbitrarily, e.g. a higher-dimension hidden layer could lower the random effect in feature learning and enhance features representation ability. The other is enhancing features nonlinear ability, since features are learned directly from the nonlinear outputs of hidden layer. Finally, comparison experiments on numerical and image datasets are implemented in this paper to verify the superior performance of the proposed ELM-AE in this paper.

Deep learning has made substantial progress in crowd density estimation, but in practical application, due to the interference factors such as uneven distribution of crowd and changes in illumination, the existing methods still have large errors in counting. To solve the above problems, a crowd density estimation method based on multichannel dense grouping network is proposed. To solve the problem that it is difficult to extract the multiscale information of the crowd due to the uneven distribution of the crowd, multichannel dense grouping module (McDGM) is designed. In the module, Improved grouping convolution block (IGCB) is dense connected with other layers to obtain different levels of crowd characteristics, so as to avoid the loss of multi-scale information. In addition, the parameters of IGCB are reduced by group convolution. Then, to overcome the change of illumination, a crowd image enhancement algorithm is designed, which makes the image clear by the average pixel value and adjusting the contrast. Finally, to enhance the sensitivity of the network to crowd counting, a new loss function is proposed, which adds counting loss to the previous pixel space loss to improve the accuracy of crowd counting. The algorithm in this paper has been tested on ShanghaiTech and UCF_CC_50 datasets. The test results show that the algorithm in this paper has better statistical accuracy.

In this paper, a neural synchronization of the optimal structure-based group of neural networks is proposed. Asymmetric cryptography is widely used to generate a key amongst two parties and to exchange the key through an insecure channel. However, since the methods that used this strategy, like RSA, have been compromised, new methods for producing a key that can offer security must be discovered. A new group of cryptography known as neural cryptography was created to solve this issue. The primary aim of neural cryptography is to produce a secret key over an unreliable medium. The optimal neural network architecture for creating and defining a secret key between the two authorized individuals is examined in this article. Furthermore, studies into the coordination of a group of neural networks are uncommon. For the design of the public key exchange protocol, synchronization of a cluster of neural networks with Three Layer Tree Parity Machine (TLTPM) is proposed. To calculate the synchronization time, steps taken, and the number of times the attacking neural network could replicate the actions of the two accepted networks, more than 15 million simulations were run. Various parametric experiments have been conducted on the proposed methodology. Simulations of the approach show that it is correct, according to the results of the paper.

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

This paper proposes a solution to translating the visible images into infrared images, which is challenging in computer vision. Our solution belongs to unsupervised learning, which has recently become popular in image-to-image translation. However, existing methods do not produce satisfactory results because (1) most existing methods are mainly used in entertainment scenarios with single scenes and low complexity. The problem solved by this article is more diverse and more complicated. (2) The infrared response of objects depends not only on itself but also on the current environment, and existing methods cannot correlate long-range dependent objects. In this paper, We propose Global Spatial Attention (GSA), which enhances dependence between long-range objects and improves the synthesized image quality. Compared with other methods, GSA can save more space and time. Moreover, we introduce the idea of subspace learning into the neural network to make training more stable. Our method takes unpaired visible images and infrared images for training, which are easy to collect. Experimental results show that our method can generate high-quality infrared images from visible images and outperforms state-of-the-art methods.

Road landmark inventory is becoming an important data product for the maintenance of transport infrastructures. Several commercial sensors are available which include synchronized optical cameras that allowto build 360° panoramic images of the surroundings of the vehicle used for road inspection. This paper is devoted to the analysis of such panorama images,specifically the area that contains themost relevant information. Road lane landmark detection is posed as a two class classification problem that may be solved bymachine learningapproaches, such as Random Forest (RF) and ensembles of Extreme Learning Machines (V-ELM). Besides model parameter selection, a central problem is the construction of a labeled training and validation datasetto cope with the highly uncontrolled conditions of image capture. Besides, human labor cost makes image data labeling a very expensive process. This paper proposes an open ended Active Learning (AL) approach involving a human oraclein the loop who provides the data labeling and can trigger the AL process when detection quality is degraded by the change in imaging conditions. The paper reports encouraging results over a collection of sample images selected from an industrial road landmark inventory operation. As an additional contribution, this paper assesses the ability of AL to overcomesome of the issues raised by highly class imbalanced datasets.

Multi-view clustering has gained soar attention in the field of data mining in recent years. Most multi-view clustering methods are considered to be complete for each view. However, in many applications, some views may contain missing instances, resulting in the incomplete multi-view learning problem. Incomplete views cannot be directly dealt with traditional multi-view clustering methods. In this paper, we propose a One-step multi-view subspace clustering with incomplete views (OMVSC-IV) method based on low-rank matrix factorization to overcome this problem. Specifically, the proposed algorithm uses low-rank matrix factorization to learn a consensus representation matrix, and then combined with the objective function of non-negative embedding and spectral embedding subspace clustering proposed in this paper. The whole process is established into the same objective function for joint optimization and requires no post-processing (e.g., K-means), thus avoiding the defect of being sensitive to initial values. Furthermore, the accuracy, parameter, sensitivity and convergence of the algorithm are studied by experiments. Experimental results on nine data sets show that the proposed algorithm performs superior to some of the latest algorithms.

Depth estimation is a classic task in computer vision, which is of great significance for many applications such as augmented reality, target tracking and autonomous driving. Traditional monocular depth estimation methods are based on depth cues for depth prediction with strict requirements, e.g. shape-from-focus/ defocus methods require low depth of field on the scenes and images. Recently, a large body of deep learning methods have been proposed and has shown great promise in handling the traditional ill-posed problem. This paper aims to review the state-of-the-art development in deep learning-based monocular depth estimation. We give an overview of published papers between 2014 and 2020 in terms of training manners and task types. We firstly summarize the deep learning models for monocular depth estimation. Secondly, we categorize various deep learning-based methods in monocular depth estimation. Thirdly, we introduce the publicly available dataset and the evaluation metrics. And we also analysis the properties of these methods and compare their performance. Finally, we highlight the challenges in order to inform the future research directions.

Recently, due to its widespread applications in public safety, anomaly detection in crowd scenes has become a hot topic. Some deep-learning-based methods attain significant achievements in this field. Nevertheless, most of them suffer from over-fitting to some extent because of scarce data, which are usually abrupt and low-frequency in the real world. To remedy the above problem, this paper firstly develops a synthetic anomaly event generating system, which could simulate typical specific abnormal events. By utilizing this system, a large synthetic, diverse anomaly event dataset is built, which contains 2,149 video sequences. After getting the dataset, a 3D CNN is designed to detect the abnormal types at the video level. However, we find that there are obvious domain differences (also named as “domain gap/shifts”) between synthetic videos and real-world data, which results in performance degradation when applying the model to the real world. Thus, this paper further proposes a cyclic 3D GAN for domain adaption to reduce the domain gap, which translates the synthetic data to the photorealistic video sequences. Then the detection model is trained on the translated data and it can perform well in the real data. Experimental results illustrate that the proposed method outperforms these baselines for the domain adaptation anomaly detection.

Machine Learning has emerged in the last years as the main solution to many of nowadays’ data-based decision problems. However, while new and more powerful algorithms and the increasing availability of computational resources contributed to a widespread use of Machine Learning, significant challenges still remain. Two of the most significant nowadays are the need to explain a model’s predictions, and the significant costs of training and re-training models, especially with large datasets or in streaming scenarios. In this paper we address both issues by proposing an approach we deem predictive and user-centric. It is predictive in the sense that it estimates the benefit of re-training a model with new data, and it is user-centric in the sense that it implements an explainable interface that produces interpretable explanations that accompany predictions. The former allows to reduce necessary resources (e.g. time, costs) spent on retraining models when no improvements are expected, while the latter allows for human users to have additional information to support decision-making. We validate the proposed approach with a group of public datasets and present a real application scenario.

With intrinsic non-uniform illumination variations in laparoscopic images, such images often suffer from insufficient lighting and low visibility. This in turn may cause incorrect targeting, surgical risk, and extended operating time during laparoscopic surgery. Although various methods for nature image enhancement have been proposed in past decades, surgical laparoscopic image enhancement still needs improving due to issues including naturalness, blurred texture, color cast, and extensive computational time. To address these problems, this paper proposes a distributed Retinex optimization method for laparoscopic image enhancement using refined information fusion. The proposed method minimizes a distributed optimization model with l 2 -norm regularization terms, resulting in a low computational complexity. By integrating the refined image information into the optimization process, our method significantly enhances dark regions while preserving naturalness and texture structures. Experimental results show that our optimization algorithm is more effective than conventional enhancement algorithms in terms of vision augmentation, performance index, and computation time.

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches.

Automatic International Classification of Diseases (ICD) coding is an important task in the future of artificial intelligence healthcare. In recent years, a lot of traditional machine learning-based methods have been proposed, and they achieved good results on this task. However, these traditional machine learning-based methods for automatic ICD coding only focus on the semantic features of clinical notes and ignore the feature extraction of ICD titles that are the descriptions of ICD codes. In this paper, we propose a knowledge attention-based deep learning framework called KAICD for automatic ICD coding. KAICD makes full use of the clinic notes and the ICD titles. The semantic features of clinic notes are extracted by a multi-scale convolutional neural network. For ICD titles, we use attention-based Bidirectional Gated Recurrent Unit (Bi-GRU) to build a knowledge database, which can offer additional information. Depending on input clinic notes, we can use the attention mechanism to obtain different knowledge vectors from the knowledge database where some ICD titles are more relevant to the input clinic notes. Last, we concatenate the knowledge vectors and the semantic features of clinic notes, and use them for the final prediction. KAICD is tested on a public dataset Medical Information Mart for Intensive Care III (MIMIC III); it achieves micro-precision of 0.502, micro-recall of 0.428, and micro-f1 of 0.462, which outperforms other competing methods. Furthermore, the results of the ablation study show that the knowledge database of ICD titles learned by the attention-based Bi-GRU enhances the feature expression and improves the prediction performance.

Methods based on Graph Convolutional Networks (GCN) for skeleton-based action recognition have achieved great success due to their ability to exploit graph structural information from skeleton data. Recently, the bone information has attracted considerable attention as an effective modality which complements the more conventional joint information for action recognition. However, most existing GCN-based methods extract the joint and bone features with two separate GCN networks, ignoring the dependencies between them. In this paper, a novel GCN model is proposed to exploit the information across joints, bones and their relationship collaboratively on a single undirected graph instead of two separate networks. We call the proposed model Vertex-Edge Graph Convolutional Network (VE-GCN) since it conducts the graph convolution operation on the sampling area containing the designated vertexes from joints and edges from bones, respectively. In addition to conducting the Vertex-Edge graph convolution based on the physical connections of the skeleton, we further apply the Vertex-Edge graph convolution to the non-physical joint-joint and joint-bone connections to capture the distal dependencies, and then the convolution results on the non-physical connections are incorporated into the VE-GCN. Moreover, the Conditional Random Field (CRF) is adopted as the loss function to achieve the task of action recognition. Experimental results on four challenging benchmarks (NTU RGB+D, NTU RGB+D 120, N-UCLA, SYSU) show that the proposed model achieves state-of-the-art performance.

Fraud detection is essential for e-commerce platforms to maintain a fair business environment. Many existing works propose manually designed methods such as label propagation and dense block mining rules on built user-item graphs to detect fraud behaviours, but they are always heuristic and thus have limited performance. Other learning-based methods can either handle only the fraud detection problem well in the transductive scenario when there is only structural information or require rich content features to obtain a good inductive ability. Considering that content features are not always available in practice and there are usually many fraudulent behaviours that belong to newly emerging users and items, how to learn effective inductive rules with structures only is still underexplored. In this paper, we propose a subgraph-based method named SubGNN for collective fraud detection. In SubGNN, first, we extract the subgraphs around the given edges (user behaviours) to be tested. Then, we remove nodes’ global IDs so that SubGNN is entity-independent. Finally, by learning knowledge reasoning rules on extracted heterogeneous subgraphs using our proposed relational graph isomorphism network (R-GIN), a powerful graph neural network (GNN) model, SubGNN can achieve precise fraud detection. Experiments are conducted on publicly available Amazon and Yelp datasets and a newly collected Taobao dataset. The results clearly show the advantages and prospects of our method. When using SubGNN to detect fraudulent transactions on Taobao, the precision is higher than 0.99 and more than 90 % of fraud samples are recalled.

Rain removal from a single image is a challenging and significant task of image pre-processing. In this paper, we learn the multi-scale streaks from rainy images using feature pyramid, and to improve the effectiveness of the learning, we focus on the feature propagation re-usage and propagation in the extremely deep de-raining network. Specifically, we design a de-raining2 unit and propose a novel deep de-raining network, respectively, called Pyramid Fully Residual Unit and Network (PFR-Unit and PFR-Net). The PFR-Unit employs fully residual learning in each level of feature pyramid and the PFR-Net connects PFR-Units by a compact dense architecture. The fully residual learning encourages the feature re-usage in PFR-Unit by performing identity mapping for all available shortcuts. The compact dense connection strengthens the feature propagation between the PFR-Units and ensures the unicity of the learning space for the PFR-Units. Along with negative SSIM loss, the PFR-Net presents a good performance in single image de-raining. Comprehensive experimental results show that the PFR-Net outperforms the state-of-the-art single de-raining methods with a big margin on Rain100H, Rain100L and Rain1200 datasets.

This paper focuses on the recognition problem of dynamical patterns consisting of sampling sequences. Specifically, based on the concept of structural stability, a novel similarity measure for dynamical patterns is first given. Then, a specific realization is provided, which consists of: (1) an approximation scheme for computation of partial derivative information by utilizing the knowledge learned through deterministic learning; (2) a similarity comparison scheme using the recognition errors generated from the discrete-time dynamical estimators; and (3) performance analysis of the recognition scheme with general recognition conditions. Compared with the existing methods, in which misrecognition may occur when the differences of dynamics between adjacent training patterns are very small, the proposed method is more appealing in the sense that, the partial derivatives of dynamics are introduced to complement the similarity measures, such that the recognition performance is much improved. Simulation studies are conducted to verify the proposed method in a relatively large data set.

Video dehazing is of crucial importance to the outdoor surveillance, and its main challenges mainly come from the spatial-temporal coherence and computational efficiency. This paper presents an efficient real-time video dehazing approach via incremental transmission learning and spatial-temporally coherent regularization, while explicitly suppressing the possible visual artifacts. First, we propose to initialize the transmission map frame-by-frame by a boundary constrained open dark channel model. Then, supposing a scene point yields highly correlated transmission values between adjacent frames, we impose a temporally coherent term to maintain the temporal consistency of consecutive transmission values, and simultaneously derive an incremental transmission adjusting term to adapt the abrupt scene depth changes between the adjacent frames. Accordingly, the highly correlated and spatial-temporally coherent transmission maps can be optimized for each video frame dehazing, whereby the flickering artifacts can be well reduced. Finally, we group each haze/haze-free images in pair and further utilize the guided joint bilateral filter to suppress the visual artifacts that possibly amplified during the dehazing process. The experimental results show that the proposed video dehazing approach is able to well preserve the spatial-temporal coherence, runs sufficiently fast, and also performs favorably compared to the state-of-the-art methods.

Metallic pieces are typically machined by continuous tool passes, which usually causes regular patterns in the form of straight edges in the surface of the pieces. An irregular pattern on the surface of the piece implies a decrease of the quality of the machined piece. In this paper, we propose an acquisition system and a machine-vision based method to describe the texture of the inner and outer surfaces of machined pieces with cylindrical holes. In order to capture images of the hole surface, we used a microscope camera connected to a rigid industrial boroscope. Considering the extracted texture descriptors, a significant correlation is shown. Consequently, the feature vector is reduced and then classified by several algorithms using an exhaustive grid search strategy with 10-fold cross validation. Best results are achieved with the Extremely Randomized Trees classifier with a mean test score on the hold out set of 92.98%, what improves previous research and meets the requirements of the field.

Face recognition systems are widely used for target recognition and identity authentication, such as automated teller machines, mobile phones, and entrance guard systems. However, face recognition systems are vulnerable to presentation attacks, such as photo, replay, and 3D mask attacks. In particular, silicone mask attacks pose a greater threat to face recognition systems because high-quality silicone masks do living properties. To promote the development of face anti-spoofing detection algorithms for silicone mask attacks, this paper constructs a Silicone Mask Face Motion Video Dataset (SMFMVD) containing 200 real face videos and 200 silicone mask face videos. These videos include different facial motions collected from 20 subjects. Moreover, inspired by the observation that the silicone mask face’s facial movement is not so natural as the real face, we propose a novel silicone mask face anti-spoofing detection method based on visual saliency and facial motion characteristics. Specifically, we compute the visual saliency map of a given face image by simulating two kinds of eye movement behaviors, namely “gaze” and “saccade”. Then, we propose a saliency-weighted histogram of local binary pattern operator to extract facial texture features in spatial domain and a saliency-guided histogram of oriented optical flow operator to extract facial motion features in temporal domain. Finally, the support vector machine is used to fuse two groups of facial features to distinguish real and spoof faces. Extensive experiments on public and self-built datasets show its superiority over the state-of-the-art methods.

Illumination normalization is one of the most challenging issues for facial analysis. To be specific, the variation of environmental illumination influences the visual quality of an image and drastically degrades the performance of face recognition, detection, landmark and other related applications. Retinex theory provides an important concept for processing illumination variation, which supposes that a face image can be decomposed into [an invariant reflectance component and a variant illumination component. Enlighten by this theory, in this paper, we put forward a novel deep learning approach which combines self-supervised learning and adversarial training for face illumination normalization (FIN-GAN). The proposed FIN-GAN framework can be implemented by two steps. Firstly, self-supervised learning is employed to decompose the original face image into the illumination component and the illumination-invariant component with Retinex constraint. Then, we employ the conditional generative adversarial network for face image reconstruction. For network optimization, we design the combined loss to ensure visual quality and preserve identity information. Experiments are performed on Extended-YaleB, CAS-PEAL, CMU-PIE and Multi-PIE datasets. Through multiple quantitative criteria, we demonstrate that the proposed FIN-GAN obtains promising performance in face illumination normalization.

The outbreak and rapid spread of coronavirus disease 2019 (COVID-19) has had a huge impact on the lives and safety of people around the world. Chest CT is considered an effective tool for the diagnosis and follow-up of COVID-19. For faster examination, automatic COVID-19 diagnostic techniques using deep learning on CT images have received increasing attention. However, the number and category of existing datasets for COVID-19 diagnosis that can be used for training are limited, and the number of initial COVID-19 samples is much smaller than the normal’s, which leads to the problem of class imbalance. It makes the classification algorithms difficult to learn the discriminative boundaries since the data of some classes are rich while others are scarce. Therefore, training robust deep neural networks with imbalanced data is a fundamental challenging but important task in the diagnosis of COVID-19. In this paper, we create a challenging clinical dataset (named COVID19-Diag) with category diversity and propose a novel imbalanced data classification method using deep supervised learning with a self-adaptive auxiliary loss (DSN-SAAL) for COVID-19 diagnosis. The loss function considers both the effects of data overlap between CT slices and possible noisy labels in clinical datasets on a multi-scale, deep supervised network framework by integrating the effective number of samples and a weighting regularization item. The learning process jointly and automatically optimizes all parameters over the deep supervised network, making our model generally applicable to a wide range of datasets. Extensive experiments are conducted on COVID19-Diag and three public COVID-19 diagnosis datasets. The results show that our DSN-SAAL outperforms the state-of-the-art methods and is effective for the diagnosis of COVID-19 in varying degrees of data imbalance.

Aspect level sentiment classification aims to extract fine-grained sentiment expressed towards specific aspects from a sentence. The key to this task lies in connecting aspects and their respective sentiment contexts. Existing methods measure the dependency weights between aspects and context words via either the semantic similarity between words captured by attention mechanism or the structural proximity between words in syntactic structures. However, methods in both groups fail to fully exploit explicit syntactic dependency, which we argue should be critical to identify sentiment contexts. In this paper, we propose a novel syntactic-dependency-based attention network (SDATT) to incorporate explicit syntactic dependency for aspect level sentiment classification. SDATT first models the dependency path between each word and the aspect to characterize aspect-oriented syntactic representation of each word. The generated syntactic representations are later fed into the attention layer to help infer the dependency weights for sentiment prediction. Experimental results on five benchmark datasets show the superior performance of the proposed model over state-of-the-art baselines.

Light Field (LF) images have the unique advantage of recording scenes from multiple viewpoints, which provides many applications, such as refocusing and depth estimation. However, low-light conditions can severely influence these applications. In this paper, we propose a two-stage deep learning framework for the LF restoration under low-light imaging. First, there is a multi-to-one (MTO) network, which restores each view separately by utilizing multiple auxiliary views. All the views share the same feature extractor, with an efficient spatial-channel attention mechanism to extract more informative features. A channel-attention feature fusion (CAFF) module is designed to selectively fuse more useful complementary information from the auxiliary views, with a learnable global scalar to adjust the importance of the auxiliary features. Then, the outputs of the MTO network are further enhanced by an (all-to-all) ATA network, which uses spatial and angular residual blocks to process all the views synchronously for fully encoding the spatial-angular information. Extensive experiments have been conducted to demonstrate the superior performance and robustness of our method, i.e., it can restore the luminance, spatial details and angular geometries of the LF images under various light levels effectively.

In recent years, deep learning techniques have significantly improved the performance of single image super-resolution (SISR). However, this improvement is often achieved at the cost of introducing a large amount of parameters, which limits the real-world applications for SISR. In this paper, we propose a lightweight SISR network called Multi-scale Channel Attention Network for Image Super-Resolution (MCSN). Our contributions are threefold. First of all, the multi-scale feature fusion block (MSFFB) can extract multi-scale features by filters with different receptive fields. Secondly, the channel shuffle attention mechanism (CSAM) encourages the flow of the information across feature channels and enhances the ability of feature selection. Thirdly, the global feature fusion connection (GFFC) can effectively improve feature utilization. Extensive experiments demonstrate that the parameter amount of our method is reduced by 3/4 compared with the current state-of-the-art MSRN method, while both the subjective visual effect and objective quality of the reconstructed high-resolution images are significantly better.

The ever-increasing complexity of robots causes failures of them as a side effect. Successful detection of anomalies in robotic systems is a key issue in order to improve their maintenance and consequently reducing economic costs and downtime. Going one step further in the detection of anomalies in robots, different mechanisms to deal with data irregularities are proposed and validated in present paper in order to increase detection rates. More precisely, strategies to overcome missing values and class imbalance are considered as complementary tools to get better one-class classification results. The effect of such strategies is evaluated through cross-validation when applying a standard supervised learning model, the Support Vector Machine. Experiments are run on an up-to-date and public dataset that contains some examples of different software anomalies that the middleware of the robot under analysis may experience.

Interpersonal relation plays an essential role to gain understandings on how people interact with each other. In computer vision, interpersonal relations provide vital information to interpret people’s behaviors. However, the existing research has either omitted the interaction information between subjects or the structural information in the images. In this paper, we propose a new architecture to reason interpersonal relations based on higher-order graph networks and multi-scale features. First, we extract features of the whole images, the facial features, and the union region of face pairs. Apart from the pixel-wise features, we also consider the positional features of face-to-face pairs and the spatial scene cues. Higher-order Graph Neural Networks (GNNs) were employed to map out the interpersonal relations based on the feature extracted. Experimental results show that the proposed Higher-order Graph Neural Networks with multi-scale features can effectively recognize the social relations in images with over 5% improvement in absolute balanced accuracy compared with the state-of-the-art work.

The graph model is a reliable propagation mechanism in saliency detection, saliency value propagation diffusion results between any two nodes are determined merely by defining an effect affinity matrix. Most existing methods generally calculamatrix using mean values of single or multiple feature vectors, not fully exploit the diversity and consistency of multi-view features, may produce poor foreground uniformity and completeness in the complex scene. Multi-views should share an affinity matrix as well as complement each other. In this paper, we propose a graph-based saliency detection with a learning joint affinity matrix. First, we capture multiple appearance features from the image and generate a learning joint affinity matrix based on low-rank representation. Then, for computing an effect affinity matrix, we linearly integrate the traditional affinity matrix and learning joint affinity matrix, helping to construct an affinity graph for diffusion-based compactness. Finally, to effectively optimize the initial saliency map, we diffuse the learning joint affinity matrix and traditional impact factor matrix via cross-view diffusion processing, which begets an approving advantage for single-layer cellular automata. Results on three benchmark datasets demonstrate that our proposed method shows the best performance against nine state-of-the-art models.

Recently, many scenarios, such as affective disorders treatment, have sparked rising needs for establishment of personalized emotion recognition (PER) models. Unfortunately, the data sparsity issue violates the basic i.i.d. assumption of supervised learning (i.e., training data and test data are independently and identically distributed). In this paper, we present a semi-supervised joint domain adaption (SSJDA) solution, aiming to inject the hidden domain knowledge from ample labeled data of multiple source individuals into the target subject’s customized model. Specifically, we put forward a novel Progressive Low-Rank Subspace Alignment (PLRSA) approach, which unifies a semi-supervised instance-transfer paradigm and an unsupervised mapping-transfer learning paradigm in a single optimization framework. We leverage the boosting-based TrAdaBoost algorithm and the Transfer Component Analysis (TCA) algorithm for the implementation of instance reweighting and feature matching, respectively. Then we introduce the l 2 , 1 - norm to pass feedback and make the joint learning feasible. The central idea is to progressively minimize the cross-domain distribution discrepancies to finally construct the optimal domain-invariant features. We systematically compare the PLRSA method with five state-of-the-art techniques using two public EEG datasets (DEAP and SEED). Both many-to-one and one-to-one evaluations are performed. The experimental results have confirmed the efficacy of the proposed method.

Current annotation for plant disease images depends on manual sorting and handcrafted features by agricultural experts, which is time-consuming and labour-intensive. In this paper, we propose a self-supervised clustering framework for grouping plant disease images based on the vulnerability of Kernel K-means. The main idea is to establish a cross iterative under-clustering algorithm based on Kernel K-means to produce the pseudo-labeled training set and a chaotic cluster to be further classified by a deep learning module. In order to verify the effectiveness of our proposed framework, we conduct extensive experiments on three different plant disease datatsets with five plants and 17 plant diseases. The experimental results show the high superiority of our method to do image-based plant disease classification over balanced and unbalanced datasets by comparing with five state-of-the-art existing works in terms of different metrics.

The recent years have witnessed significant advances in deep video action recognition. However, the performances of deep learning-based video action recognition methods are limited in the case of relatively small samples or no labeled samples. Therefore, trying to use unlabeled video data to generate clustering label is essential for small sample learning and zero sample learning. In this paper, we propose a novel deep video action clustering network, which aims to learn the similarity relationship among the unlabeled video samples, and generate the clustering label for each video sample. Specifically, the proposed method simultaneously learns the spatio-temporal features and subspace representations under a jointly optimized framework. It consists of a 3D U-Net self-representation generator, a video-clip reconstruction discriminator, and a confidence-based feedback mechanism. The 3D U-Net self-representation generator learns the spatio-temporal features of the video clips and produces subspace representation matrix. Then, the similarity graph is constructed based on this subspace representation matrix, and the clustering result is obtained. In the learning procedure, a confidence-based feedback mechanism is designed to feed the high-confidence labels of partial samples back to further guide the subspace structure learning, so that the optimal result can be obtained. During training, the video-clip reconstruction discriminator is introduced to evaluate the reconstructed video clips, which is beneficial for capturing the discriminative spatio-temporal features. Experimental results on a video benchmark dataset demonstrate the effectiveness of the proposed method.

Interpretability is a significant aspect of the distributed word representation learning model. Although the most advanced pretrained models have achieved the best results till date, the interpretability of a pretrained model is difficult to explain clearly. For this reason, based on the interpretability of distributed word embeddings, this paper presents a method of learning word representation using joint context. At present, the existing distributed word representation models for learning word representations usually focus on either neighbor or syntactic context. We argue that it is necessary to simultaneously model both contexts. In particular, the point mutual information obtained by combining the two types of contexts can efficiently express the correlation between the words. We propose two alternative distribution models for learning word representations by employing the neighbor and syntactic contexts via a simple and effective joint learning framework. Furthermore, the proposed models are trained on a public corpus, and the learned representations are evaluated in word analogy, word similarity, and sentence classification tasks. The experimental results demonstrate the potential of the proposed method.

Deep convolutional neural networks have shown great potential in image recognition tasks. However, the fact that the mechanism of deep learning is difficult to explain hinders its development. It involves a large amount of parameter learning, which results in high computational complexity. Moreover, deep convolutional neural networks are often limited by overfitting in regimes in which the number of training samples is limited. Conversely, kernel learning methods have a clear mathematical theory, fewer parameters, and can contend with small sample sizes; however, they are not able to handle high-dimensional data, e.g., images. It is important to achieve a performance and complexity trade-off in complicated tasks. In this paper, we propose a novel scalable deep convolutional random kernel learning in Gaussian process architecture called SDCRKL-GP, which is characterized by excellent performance and low complexity. First, we successfully incorporated the deep convolutional architecture into kernel learning by implementing the random Fourier feature transform for Gaussian processes, which can effectively capture hierarchical and local image-level features. This approach enabled the kernel method to effectively handle image processing problems. Second, we optimized the parameters of deep convolutional filters and Gaussian kernels by stochastic variational inference. Then, we derived the lower variational bound of the marginal likelihood. Finally, we explored the model architecture design space selection method to determine the appropriate network architecture for different datasets. The design space consists of the number of layers, the channels per layer, and so on. Different design space selections improved the scalability of the SDCRKL-GP architecture. We evaluated SDCRKL-GP on the MNIST, FMNIST, CIFAR10, and CALTECH4 benchmark datasets. Taking MNIST as an example, the error rate of classification is 0.60%, and the number of parameters, number of computations and memory access cost of the architecture are 19.088k, 0.984M, and 1.057M, respectively. The experimental results verified that the proposed SDCRKL-GP method outperforms several state-of-the-art algorithms in both accuracy and speed in image recognition tasks. The code is available at https://github.com/w-tingting/deep-rff-pytorch.

Today, artificial intelligence is everywhere in people’s daily lives. Visual tracking, which is used to identify and continuously track specific targets, is an important research domain in the study of artificial intelligence. However, current visual tracking methods are not accurate enough for object tracking with background clutter, which can easily lead to tracking failures. Therefore, in this paper, in order to solve the problem of tracking failure in clutter background, we propose a template update mechanism to improve the accuracy of visual tracking. First, an original template is saved when the background clutter is detected. During background clutter, we use both the original template and the current template at the location estimated by the optical flow and choose better one. Next, the original template is reused after the background clutter is ended. Finally, the proposed mechanism is used both in the KCF and BACF algorithm to verify the effectiveness of the mechanism. With experiments on the OTB2015 dataset, results show that the proposed mechanism has improved accuracy and success rate of the two baseline algorithms. Meanwhile, in state-of-the-art algorithms, the algorithm using the proposed mechanism also has excellent tracking performance. In addition, this method also has strong tracking robustness and adaptation capability to sequential learning for video data.

Magnetic Resonance Imaging (MRI) has been widely used in clinical application and pathology research to help doctors provide better diagnoses. However, accurate diagnosis by MRI remains a great challenge, as images obtained via current MRI techniques usually have low resolutions. Improving MRI image quality and resolution has thus become a critically important task. This paper presents an innovative Coupled-Projection Residual Network (CPRN) for MRI super-resolution. CPRN consists of two complementary sub-networks: a shallow network and a deep one, which maintain content consistency while learning high frequency differences between low-resolution and high-resolution images. The shallow sub-network employs coupled-projection to better retain the MR image details, where a novel feedback mechanism is introduced to guide the reconstruction of high-resolution images. The deep sub-network learns from the residuals of the high-frequency image information, where multiple residual blocks are cascaded to magnify the MR images at the last network layer. Finally, the features from the shallow and deep sub-networks are fused for the reconstruction of high-resolution MR images. For effective feature fusion between the deep and shallow sub-networks, a step-wise connection (CPRN_S) is designed, inspired by the human cognitive process (from simple to complex). Experiments over three public MRI datasets show that our proposed CPRN achieves superior MRI super-resolution performance compared with the state-of-the-art.

Moire patterns, mainly due to the aliasing between the grids of display devices and camera sensors, often occur in recaptured screen images (videos). The spatial variety in densities and scales makes it difficult to remove moire patterns with hand-crafted priors. Prevalent supervised-learning based methods require huge amount of paired images, which are difficult to capture and align. This paper proposes an unsupervised Generative Adversarial Network for Moire Removal (MR-GAN), which is the first attempt at unsupervised learning based moire removal. The recaptured images are corrected against vignetting artifacts to make MR-GAN focus on correcting brightness and contrast distortions while removing moire patterns. We propose two complementary discriminator groups to effectively distinguish moire patterns and image features at both large scales and small scales. A group of self-supervised loss functions, including cycle consistent loss, identity loss, cosine similarity loss, and content leakage loss, are designed to train effective generators. Experimental results demonstrate that our method outperforms state-of-the-art demoireing methods on a large set of test images. Our code and dataset are released in the link: https://github.com/JerryLeolfl/pytorch-MRGAN-master .

Relation extraction has been an active research interest in the field of Natural Language Processing (NLP). The past works primarily focused on a corpus of formal text which is inherently non-dialogic. Recently, the dialogue-based relation extraction task, which detects relations among speaker-aware entities scattering in dialogues, has been gradually arousing people’s attention. Some sequence-based neural methods have been carried out to obtain the relevant information. However, identifying cross-sentence relations remains unsolved, especially in the context of a specific-domain dialogue system. In this paper, we propose a Relational Attention Enhanced Graph Convolutional Network (RAEGCN), which constructs the whole dialogue as a semantic interactive graph by emphasizing the speaker-related information and leveraging various inter-sentence dependencies. A dense connectivity mechanism is also introduced to empower the multi-hop relational reasoning across sentences, which can capture both local and non-local features simultaneously. Experiments show the significant superiority and robustness of our model on a real-world dataset DialogRE, as compared with previous approaches.

The most existing deep neural networks (DNN)-based methods for fault diagnosis only focus on prediction accuracy without considering the limitation of labeled sample size. In practical applications of DNN-based methods, it is time-consuming and costly to collect massive labeled samples. In this paper a task named few-shot fault diagnosis is defined as training model given small labeled samples in source domain and testing given small samples in target domain. We develop a novel intelligent fault diagnosis model for few-shot fault diagnosis which is using similarities of sample pairs to classify samples, rather than end-to-end classification. The proposed model contains modules of feature learning and metric learning. The module of feature learning has twin neural networks aiming to extract features from the sample pair. The module of metric learning is to predict similarity of the sample pair. The similarities of sample pairs combined the test sample with each labelled sample are utilized to complete the classification task. Label smoothing is utilized to further improve performance of classification. The performance of the proposed model is verified by two fault diagnosis cases which are bearing fault diagnosis cross different working conditions and cross bearing locations. The comparison studies with other models demonstrate the superiority of the proposed model.

Taking advantage of labeled auxiliary training data whose distribution is similar to the distribution of the gallery, single sample face recognition (SSFR) has achieved encouraging performance. However, in many real-world applications, it is difficult to collect such an auxiliary training dataset, while it may be easier to collect an unlabeled target training dataset whose distribution is similar to the distribution of the gallery and a labeled source training dataset whose distribution may be different to the distribution of the gallery. How can these three datasets be effectively leveraged to handle SSFR? To address this issue, this paper proposes a new method of G allery- S ensitive Single Sample Face Recognition based on D omain A daptation (GS-DA). First, GS-DA employs the method of TSD (targetize the source domain) to construct a common subspace and a targetized source domain. Secondly, it projects each gallery image into the common subspace and obtains the sparse representation of each gallery image in the common subspace. Thirdly, it reconstructs each gallery image from the targetized source domain to estimate the within-class scatter matrix and the between-class scatter matrix of the gallery. Lastly, it learns a discriminant model by maximizing the sum of the traces of the between-class scatter matrix of the gallery and the between-class scatter matrix of the targetized source domain as well as minimizing the sum of the traces of the total scatter matrix of the gallery and the total scatter matrix of the target training data. The experimental results on five datasets illustrate the superiority of GS-DA in leveraging these three datasets for SSFR.

Spoken language understanding (SLU) plays an indispensable role in the dialogue system. The traditional intention detection task is regarded as a classification problem where utterances are associated with pre-defined intents. However, the various expressions of user's intents and constantly emerging novel intents make the annotating time-consuming and labor-intensive, building massive obstacles for extending the model to new tasks. Identifying unexpected user intention and achieving the user's desire goal is a challenging task. Therefore, we conduct zero-shot intention detection based on a transformation-based learning manner. In this paper, we propose an intent-enhanced attentive capsule network (IE-BertCapsNet) further guides the aggregation process of the capsule network and generalizable useful features that can be adapted to emerging intentions. Coupling with the large margin cosine loss function, the proposed model can identify discriminative features by forcing the whole network to minimize inter-class distance and minimize intra-class distance. Finally, we leverage the IE-BertCapsNet’s feature extraction ability and knowledge transferring capability to conduct zero-shot intent detection and generalized zero-shot intent detection. Extensive experiments on five benchmark task-oriented datasets in four languages demonstrate that the proposed model can achieve competitive performance that can better discriminate known intents and detect unknown intents.

Recently, many methods enlarged the kernel size or fused multi-scale features to capture contextual information for parsing the hard samples, such as heavy occlusions, limb absence, and complex poses. Nevertheless, those required higher computational and cannot explore the effective global context to achieve more precise parsing results. In this paper, we propose an end-to-end network for human parsing, named Context Prior based Semantic-Spatial Graph Network (CP-SSGNet), which achieves higher precision and consistency by encoding the context prior in the graph model. CP-SSGNet consists of three modules, Semantic Constraint Module (SCM), Spatial Perceiving Module (SPM), and Intra-class Attention Module (IAM). SCM captures richer global dependencies by encoding semantic structure context prior in a semantic graph, which can reduce semantic structure errors. SPM learns enhanced local features by encoding the spatial consistency context prior in the spatial graph, which can optimize boundaries and reduce the local consistency errors. IAM utilizes the spatial graph with strong–weak connections for intra-class features aggregation to distinguish the inter-class features clearly. Extensive experiments are conducted on two challenging datasets, PASCAL-Person-Part and LIP, effectively achieving state-of-the-arts performance.

Link prediction aims to predict the missing interactions in evolving networks that may appear in the future. It has practical importance in various real-world applications, ranging from friendship recommendation, knowledge graph completion, target advertising, and protein–protein interaction prediction. Most of the recent efforts focus on the structure of the network while ignoring many other essential factors. In this paper, we present a modified Latent Dirichlet Allocation (LDA), and Hidden Naive Bayesian (HNB) based link prediction technique named PILHNB model for link prediction in dynamic social networks by considering behavioral controlling elements like relationship network structure, nodes’ attributes, location-based information of nodes, nodes’ popularity, users’ interests, and learning the evolution pattern of these factors in the networks. Experimental results on six real-world networks demonstrate our proposed models’ effectiveness and efficiency compared with existing state-of-the-art link prediction techniques.