Search

Steel surface defects have a significant impact on the quality and performance of many industrial products and cause huge economic losses. Therefore, it is meaningful to detect steel surface defects in real time. To improve the detection performance of steel surface defects with variable scales and complex backgrounds, in this paper, a novel method for detecting steel surface defects through a multiscale local and global feature fusion mechanism is proposed. The proposed method uses a convolution operation with a downsampling mechanism in the convolutional neural network model to obtain rough multiscale feature maps. Then, a context-extraction block (CEB) is proposed to adopt self-attention learning on the feature maps extracted by the convolution operation at each scale to obtain multiscale global context information to make up for the shortcomings of convolutional neural networks (CNNs), thus forming a novel multiscale self-attention mechanism. Afterwards, using the feature pyramid structure, multiscale feature maps are fused to improve multiscale object detection. Finally, the channel and spatial attention module and the WIOU (Wise Intersection over Union) loss function are introduced. The model achieved 78.2% and 71.9% mAP respectively on the NEU-DET and GC10-DET dataset. Compared to algorithms such as Faster RCNN and EDDN, this method is effective in improving the detection performance of steel surface defects. [ABSTRACT FROM AUTHOR]

Treating patients with complex diseases or co-existing conditions by polypharmacy (i.e., the use of drug combination) is very common. However, due to drug-drug interactions, polypharmacy often results in unpredictable side effects, which may endanger patients' life. Moreover, since adverse drug reactions are rare, discovering polypharmacy side effects only from sparse drug-drug interactions remains challenging. Thus, it is necessary to explore the knowledge of polypharmacy side effects from the interaction network with complex relationships. In this paper, we propose a novel multi-modal graph convolutional neural network (M2GCN) for link prediction in multi-modal networks which consist of protein-protein interactions, drug-protein interactions and drug-drug interactions. Specifically, we first propose a propagation strategy to perform graph aggregations on each subgraph. Then we leverage consistency regularization to align the consistency across different subgraphs. Finally, referring to the DistMult method, we use the embeddings obtained above to calculate the probability of side effects between drugs. Experimental results on benchmark dataset show that our method significantly outperforms the compared network embedding models. [ABSTRACT FROM AUTHOR]

Notwithstanding the great progress on deep convolutional neural networks (CNNs) has been made during last decade, the representation ability may still be restricted and it usually needs more epochs to converge in training, due to the information loss caused by the up-/down-sampling operations. In this paper, we propose a general deep feature recovery layer, termed Low-rank Deep Feature Recovery (LDFR), to enhance the representation of convolutional features by seamlessly integrating the low-rank recovery into CNNs, which can be easily extended to all CNNs-based models. To be specific, to recover the lost useful information, LDFR learns the low-rank projections to embed feature maps onto a low-rank subspace based on the selected informative convolutional feature maps. Such operation can ensure all the convolutional feature maps to be reconstructed easily to recover the underlying subspace, with more useful detailed information discovered, e.g., the strokes of characters or the texture information of clothes. To make the learnt low-rank subspaces more powerful for feature recovery, we design a fusion strategy to obtain a generalized subspace, which averages over all learnt subspaces in each LDFR layer, so that the convolutional features in test phase can be recovered effectively via low-rank embedding. We also present a fast version of LDFR, called FLDFR, to speedup the optimization of LDFR by flattening all feature maps of batch images to recover the lost information. Extensive simulations on several image datasets show that the existing CNN models equipped with our LDFR layers can obtain better performance. [ABSTRACT FROM AUTHOR]

This paper proposes a new graph convolutional operator called central difference graph convolution (CDGC) for skeleton based action recognition. It is not only able to aggregate node information like a vanilla graph convolutional operation but also gradient information. Without introducing any additional parameters, CDGC can replace vanilla graph convolution in any existing Graph Convolutional Networks (GCNs). In addition, an accelerated version of the CDGC is developed which greatly improves the speed of training. Experiments on two popular large-scale datasets NTU RGB+D 60 & 120 have demonstrated the efficacy of the proposed CDGC. Code is available at https://github.com/iesymiao/CD-GCN. [ABSTRACT FROM AUTHOR]

Trajectory prediction is a fundamental and challenging task for numerous applications, such as autonomous driving and intelligent robots. Current works typically treat pedestrian trajectories as a series of 2D point coordinates. However, in real scenarios, the trajectory often exhibits randomness, and has its own probability distribution. Inspired by this observation and other movement characteristics of pedestrians, we propose a simple and intuitive movement description called a trajectory distribution, which maps the coordinates of the pedestrian trajectory to a 2D Gaussian distribution in space. Based on this novel description, we develop a new trajectory prediction method, which we call the social probability method. The method combines trajectory distributions and powerful convolutional recurrent neural networks. Both the input and output of our method are trajectory distributions, which provide the recurrent neural network with sufficient spatial and random information about moving pedestrians. Furthermore, the social probability method extracts spatio-temporal features directly from the new movement description to generate robust and accurate predictions. Experiments on public benchmark datasets show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Multi-task learning in Convolutional Neural Networks (CNNs) has led to remarkable success in a variety of applications of computer vision. Towards effective multi-task CNN architectures, recent studies automatically learn the optimal combinations of task-specific features at single network layers. However, they generally learn an unchanged operation of feature combination after training, regardless of the characteristic changes of task-specific features across different inputs. In this paper, we propose a novel Adaptive Feature Aggregation (AFA) layer for multi-task CNNs, in which a dynamic aggregation mechanism is designed to allow each task adaptively determines the degree to which the knowledge sharing or preserving between tasks is needed based on the characteristics of inputs. We introduce two types of aggregation modules to the AFA layer, which realize the adaptive feature aggregation by capturing the feature dependencies of different tasks along the channel and spatial axes, respectively. The AFA layer is a plug-and-play component with low parameter and computation overheads, and can be trained end-to-end along with backbone networks. For both pixel-level and image-level tasks, we empirically show that our approach substantially outperforms the previous state-of-the-art methods of multi-task CNNs. The code and models are available at https://github.com/zhenshen-mla/AFANet. [ABSTRACT FROM AUTHOR]

This article proposes a fast and accurate network for surface defect detection, termed SDDNet. SDDNet mainly addresses two challenging issues—large texture variation and small size of defects—by introducing two modules: feature retaining block (FRB) and skip densely connected module (SDCM). FRB fuses multiple pyramidal feature maps with different resolutions and is plugged on the top of pooling layers, aiming to preserve the texture information, which may be lost because of downsampling. SDCM is designed to propagate the fine-grained details from low- to high-level feature maps for better prediction of defects, especially small defects. Extensive experiments conducted on the publicly available data sets NEU-DET (88.8% mAP), DAGM (99.1% mAP), and Magnetic-Tile (93.4% mAP) have demonstrated the effectiveness of the proposed SDDNet and its feasibility for real-time industrial applications. [ABSTRACT FROM AUTHOR]

RGB-D based action recognition is attracting more and more attention in both the research and industrial communities. However, due to the lack of training data, pre-training based methods are popular in this field. This paper presents a review of the concept of dynamic maps for RGB-D based human motion recognition using pretrained models in image domain. The dynamic maps recursively encode the spatial, temporal and structural information contained in the video sequence into dynamic motion images simultaneously. They enable the usage of Convolutional Neural Network and its pretained models on ImageNet for 3D human motion recognition. This simple, compact and effective representation achieves state-of-the-art results on various gesture/action/activities recognition datasets. Based on the review of previous methods using this concept upon different modalities (depth, skeleton or RGB-D data), a novel encoding scheme is developed and presented in this paper. The improved method generates effective flow-guided dynamic maps, and they could select the high motion window and distinguish the order among the frames with small motion. The improved flow-guided dynamic maps achieve state-of-the-art results on the large Chalearn LAP IsoGD and NTU RGB+D datasets. [ABSTRACT FROM AUTHOR]

Most person re-identification (re-ID) approaches are based on supervised learning, which requires manually annotated data. However, it is not only resource-intensive to acquire identity annotation but also impractical for large-scale data. To relieve this problem, we propose a cross-camera unsupervised approach that makes use of unsupervised style-transferred images to jointly optimize a convolutional neural network (CNN) and the relationship among the individual samples for person re-ID. Our algorithm considers two fundamental facts in the re-ID task, i.e., variance across diverse cameras and similarity within the same identity. In this paper, we propose an iterative framework which overcomes the camera variance and achieves across-camera similarity exploration. Specifically, we apply an unsupervised style transfer model to generate style-transferred training images with different camera styles. Then we iteratively exploit the similarity within the same identity from both the original and the style-transferred data. We start with considering each training image as a different class to initialize the Convolutional Neural Network (CNN) model. Then we measure the similarity and gradually group similar samples into one class, which increases similarity within each identity. We also introduce a diversity regularization term in the clustering to balance the cluster distribution. The experimental results demonstrate that our algorithm is not only superior to state-of-the-art unsupervised re-ID approaches, but also performs favorably compared with other competing unsupervised domain adaptation methods (UDA) and semi-supervised learning methods. [ABSTRACT FROM AUTHOR]

Atrial fibrillation is the most common arrhythmia and is associated with high morbidity and mortality from stroke, heart failure, myocardial infarction, and cerebral thrombosis. Effective and rapid detection of atrial fibrillation is critical to reducing morbidity and mortality in patients. Screening atrial fibrillation quickly and efficiently remains a challenging task. In this paper, we propose SS-SWT and SI-CNN: an atrial fibrillation detection framework for the time-frequency ECG signal. First, specific-scale stationary wavelet transform (SS-SWT) is used to decompose a 5-s ECG signal into 8 scales. We select specific scales of coefficients as valid time-frequency features and abandon the other coefficients. The selected coefficients are fed to the scale-independent convolutional neural network (SI-CNN) as a two-dimensional (2D) matrix. In SI-CNN, a convolution kernel specifically for the time-frequency characteristics of ECG signals is designed. During the convolution process, the independence between each scale of coefficient is preserved, and the time domain and the frequency domain characteristics of the ECG signal are effectively extracted, and finally the atrial fibrillation signal is quickly and accurately identified. In this study, experiments are performed using the MIT-BIH AFDB data in 5-s data segments. We achieve 99.03% sensitivity, 99.35% specificity, and 99.23% overall accuracy. The SS-SWT and SI-CNN we propose simplify the feature extraction step, effectively extracts the features of ECG, and reduces the feature redundancy that may be caused by wavelet transform. The results shows that the method can effectively detect atrial fibrillation signals and has potential in clinical application. [ABSTRACT FROM AUTHOR]

Fine-grained recognition is one of the most difficult topics in visual recognition, which aims at distinguishing confusing categories such as bird species within a genus. The information of part and bounding boxes in fine-grained images is very important for improving the performance. However, in real applications, the part and/or bounding box annotations may not exist. This makes fine-grained recognition a challenging problem. In this paper, we propose a jointly trained Convolutional Neural Network (CNN) architecture to solve the fine-grained recognition problem without using part and bounding box information. In this framework, we first detect part candidates by calculating the gradients of feature maps of a trained CNN model w.r.t. the input image and then filter out unnecessary ones by fusing two saliency detection methods. Meanwhile, two groups of global object locations are obtained based on the saliency detection methods and a segmentation method. With the filtered part candidates and approximate object locations as inputs, we construct the CNN architecture with local parts and global discrimination (LG-CNN) which consists of two CNN networks with shared weights. The upper stream of LG-CNN is focused on the part information of the input image, the bottom stream of LG-CNN is focused on the global input image. LG-CNN is jointly trained by two stream loss functions to guide the updating of the shared weights. Experiments on three popular fine-grained datasets well validate the effectiveness of our proposed LG-CNN architecture. Applying our LG-CNN architecture to generic object recognition datasets also yields superior performance over the directly fine-tuned CNN architecture with a large margin. [ABSTRACT FROM AUTHOR]

Many internet users are potential victims of the pornographic images and a large part of them are underage children. Thus, content-based pornographic images detection is an important task in computer vision and multimedia research. Previous solutions usually rely on hand-engineered visual features that are hard to analyze and select. In this paper, to detect pornographic images in any style accurately and efficiently with a single model, a novel scheme utilizing the deep convolutional neural networks (CNN) is proposed. The training data are obtained from internet followed by an improved sliding window method and some novel data augmentation approaches. Then a highly efficient training algorithm is proposed based on two strategies. The first is the pre-trained mid-level representations non-fixed fine-tuning strategy. The second is adjusting the training data at the appropriate time on the basis of the performance of the proposed network on the validation set. Furthermore, we introduce a fast image scanning method which is also based on the sliding window approach in the test. We further propose a fast forward pass method based on the “fixed-point algorithm”. So our CNN could detect all scale images so fast by one forward pass. The effectiveness of the proposed method is demonstrated in experiments on the proposed dataset and the comparative results show that our method lead to state-of-the-art detection performance. [ABSTRACT FROM AUTHOR]

Electrocardiographic (ECG) abnormalities are the most intuitive manifestation in the clinical diagnosis of cardiovascular disease. Although significant progress has been achieved in diagnosing ECG abnormalities, current convolutional neural network (CNN) based methods still show a relatively high rate of misdiagnosis. On the one hand, the ECG data an abnormality imbalance problem. On the other hand, the previous work did not consider the multi-label correlations of the ECG abnormalities. The main objective of this paper is to design an ECG abnormal event detection model based on multi-label correlation guided feature fusion. First, the proposed method calculates the correlation of different ECG abnormality labels on the basis of their frequency and Bayesian conditional probability and divides the ECG abnormalities into groups of different levels. The first-level ECG abnormalities are those that have the highest frequency and the second-level abnormalities are those that have a higher probability of other ECG abnormalities given the first-level ECG abnormalities, and so on. Then, the feature fusion strategy uses the multi-label correlation to guide the fusion of features of different levels. Specifically, the features of the first-level ECG signals are integrated into the features of the second-level ECG signals and the features of the former two-level ECG signals are integrated into the features of the third-level ECG signals. Furthermore, a multi-scale receptive field fusion module is proposed to enhance the multi-scale representation ability of the features. To evaluate the performance of the proposed method, it is validated against the 1st China physiological signal challenge dataset, and it achieved better results compared with other state-of-the-art methods. This proposed method achieves a precision score of 0.816, a sensitivity score of 0.845 and an F1-score of 0.827, which are better than other methods. The comparative analysis shows that the proposed method demonstrates the effectiveness in abnormal ECG diagnosis. [ABSTRACT FROM AUTHOR]