Search

View Transformation Model(VTM) is extensively employed in multi-view gait recognition. However, there still exists decline of matching accuracy among view transformation procedures. Particularly, the loss grows rapidly with the increase of the disparity of views. In the face of this difficulty, firstly, Non-negative Matrix Factorization(NMF) is introduced to obtain local structured features of human body to compensate accuracy loss. Moreover, 2D Linear Discriminant Analysis(2DLDA) is applied to improve classification ability by projecting features into a discriminant space. In the end, gait features, the Gait Energy Images(GEIs), is strengthened as 2D Enhanced GEI(2D-EGEI) by using the reconstruction of 2D Principal Component Analysis(2DPCA). Compared with the state-of-the-art, proposed method significantly outperforms the others. Furthermore, the comparisons of two deep learning methods is evaluated as well. Experimental outcomes show that the proposed method provides an alternative way to obtain the approximative outcomes compared with the deep learning methods.

Gait recognition is a hot topic in the field of biometrics because of its unique advantages such as non-contact and long distance. The appearance-based gait recognition methods usually extract features from the silhouettes of human body, which are easy to be affected by factors such as clothing and carrying objects. Although the model-based methods can effectively reduce the influence of appearance factors, it has high computational complexity. Therefore, this paper proposes a gait recognition method based on the 3D skeleton data and graph convolutional network. The 3D skeleton data is robust to the change of view. In this paper, we extract 3D joint feature and 3D bone feature based on 3D skeleton data, design a dual graph convolutional network to extract corresponding gait features and fuse them at feature level. At the same time, we use a multi-loss strategy to combine center loss and softmax loss to optimize the network. Our method is evaluated on the dataset CASIA B. The experimental results show that the proposed method can achieve state-of-the-art performance, and it can effectively reduce the influence of view, clothing and other factors.

Scene text detection has been a long standing hot and challenging research topic in pattern recognition. In this paper a novel coarse-to-fine text detection method is proposed to solve edge-adhesion problem. In coarse detection stage, Skeleton-cut detector is proposed. At first,8-Neighborhoods-Search is applied on skeletons map to find the adhesion junctions between text and background skeletons. Then junctions in disordered skeletons are picked out by hysteresis selection and cut to separate text skeletons from background. And the text skeletons are verified through a two-stage classifier to obtain the coarse detection result. In fine detection stage, bounding boxes of all these filtered skeletons are weighted accumulated to obtain the Static Skeleton Response(SSR). Then many finer text lines candidates can be calculated through the gradient operation to the SSR’s horizontal projection. And the text rectification based on Binary-Tree-Search is proposed to find a path from text lines’ search space to the fine detection result. Experimental results on ICDAR dataset, SVT dataset and MSRA-TD500 dataset demonstrate that our algorithm achieves state of art performance in scene text detection.

Real-time detection of wire surface defects is an important part of wire quality detection. Because traditional algorithms need lots of parameters and have weak universality, besides, time performance of detector based on candidate region is poor. To solve these problems, we study the effectiveness of single-stage detector in real-time detection, and propose a detection algorithm of wire surface defects combining enhanced darknet53 and feature pyramid (FPN). Firstly, we use CBAM_Darknet53 which introduces channel attention and spatial attention to extract more differentiated features. Secondly, considering the large change of defects, we use the multi-level feature pyramid (MLFPN) which adds the maximum pooling layer to fuse multi-level features to detect multi-scale defects. Then we reprocess the detector to improve the detection rate of defects and the accuracy of the detection box. Finally, network structure is optimized by modifying loss function. Experiments on defect datasets in real industrial environments show that recall and mAP of this method reach 94.49% and 88.46%, which is higher than state-of-the-art methods.

Skeleton-based action recognition has catched more eyes recently. Many approaches model the human skeleton sequences spatio-temporal representation by Recurrent Neural Networks, while RNNs-based approaches don’t explicitly have the attention scheme. In this paper, a body part level attention method is proposed for determining which parts are discriminative. On the basis of human joints natural connection, human body is divided to disjoint parts which then are fed into corresponding modules to generate action scores. A body part level attention module which distributes different importance to different action scores is proposed to adaptively learn informative body parts. The final representative of the action video is produced in a weighted average way and it will be fed into DNNs for classification. Out method outperforms other attention model and we obtain the state-of-the-art performance compared to other methods on two popularly used benchmark NTU dataset and UT Kinect dataset.

Due to the large scale variation of surface defects of different types of strip steel, there are limitations in using threshold segmentation to locate objects, we propose a surface defect detection algorithm combining selective search and class activation mapping (CAM) to improve objects localization. First, we use selective search to generate defect bounding box in the image, and predicts the classification and CAM of the defect in the image through the trained model. Then, in the defect detection, filter the bounding box with the classification information of the defect as priori knowledge. We only retain the bounding box that approximate the shape of the defect and map the filtered defect bounding box to the CAM of the corresponding defect. Finally, select the bounding box with the highest score as a detection result. Experiment results show that the proposed method can achieve a mean average precision of 91.1% on our dataset. And it can more accurately locate defects in the image. Compared with traditional CAM, our method has more excellent detection performance in surface defect detection applications of strip steel.

EAST [1] is a popular end to end text detector, however, it performs deficiently with long or large texts, because of limited network's feature extraction capacity and narrow receptive fields. Based on EAST, we propose an approach named Enhanced EAST. Firstly, we offer low-level feature layers more semantic information by introducing information from high levels to low levels, which reduces the information gap between different layers. Meanwhile, we utilize a two-stream large kernel convolution to increase receptive fields with reasonable computational cost, therefore, improving the network's features detection and fusion ability. In addition, we also optimize the label generation of training data and design a weighted mask for each text, which can guide the training process to enhance the network's complete shape perception of texts, thus impelling the predicted text boxes locate more accurately. In the end, we perform data equalization and augmentation in the experiments and experiment results on ICDAR 2015, MSRA-TD500 and ICDAR 2017 MLT datasets demonstrate the proposed algorithm achieves a state-of-art performance in multi-oriented scene text detection.

Skeleton-based action recognition has made impressive progress these years. Yet few methods consider spatial configuration of joints and temporal correlation meanwhile as a unity. To model action sequences in a way which regard both two dimensions, a Graph Convolutional Long Short Term Memory Networks (GC-LSTM) model is proposed in this paper, which automatically learns spatiotemporal features to model the action. Our model introduces the GCN operation into conventional RNN unit including graph convolution at each time step for input-to-state and state-to-state transition. Plenty of experiment analyses show that the proposed GC-LSTM model strives (1) to focus more on discriminative parts at discriminative frames and (2) to be insensitive to the redundant parts which are irrelevant for recognition. Moreover, several methods are compared with ours on two publicly available datasets and experimental results demonstrate that our model achieves the state-of-the-art performance.

Text detection in natural scene images is a challenging problem in computer vision. To robust detect various texts in complex scenes, a hierarchical recursive text detection method is proposed in this paper. Usually, texts in natural scenes are not alone and arranged into lines for easy reading. To find all possible text lines in an image, candidate text lines are obtained using text edge box and conventional neural network at first. Then, to accurately find out the true text lines in the image, these candidate text lines are analyzed in a hierarchical recursive architecture. For each of them, connected components segmentation and hierarchical random field based analysis are recursively employed until the detected text line no more changes. Now the detected text lines are output as the text detection result. Experiments on ICDAR 2003 dataset, ICDAR 2013 dataset and Street View Dataset show that the hierarchical recursive architecture can improve text detection performance and the proposed method achieves the state-of-art in scene text detection.

Traditional pattern recognition methods are widely used to detect defects in the industry, however most of existing methods are not universal for all kinds of defects on bars. This paper proposes a method which combines Rectified Linear Units (ReLU) and Batch Normalization(BN) in Sacked Denoising Autoencoders(SDA) for surface defect detection of bars. Gradient diffusion often occurs in traditional SDA, which leading to inefficient learning. In order to solve gradient diffusion, we replace the Sigmod activation function with ReLU function. The activation value calculated by ReLU is often oversparing which leading to loss of features. For solving the oversparing of ReLU, we add BN layer into SDA to normalize each batch. Finally we obtain network weights through unsupervised pre-training and supervised fine-tuning. We train two models which one for prediction and the other for reconstruction. Experiment results show that the proposed method can achieve an average accuracy rate of 99.1% on our data set. Compared with the traditional pattern recognition method, traditional SDA and Fisher criterion-based stacked denoising autoencoders(FCSDA), our method shows higher accuracy and TPR. Moreover, due to the addition of BN, the time complexity of our method is significantly lower than the SDA and FCSDA.

Improving the precision of weakly supervised multiscale objects localization is of significant challenge in computer vision, especially when tackling the small objects. in this paper, we propose to integrate the feature pyramid network (FPN) with convolutional neural network (CNN) for weakly supervised object localization, where the FPN is built upon the outputs of different layers of the CNN. Then, we upsample the high-level maps by nearest-neighbor interpolation and fuse with the low-level maps in the FPN to produce multi-scale fused maps which features of both high resolution and strong semantics. Finally, we produce class activation maps by each layer of the FPN and gain multiple prediction scores by wildcat spatial pooling. To acquire more precise localization, we select the class activation map that corresponds to the highest score across all multiscale maps for object localization. In particular, we choose the maximum response regions of the class activation map for pointwise localization and choose the largest connected component above the threshold in the class activation map for bounding box localization. By applying the proposed strategy over PASCAL VOC dataset and MS COCO dataset, it is demonstrated that our strategy is highly effective in improving the precision of weakly supervised object localization as compared with some of the state-of-the-art weakly supervised methods.

Due to the continuous development of GAN, vivid faces can be generated, and the use of GAN for face aging becomes a novel trend. However, many existing works for face aging require tedious pre-processing of datasets. This brings a lot of computational burden and limits the application of face aging. In order to solve these problems, a face aging network is constructed using IcGAN without any data pre-processing which map a face image into personality and age vector spaces through encoders Z and Y. Different from the previous work, we make an emphasis on the preservation of both personalized and aging features. Thus, the minimize absolute reconstructing loss is proposed to optimize vector z, which can remain the personality characteristics, meanwhile preserving the pose, hairstyle and background of the input face. Additionally, we introduce a novel age vector optimization approach by classifying reconstruction loss and introduce the parameter λ which is well-balanced between large age features and subtle texture features. The experimental results demonstrate our proposed AIGAN provides better aging faces over other state-of-the-art age progression methods.

Person re-identification is a challenging task due to the background clutters, occlusion and illumination variations. In addition, the pedestrian misalignment always exists in some automatic-detection datasets. In this paper, we propose a Multi-Part Competition Network (MPCN) consisting of Multi-Part Network (MPN) and Part Competition Network (PCN), which aims to solve the misalignment problem caused by the detector errors and human pose variations. First, we construct original body parts and enlarged body parts using human pose estimation algorithm. These two kinds of body parts not only alleviate the misalignment from background and varying human pose but also solve the missing details and imprecise body parts introduced by human pose estimator. Then, we use MPN to acquire global features and two different body parts features. The components of MPN, a global branch and two part branches, are combined by ROI pooling layer. Finally, we apply PCN to achieve a tradeoff between the original body parts and the enlarged body parts and acquire discriminative part features from these two different body parts. Extensive evaluations on three widely used re-id datasets, Market-1501, CUHK03, VIPeR demonstrate that our proposed network have a competitive result compared to the state-of-the-art methods.

TextBoxes[1] is one of the most advanced text detection method in both aspects of accuracy and efficiency, but it is still not very sensitive to the small text in natural scenes and often can not localize text regions precisely. To tackle these problems, we first present a Bidirectional Information Aggregation (BIA) architecture by effectively aggregating multi-scale feature maps to enhance local details and strengthen context information, making the detector not only work reliably on multi-scale text, especially the small text, but also predict more precise boxes for texts. This architecture also results in a single classifier network, which allows our model to be trained much faster and easily with better generalization power. Then, we propose to use multiple symmetrical feature maps for feature extraction in the test stages for further improving the performance on the small text. To further promote precise predicting boxes, we present a statistical grouping method that operates on the training set bounding boxes to generate aspect ratios for default boxes. Finally, our model not only outperforms the TextBoxes without much time overhead, but also provides promising performance compared to the recent state-of-theart methods on the ICDAR 2011 and 2013 database.

Due to the rapid development of machine learning and data mining in nowadays, how to acquire information from images becomes more and more important. The direct information of an image is the texts inside. However, detecting such texts in images is always a challenging problem in computer vision area.Edge is one of the most important clues in scene character detection task. However, many edge based text detection methods usually had trouble with sticky edges and did not fully utilize characteristic of texts. In this paper, we proposed a method for detecting and localizing texts in natural scene images, by edge recombining, edge filtering and multi-channel processing. In order to segment texts from backgrounds, edges are firstly over-segmented into edge segments during edge analysis. These edge segments are then recombined to candidate characters and an edge filter is used to filter out most of background edges. The left candidate character edges are linked up to candidate text lines. We use two different classifiers to filter out non-text lines. To classify more accurately, extracted edge-based and region-based features are firstly stored in feature pools. Then we use liner SVM to select the most effective features from the feature pool to train classifiers. Finally, multi-channel is used to ensure the recall and a modified non-maximal suppress is applied to eliminate duplicate results. Experimental results on the ICDAR 2011 competition dataset and SVT database demonstrate the effectiveness of our method.

This paper proposes an effective method to extract printed and handwritten characters from multilingual document images to build corpus. To extract the characters from the document images, a connected component analysis method is used to remove the graphics. After that, multiple types of features and AdaBoost algorithm are introduced to classify printed and handwritten characters in a more versatile and robust way. Firstly, the content of the image is divided into several text patches which are then used to distinguish different languages. Secondly, we use the multiple types of features and AdaBoost algorithm to train the classifiers based on the segmented patches. Finally, we can separate printed and handwritten parts of new image set by the trained classifiers. The proposed method improves the precision of the extraction of written materials in text images of different languages. Experimental results demonstrate that the proposed method is more accurate in terms of precision and recall rate compared with the state-of the-art methods.

Gait is an important biometrics in human identification, but the view variation problem seriously affects the accuracy of gait recognition. Existing methods for multi-view gait-based identification mainly focus on transforming the features of one view to another view, which might be unsuitable for the real applications. In this paper, we propose a multi-view gait recognition method based on RBF network that employs a unique view-invariant model. First, extracts the gait features by calculating the gait individual image (GII), which could better capture the discriminative information for cross view gait recognition. Then, constructs a joint model, use the DLDA algorithm to project the model and get a projection matrix. Finally, the projected eigenvectors are classified by RBF network. Experiments have been conducted in the CASIA-B database to prove the validity of the proposed method. Experiment results shows that our method performs better than the state-of-the-art multi-view methods.

Skeleton based human gesture detection and recognition have been attracting increasing attention in the field of human action understanding. A number of approaches have been proposed to only explore skeleton inherent pose features by deep learning method. In this paper, we propose a novel instantaneous kinematic descriptor and a motion-pose recurrent neural network (RNN) for skeleton based gesture detection and recognition. Instead of the raw skeleton joint position as input, we propose instantaneous kinematic descriptor to represent not only the skeleton inherent pose but also the instantaneous movement at a current frame. Meanwhile, the proposed network is capable of transforming the gesture detection to the problem of frame labeling, which can model both the frame-wise dynamic motion and long-term temporal context of gestures. The proposed method is evaluated on the Chalearn LAP gesture dataset, and the result demonstrates that our method achieves the state-of-art performance in the task of skeleton-based gesture detection and recognition.

As the structural information of an object can be well descripted by edge pixels, we observe that the greatest challenge for locating text edges on scene image is how to handle the edge-adhesion problem. In this paper we propose the Skeleton-cut Text Detector, which take text-specific edge cues such as a novel presentation skeleton into account to hunt text efficiently with improved recall rate. To address edge-adhesion problem, skeleton-junctions detection and elimination are performed first to cut candidate text out of the edge map. Then the candidates are verified through a two-stage classifier based on properties like concentration ratio. Finally iteratively local refinement (IRL) is applied to enhance the overlap of proposals. Experimental results on public benchmarks, ICDAR 2013 and MSRA, demonstrate that our algorithm achieves state-of-the-art performance. Moreover in severe scenarios, our proposed method shows stronger adaptability to texts by exploiting skeleton compared to conventional presentations like MSERs.

Recently, the recurrent neural network(RNN) has been widely used for skeleton based action recognition because of its ability to model long-term temporal dependencies automatically. However, current methods cannot accurately describe the characteristics of actions, because they only consider joint positions rather than high order features like relative motion to different joints and ignore the impact of human physical structure. In this paper, a novel high order joint relative motion feature(JRMF) and a novel human skeleton tree RNN network(HST-RNN) are proposed. Human skeleton joints structure can be represented by a tree. The JRMF for each skeleton joint consists of the relative position, velocity and acceleration to this joint of all its descendant joints. It describes the instantaneous status of the skeleton joint better than joint positions. The HST-RNN network is constructed with the same tree structure as the human skeleton joints. Each node of the tree is a Gated Recurrent Unit(GRU) and represents a skeleton joint. The outputs of its child nodes and the corresponding JRMF are concatenated and fed into each GRU. The network combines low-level features and extracts high level features from the leaf nodes to the root node in a hierarchical way according to the human physical structure. The experimental results demonstrates that the proposed HST-RNN with JRMF achieves the state-of-art performance on challenging datasets like MSR-Action3D, UT-Kinect and UTD-MHAD.

As an important part of scene understanding, scene text extraction can promote the performance of text recognition significantly. This paper focuses on extracting text pixels from background in images of natural scenes. First, a novel feature descriptor called local symmetry transform is proposed to detect local symmetry relationship of strokes for each image pixel. The presented method is local and color invariant, which makes it robust enough to handle many complex situations for scene text extraction such as variation of lighting, blur of text regions, shadow and complex background. Then, guided filter is introduced to enhance the local symmetry map produced by the local symmetry transform, which can smooth the feature map and reduce noise noticeably. Finally, text candidates are generated as seed points, and by seed-based segmentation we automatically judge the text polarity to obtain correct text regions. Experimental results on KAIST dataset show that our method leads to a high performance.

In this paper, we propose a gesture recognition method using Enhanced Depth Motion Map (eDMM) and Static Pose Map (SPM) from depth videos. Firstly, the eDMM is proposed to describe motions in gesture videos, which is more robust to noise than DMM. The SPM is constructed to describe static postures of a gesture, which provides complementary information for the eDMM. Then a 2-CNN architecture is used to extract features from eDMM and SPM. The extracted features are fused to form gesture feature which contains both dynamic movement information and static pose information. Finally an ANN is trained for gesture recognition. The proposed method is evaluated on the Chalearn IsoGD dataset and the NATOPS dataset. The experimental results show that the proposed method achieves higher recognition rate than the baseline method of the Chalearn IsoGD dataset and is competitive with the-state-of-art of the NATOPS dataset.

This paper presents a text detection method based on multi-scale Stroke Width Transform (SWT). First, an image pyramid is built and SWT is performed on each level of the pyramid. Second, edge components are filtered using two novel features, stroke pair ratio (SPR) and edge density of a connected component (EDC). Next, the remaining edge components on each level are grouped into text lines. And these lines are projected back onto a single image and merged. Finally, candidate text lines are verified by integrating block level features and line level features. The multi-scale mechanism makes it possible to detect text defected by reflection or blurring. And the two features are proved to be both effective and efficient in filtering non-text edges. Moreover, experimental results on the ICDAR Robust Reading Competition datasets show that the proposed text detection method provides promising performance.

In this paper we propose a multi-view method to recognize hand gestures using point cloud. The main idea of this paper is to project point cloud into view images and hand gestures are described by extracting and fusing features in view images. The conversion of feature space increases the inner-class similarity and meanwhile reduces the inter-class similarity. The features of view images are extracted in parallel so the scale of each feature extractor can be reduced to converge easily. In our method we perform a refined hand segmentation to segment hand form background firstly. Then the segmented hand point cloud is projected into different view planes to form view images. Next we use convolutional neural networks as feature extractors to extract features of view images. The extracted view image features are fused to form the features of hand gestures. Finally a SVM is trained for hand gesture recognition. The experimental results show that our multiview method achieves higher recognition rate and more robust to the challenging rotation changes especially out-plane rotations.

Active Energy Image (AEI) is an efficient template for gait recognition. However, the AEI is short of the temporal information. In this paper, we present a novel gait template, named Enhanced Active Color Image (EACI). The EACI is extract the difference of two interval in each gait frame, followed by calculating the width of that difference image and then mapping into RGB space with the ratio, describing the relative position, and composition them to a single EACI. To prove the validity of the EACI, we employ experiments on the USF HUMANID database. Experiment result shows that our EACI describes the dynamic, static and temporal information better. Compared with other published gait recognition approaches, we achieve competitive performance in gait recognition.