Search

Machine vision-based planar grasping detection is challenging due to uncertainty about object shape, pose, size, etc. Previous methods mostly focus on predicting discrete gripper configurations, and may miss some ground-truth grasp postures. In this article, a pixel-level grasp detection method is proposed, which uses deep neural network to predict pixel-level gripper configurations on RGB images. First, a novel oriented arrow representation model (OAR-model) is introduced to represent the gripper configuration of parallel-jaw and three-fingered gripper, which can partly improve the applicability to different grippers. Then, the adaptive grasping attribute model is proposed to adaptively represent the grasping attribute of objects, for resolving angle conflicts in training and simplifying pixel-level labeling. Lastly, the adaptive feature fusion and grasp-aware network (AFFGA-Net) is proposed to predict pixel-level OAR-models on RGB images. AFFGA-Net improves the robustness in unstructured scenarios by using hybrid atrous spatial pyramid and adaptive decoder connected in sequence. On the public Cornell dataset and actual objects, our structure achieves 99.09% and 98.0% grasp detection accuracy, respectively. In over 2400 robotic grasp trials, our structure achieves an average success rate of 98.77% in single-object scenarios and 93.69% in cluttered scenarios. Moreover, AFFGA-Net completes a grasp detection pipeline within 15 ms. [ABSTRACT FROM AUTHOR]

Machine vision based vehicle re-identification (ReID) plays an important role in some Intelligent Transportation Systems (ITS). Yet, most previous methods mainly focus on fixed surveillance cameras instead of Unmanned Aerial Vehicle (UAV). With high flexibility, the UAV-based vehicle ReID problem has some special challenges including complicated shooting angles, low discrimination of top-down features, and large variance in vehicle scales, etc. To overcome these challenges, we propose a novel structure for UAV-based vehicle ReID without license plates. Firstly, a triple-head segmentation net is proposed for segmenting UAV-captured vehicles under different heights and directions. Secondly, a posture calibration model is designed to uniform the vehicle postures based on the segmentation results, with the purpose of reducing the influence of different postures. Thirdly, the novel Cross-View & Hard-Sensitive Metric Learning (CHSML) method is proposed to train a ReID network with cross-view training constraint and hard sensitive principles; the mechanism of CHSML takes the cross-view samples of same ID as a training unit to learn the potential visual relationship in cross-view and builds a hard sensitive weight matrix to make learning more focus on hard samples, which improves the low ReID accuracy brought by cross-view or hard samples. Moreover, to facilitate the research of UAV-based vehicle ReID, a large-scale UAV-based vehicle ReID dataset called VeRi-UAV is released with 17 516 vehicles of 453 IDs. The experiments show that the proposed structure gains a better performance compared with the representative methods in the UAV-based vehicle ReID task. [ABSTRACT FROM AUTHOR]

Driver attention prediction can solve the problem of ‘Where should the driver pay attention?’, Most previous methods are designed to predict regional attention with redundant regions. Furthermore, popular spatial-temporal feature extraction networks such as ConvLSTM and 3D-CNN are difficult to achieve real-time. To overcome these difficulties, we propose an Adaptive Short-temporal Induced Aware Fusion Network (ASIAF-Net) for region-level and object-level driver attention prediction. In ASIAF-Net, we design an Attention Related Spatial Feature Encoder (AF-Encoder) and an Induced Aware Fusion Network (IAF-Net) as the main network; with an Association Analysis Cell (AAC), the AF-Encoder makes it possible to effectively capture the relationship information of different objects. Considering most vital visual cues from moving objects, we propose a Self-adaptive Short-temporal Feature Extraction Module (SSFE-Module) to obtain inter-frame motion features. In IAF-Net, a Multi-scale Driver Attention Region Prediction Branch is designed to predict the regional attention, and an Object Saliency Estimation Branch is proposed to fuse the perception results and the regional attention map to estimate the object-level attention. Experiments show that the proposed ASIAF-Net can predict driver’s attention on regions and objects more robustly and precisely than state-of-the-art methods on three datasets, and that it achieves real-time on our ADAS platform. [ABSTRACT FROM AUTHOR]

On-board vision based traffic risk assessment is a challenging task for intelligent driving systems, which has some special challenges including spatial-temporal feature extraction, different judgements of risks, real-time requirement, lacking data, etc. To overcome these difficulties, we propose a novel Temporal Shift and Spatial Attention based Two-stream Network (TSSAT-Net) for on-board vision based traffic risk assessment. Firstly, we build new judgement measures that integrate actual driving experience and scenario complexity, and release an on-board vision based traffic risk assessment dataset. Secondly, a novel weighted Temporal Shift Module (weighted-TSM) based two-stream network is proposed; unlike previous methods that rely on complex and time-consuming 3D CNN or LSTM calculation, the proposed two-stream network can effectively extract spatial-temporal features using a weighted temporal shift mechanism with just 2D CNN calculation requirement. Thirdly, a spatial and channel attention mechanism is proposed to make the TSSAT-Net more focus on the features closely related to traffic risks, avoiding redundant information in complex traffic scenarios. Experiments based on the released comprehensive dataset show that our method achieves the state-of-the-art classification accuracy in real-time. [ABSTRACT FROM AUTHOR]

Machine vision based vehicle counting and traffic flow estimation are challenging problems especially for dense traffic scenarios. Previous line of interest (LOI) counting methods rarely focus on dense scenarios and their performance largely relies on the accuracy of tracking. Avoiding the use of complex tracking methods, an LOI counting framework is proposed to address the bi-directional LOI counting problem in dense scenarios. There are three main contributions. Firstly, instead of treating the LOI vehicle counting problem as a combination of detecting and tracking of individual vehicles, the bi-directional traffic flow is taken as a whole and a novel spatio-temporal counting feature (STCF) is proposed for extracting bi-directional traffic flow features in dense traffic scenarios. Secondly, without relying on a multi-target tracking process for tracking and counting each vehicle, a counting network is proposed, called the counting Long Short-Term Memory (cLSTM) network, to do analysis of the bi-directional STCF features and vehicle counting in successive video frames. Lastly, an estimation model is designed for estimating traffic flow parameters including speed, volume and density. Experiments performed on the UA-DETRAC dataset and the captured videos show that the proposed vehicle counting method outperforms the tested representative LOI counting methods in both accuracy and speed, and that the proposed framework can efficiently estimate traffic flow parameters including speed, volume and density in real time. [ABSTRACT FROM AUTHOR]

Driver gaze zone estimation is an important task in Advanced Driver Assistance Systems (ADAS), which suffers difficulties including head pose, capture direction, glass occlusion, and real-time requirement, etc. Most previous methods combine face modalities and head pose using concat process, which may result in over-fitting due to the unbalanced dimension. Focusing on gaze zone estimation problems, we propose the Head Pose Fusion Assisted supervision & Eye Region Weighted Encoding (HP-ERW) structure, which fuses head pose attribute and face modalities together through spatial attention and Kronecker product mechanisms. Firstly, we introduce a pre-processing module dealing with head pose and face information, with the purpose of extracting input vectors and improving the fusion speed of the HP-ERW structure. Secondly, an Eye Region Weighted Encoding Network (ERW-Net) based on spatial attention is proposed to strengthen the networks perception ability for encoding features. Finally, we propose a dual-channel Head Pose Fusion Network (HP-Net) based on the Kronecker product mechanism, with the purpose of fusing head pose and improving the estimation accuracy. Experiments show that the HP-ERW outperforms compared existing methods on several public datasets. The designed ADAS using the proposed method achieves 23.5 fps real-time application with small memory requirement of 4,884 KB. [ABSTRACT FROM AUTHOR]

Though license plate detection has been successfully applied in some commercial products, the detection of small and vague license plates in real applications is still an open problem. In this paper, we propose a novel hybrid cascade structure for fast detecting small and vague license plates in large and complex visual surveillance scenes. For rapid license plate candidate extraction, we propose two cascade detectors, including the Cascaded Color Space Transformation of Pixel detector and the Cascaded Contrast-Color Haar-like detector; these two cascade detectors can do coarse-to-fine detection in the front and in the middle of the hybrid cascade. In the end of the hybrid cascade, we propose a cascaded convolutional network structure (Cascaded ConvNet), including two detection-ConvNets and a calibration-ConvNet, which is designed to do fine detection. Through experiments with different evaluation data sets with many small and vague plates, we show that the proposed framework is able to rapidly detect license plates with different resolutions and different sizes in large and complex visual surveillance scenes. [ABSTRACT FROM AUTHOR]

In this paper, we propose a high-performance traffic sign recognition (TSR) framework to rapidly detect and recognize multiclass traffic signs in high-resolution images. This framework includes three parts: a novel region-of-interest (ROI) extraction method called the high-contrast region extraction (HCRE), the split-flow cascade tree detector (SFC-tree detector), and a rapid occlusion-robust traffic sign classification method based on the extended sparse representation classification (ESRC). Unlike the color-thresholding or extreme region extraction methods used by previous ROI methods, the ROI extraction method of the HCRE is designed to extract ROI with high local contrast, which can keep a good balance of the detection rate and the extraction rate. The SFC-tree detector can detect a large number of different types of traffic signs in high-resolution images quickly. The traffic sign classification method based on the ESRC is designed to classify traffic signs with partial occlusion. Instead of solving the sparse representation problem using an overcomplete dictionary, the classification method based on the ESRC utilizes a content dictionary and an occlusion dictionary to sparsely represent traffic signs, which can largely reduce the dictionary size in the occlusion-robust dictionaries and achieve high accuracy. The experiments demonstrate the advantage of the proposed approach, and our TSR framework can rapidly detect and recognize multiclass traffic signs with high accuracy. [ABSTRACT FROM PUBLISHER]

This paper describes a traffic sign detection (TSD) framework that is capable of rapidly detecting multiclass traffic signs in high-resolution images while achieving a high detection rate. There are three key contributions. The first is the introduction of two features called multiblock normalization local binary pattern (MN-LBP) and tilted MN-LBP (TMN-LBP), which are able to express multiclass traffic signs effectively. The second is a tree structure called split-flow cascade, which utilizes common features of multiclass traffic signs to construct a coarse-to-fine TSD detector. The third contribution is the Common-Finder AdaBoost (CF.AdaBoost) algorithm, which is designed to find common features of different training sets to develop an efficient Split-Flow Cascade tree (SFC-tree) for multiclass TSD. Through experiments with an evaluation data set of high-resolution images, we show that the proposed framework is able to detect multiclass traffic signs with high detection accuracy in real time and that it outperforms the state-of-the-art approaches at detecting a large number of different types of traffic signs rapidly without using any color information. [ABSTRACT FROM PUBLISHER]