Search

With the increasing deployment of autonomous taxis in different cities around the world, recent studies have stressed the importance of developing new methods, models and tools for intuitive human–autonomous taxis interactions (HATIs). Street hailing is one example, where passengers would hail an autonomous taxi by simply waving a hand, exactly like they do for manned taxis. However, automated taxi street-hailing recognition has been explored to a very limited extent. In order to address this gap, in this paper, we propose a new method for the detection of taxi street hailing based on computer vision techniques. Our method is inspired by a quantitative study that we conducted with 50 experienced taxi drivers in the city of Tunis (Tunisia) in order to understand how they recognize street-hailing cases. Based on the interviews with taxi drivers, we distinguish between explicit and implicit street-hailing cases. Given a traffic scene, explicit street hailing is detected using three elements of visual information: the hailing gesture, the person's relative position to the road and the person's head orientation. Any person who is standing close to the road, looking towards the taxi and making a hailing gesture is automatically recognized as a taxi-hailing passenger. If some elements of the visual information are not detected, we use contextual information (such as space, time and weather) in order to evaluate the existence of implicit street-hailing cases. For example, a person who is standing on the roadside in the heat, looking towards the taxi but not waving his hand is still considered a potential passenger. Hence, the new method that we propose integrates both visual and contextual information in a computer-vision pipeline that we designed to detect taxi street-hailing cases from video streams collected by capturing devices mounted on moving taxis. We tested our pipeline using a dataset that we collected with a taxi on the roads of Tunis. Considering both explicit and implicit hailing scenarios, our method yields satisfactory results in relatively realistic settings, with an accuracy of 80%, a precision of 84% and a recall of 84%. [ABSTRACT FROM AUTHOR]

[Display omitted] Disparity estimation of satellite stereo images is an essential and challenging task in photogrammetry and remote sensing. Recent researches have greatly promoted the development of disparity estimation algorithms by using CNN (Convolutional Neural Networks) based deep learning techniques. However, it is still difficult to handle intractable regions that are mainly caused by occlusions, disparity discontinuities, texture-less areas, and repetitive patterns. Besides, the lack of training datasets for satellite stereo images remains another major issue that blocks the usage of CNN techniques due to the difficulty of obtaining ground-truth disparities. In this paper, we propose an end-to-end disparity learning model, termed hierarchical multi-scale matching network (HMSM-Net), for the disparity estimation of high-resolution satellite stereo images. First, multi-scale cost volumes are constructed by using pyramidal features that capture spatial information of multiple levels, which learn correspondences at multiple scales and enable HMSM-Net to be more robust in intractable regions. Second, stereo matching is executed in a hierarchical coarse-to-fine manner by applying supervision to each scale, which allows a lower scale to act as prior knowledge and guides a higher scale to attain finer matching results. Third, a refinement module that incorporates the intensity and gradient information of the input left image is designed to regress a detailed full-resolution disparity map for local structure preservation. For network training and testing, a dense stereo matching dataset is created and published by using GaoFen-7 satellite stereo images. Finally, the proposed network is evaluated on the Urban Semantic 3D and GaoFen-7 datasets. Experimental results demonstrate that HMSM-Net achieves superior accuracy compared with state-of-the-art methods, and the improvement on intractable regions is noteworthy. Additionally, results and comparisons of different methods on the GaoFen-7 dataset show that it can severs as a challenging benchmark for performance assessment of methods applied to disparity estimation of satellite stereo images. The source codes and evaluation dataset are made publicly available at https://github.com/Sheng029/HMSM-Net. [ABSTRACT FROM AUTHOR]

The binary segmentation of roads in very high resolution (VHR) remote sensing images (RSIs) has always been a challenging task due to factors such as occlusions (caused by shadows, trees, buildings, etc.) and the intraclass variances of road surfaces. The wide use of convolutional neural networks (CNNs) has greatly improved the segmentation accuracy and made the task end-to-end trainable. However, there are still margins to improve in terms of the completeness and connectivity of the results. In this article, we consider the specific context of road extraction and present a direction-aware residual network (DiResNet) that includes three main contributions: 1) an asymmetric residual segmentation network with deconvolutional layers and a structural supervision to enhance the learning of road topology (DiResSeg); 2) a pixel-level supervision of local directions to enhance the embedding of linear features; and 3) a refinement network to optimize the segmentation results (DiResRef). Ablation studies on two benchmark data sets (the Massachusetts data set and the DeepGlobe data set) have confirmed the effectiveness of the presented designs. Comparative experiments with other approaches show that the proposed method has advantages in both overall accuracy and F1-score. The code is available at: https://github.com/ggsDing/DiResNet. [ABSTRACT FROM AUTHOR]

In the multi-object tracking process, a long-term tracking algorithm for traffic scene based on deep learning is proposed to handle several challenging problems, such as the complex variation of background illumination, change of pixel due to partial occlusion, cumulative error and the short-term disappearance of the target. Firstly, we train a CNN to identify and determine the target bounding box in a traffic scene. Secondly, we use a particle filter (PF) as the tracker to implement the preliminary multi-object tracking. Finally, the multi-object tracking trajectory then is generated by the mutual supervision of the PF tracker and CNN detector. In order to evaluate the experimental results, we use the forward-backward (FB) error of our tracker at a certain moment. The experimental results show that the method can track single and multi-objects in long-term tracking in real-time. For the situation of target disappearance and reappearance, the proposed algorithm can also recover its long-term tracking. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) surface information is becoming the data of choice for pavement inspection and maintenance. Pavement digital reconstruction still faces various challenges, including field-of-view (FOV) limits, traffic influences, data acquisition speed and cost. This paper presents a full-FOV pavement stereo reconstruction framework integrating unmanned aerial vehicle (UAV) photography, object detection and multi-view occlusion optimization. A UAV-YOLO vehicle detector embedding depth-wise separable convolution and resolution adjustment unit is developed for noise localization. To improve reconstruction quality and speed, multi-view occlusion optimization is proposed for determining the optimal image series spatial distribution. The results show that the UAV-YOLO detector achieves an overall AP 75 of 93.69% with an inference speed of 157 FPS. Through multi-criterion evaluation, the pavement digital models reconstructed under dynamic traffic conditions have satisfactory performance in terms of point cloud noise, density, similarity and accuracy. In addition, the proposed stereo reconstruction workflow saves 30.27% processing time over conventional SfM workflow. • UAV photogrammetry and deep learning are integrated for pavement stereo reconstruction with traffic influences. • A UAV-YOLO detector is developed for high-accurate vehicle localization in multi-altitude UAV photography. • Multi-view occlusion optimization is proposed to improve full FOV pavement reconstruction quality and speed. • A 2D/3D digital surface dataset is established for future study of automated defect detection. [ABSTRACT FROM AUTHOR]

• A deep learning method for oriented and small wheat spike detection was proposed to accurately detect wheat spikes in UAV images. • The method introduces the orientation feature of wheat spikes into the network and reduces the negative effect of background on detection based on circular smooth label. • The method can better solve tiny wheat spikes' occlusion and overlap problems. Detecting and characterizing spikes from wheat field images is essential in wheat growth monitoring for precision farming. Along with various technological developments, deep-learning-based methods have remarkably improved wheat spike detection performance. However, detecting small and overlapping wheat spikes in UAV images is still challenging because high spike occlusion and complex background can cause error detection and miss detection problems. This paper proposes a deep learning method for oriented and small wheat spike detection (OSWSDet). Unlike classical wheat spike detection methods, OSWSDet introduces the orientation of wheat spikes into the YOLO framework by integrating a circle smooth label (CSL) and a micro-scale detection layer. These improvements enhance the ability to detect small-sized wheat spikes and prevent wheat spike detection errors. The experiment results show that OSWSDet outperforms classical wheat spike detection methods, and the average accuracy (AP) is 90.5%. OSWSDet can accurately detect spikes in UAV images with complex field backgrounds and provides technical references for future field wheat phenotype monitoring. [ABSTRACT FROM AUTHOR]

Lane mark detection plays an important role in autonomous driving under structural environments. Many deep learning-based lane mark detection methods have been put forward in recent years. However, most of current methods limit their solutions within one single image and do not make use of the de facto successive image input during the driving scene, which may lead to inferior performance in some challenging scenarios such as occlusion, shadows, and lane mark degradation. To address the issue, we propose a novel lane mark detection network which takes pre-aligned multiple successive frames as inputs to produce more stable predictions. A Spatial-Temporal Attention Module (STAM) is designed in the network to adaptively aggregate the feature information of history frames to the current frame. Various structure of the STAM is also studied to ensure the best performance. Experiments on Tusimple and ApolloScape datasets show that our method can effectively improve lane mark detection and achieve state-of-the-art performance. [ABSTRACT FROM AUTHOR]