Search

Convolutional neural networks (CNNs) have shown excellent performance for vision-based lane detection. However, maintaining the performance of the trained models under new test scenarios still remains challenging due to the dataset bias between the training and test datasets; In lane detection processes, the dataset bias can be categorized into lane position bias and lane pattern bias, with the former one particularly influences the lane detection performance. To tackle this dataset bias, this article proposes a unified viewpoint transformation (UVT) method that transforms the camera viewpoints of different datasets into a common virtual world coordinate system, such that the mismatched lane position distributions can be effectively aligned. Experiments are conducted on multiple datasets including the Caltech [1] , Tusimple [2] , and KITTI [3] dataset. The results demonstrate the effectiveness of the UVT algorithm in improving the lane detection performance on the test datasets. Moreover, by incorporating the UVT into other techniques that tackling the dataset bias, the lane position and pattern differences are disentangled and separately minimized. As a result, the performance gap between the training data and the test scenarios can be bridged. Specifically, the model trained on the KITTI dataset have achieved high performance in the Tusimple and the Caltech dataset (F1-score: 84.8 and 87.1%). With the proposed algorithm, a lane detection model trained on one dataset can be effectively applied to datasets with different camera settings in vastly different localities, and achieve better generalization ability compared to the state of the art methods.

Sensor calibration is a prerequisite for autonomous driving and is vital for accurate perception, ensuring precise planning and control of the autonomous vehicle. The modern self-driving car considers data inputs from multiple sensors to construct an understanding of its surroundings. These sensors are inherently susceptible to errors arising from measurement uncertainty, and the risk is multiplied under sensor fusion, a critical function for vehicle localization and object detection. Existing works in this field of study largely focus on offline sensor calibration methods where the vehicle is assumed to have undergone the calibration process prior to vehicle operation on the road, typically using a known calibration target such as a checkerboard pattern target. This paper proposes a sensor calibration method for the LiDAR and the inertial navigation system (INS) using high definition (HD) maps as a readily available source of ground truth, giving potential to an online approach to calibration. To match the data between the LiDAR and the INS system, the trajectory from LiDAR odometry based on the normal distributions transform (NDT) and the trajectory from the INS are utilized. The HD map acts as an online calibration target and provides real positional coordinates which can correct for errors inherent to the sensor outputs and enhance the trajectory estimations. Upon the detection of miscalibration, the algorithm finds the optimum transformation matrix that aligns the LiDAR with the INS. Real-world data was used to verify the proposed approach and can be applied to find the correct extrinsic transformation matrix.

Many researchers have used the map-matching algorithm to leverage inadequate traditional vehicle localization with HD maps for fast and efficient vehicle localization. The initialization process in the map-matching algorithm has always been problematic due to the nature of GNSS error, which might exceed the average lane width of approximately 3.5m. Thus, directly using the GNSS data to obtain the rough initial pose may cause the failure of the initialization process. The general solution is to randomly sample around the GNSS data and test the initialization with these hypotheses. However, this often leads to expensive computation and thus fails to run in realtime or online mode, as a dense sampling is required to achieve an acceptable level of initial estimation accuracy. As a viable alternative, we propose a multi-lane hypotheses approach to narrow down the search by limiting the sample pose to the number of lanes within the GNSS data's error radius. From these poses, we then perform an efficient map-matching to refine these pose candidates. Moreover, a novel belief function to evaluate the hypothesis is proposed to select the best hypothesis for system initialization robustly. Our evaluation result shows that we have outperformed the primary random sampling method in both accuracy and efficiency.

This article proposes an improved lane detection and tracking method for autonomous vehicle applications. In real applications, when the pose and position of the camera are changed, parameters and thresholds in the algorithms need fine adjustment. In order to improve adaptability to different perspective conditions, a width-adaptive lane detection method is proposed. As a useful reference to reduce noises, vanishing point is widely applied in lane detection studies. However, vanishing point detection based on original image consumes many calculation resources. In order to improve the calculation efficiency for real-time applications, we proposed a simplified vanishing point detection method. In the feature extraction step, a scan-line method is applied to detect lane ridge features, the width threshold of which is set automatically based on lane tracking. With clustering, validating, and model fitting, lane candidates are obtained from the basic ridge features. A lane-voted vanishing point is obtained by the simplified grid-based method, then applied to filter out noises. Finally, a multi-lane tracking Kalman filter is applied, the confirmed lines of which also provide adaptive width threshold for ridge feature extraction. Real-road experimental results based on our intelligent vehicle testbed proved the validity and robustness of the proposed method.

The Visual-Inertial navigation system(VINS) has become a popular navigation approach in the field of unmanned aerial vehicles(UAV) or robotics. While its performance under autonomous driving scenario is not satisfactory due to the fact that autonomous driving scenario is more challenging and dynamic than the UAV scenario. Thus, the Visual-Inertial navigation system will collapse occasionally and thus undermine the navigation result. In this work, we propose a adaptive mechanism that could switch between three modes, only VINs, only GNSS and VINS&GNSS fusion. When Visual-Inertial component breaks down, our algorithm could only rely on the GNSS signal until VINS recovers. Similarly, when GNSS signal is not very accurate, our system could only rely on the VINS-Mono. We demonstrate our algorithm under challenging scenarios such as night sight and high speed road and do both qualitative analysis and quantitative analysis.

Matching low-cost camera and vector HD map is proven to be a practical and effective way of estimating the location and orientation of intelligent vehicles. However, map-based approach is viable only when the landmark observation is adequate and precise. In some areas with sparse and noisy observation, or even non-existent map matching features, the localization results may be unstable. In this paper, we introduce a novel algorithm by fusing visual odometry and vector HD map in a tightly-coupled optimization framework to tackle these problems. Our algorithm exploits the observation of visual feature points and vector HD map landmarks in the sliding window manner and optimize their residuals in a tightly-coupled approach. In this way, the system is more robust against the noisy HD map landmark observations. In addition, our method is able to accurately estimate vehicle pose even when landmarks are sparse. Experiments under two challenging scenarios with noisy and sparse landmark observations show that our method can achieve the Mean Absolute Error (MAE) at 0.1473m and 0.2496m respectively.

Real-time vehicle localization (i.e., position and orientation estimation in the world coordinate system) with high accuracy is the fundamental function of an intelligent vehicle (IV) system. In the process of commercialization of IVs, many car manufacturers attempt to avoid high-cost sensor systems (e.g., RTK GNSS and LiDAR) in favor of low-cost optical sensors such as cameras. The same cost-saving strategy also gives rise to an increasing number of vehicles equipped with High Definition (HD) maps. Rooted upon these existing technologies, this article presents the concept of Monocular Localization with Vector HD Map (MLVHM), a novel camera-based map-matching method that efficiently aligns semantic-level geometric features in-camera acquired frames against the vector HD map in order to achieve high-precision vehicle absolute localization with minimal cost. The semantic features are delicately chosen for the ease of map vector alignment as well as for the resiliency against occlusion and fluctuation in illumination. The effective data association method in MLVHM serves as the basis for the camera position estimation by minimizing feature re-projection errors, and the frame-to-frame motion fusion is further introduced for reliable localization results. Experiments have shown that MLVHM can achieve high-precision vehicle localization with an RMSE of 24 cm with no cumulative error. In addition, we use low-cost on-board sensors and light-weight HD maps to achieve or even exceed the accuracy of existing map-matching algorithms.

Application of human pose estimation in intelligent driving has been gradually popular in recent years, as human pose could be informative in depiction of their actions and intentions. The focus of this paper is pose estimation for vulnerable road users with relatively small amounts of data. We present an improved framework, transferring general human pose estimation algorithms based on convolutional neural network to intelligent vehicle domain. Limb detection is introduced as an auxiliary task to allow explicit appearance-based modeling of spatial relationships among human keypoints. The network for keypoint detection is initialized with the parameters in limb detection task. Besides, the produced limb heatmaps are incorporated with keypoint heatmaps as extra source to gain the final keypoint locations. Concerning practical application, we adapt the widely used pose estimation networks to our proposed framework and evaluate them on the newly introduced TDCB-Pose dataset. The experiment results validate our method, and demonstrate the leverage of limb detection in human pose estimation without abundant data.

Vehicle-road cooperative perception system can greatly improve the perception ability of intelligent vehicles by making use of perception information from road side units (RSU). This paper focuses on the target positioning of static camera for vehicle-road cooperation. A low-cost camera calibration method is proposed to complete the accurate mapping between the image plane and the 3D world space. Precise location of interested targets are achieved by an efficient tracking strategy. Real test scenarios show that our algorithm can effectively locate vehicles, pedestrians, non-motor vehicles and other targets with high accuracy. Our algorithm won the Monocular Static Camera Positioning and Ranging Competition for Autonomous Driving championship in 2019.

Deep learning based lane detection algorithms inevitably suffer performance drop brought by domain shift, which usually consists of difference image appearance, spatial layout, etc. To address this problem, we propose a Common Bird-view Transformation that eliminates the lane geometric differences between training and test datasets by projecting images in both domains into a common aerial viewpoint. Experiments show that by applying our methods, the performance on test datasets can be significantly improved.

High precision localization is a crucial requirement for the autonomous driving system. Traditional positioning methods have some limitations in providing stable and accurate vehicle poses, especially in an urban environment. Herein, we propose a novel self-localizing method using a monocular camera and a 3D compact semantic map. Pre-collected information of the road landmarks is stored in a self-defined map with a minimal amount of data. We recognize landmarks using a deep neural network, followed with a geometric feature extraction process which promotes the measurement accuracy. The vehicle location and posture are estimated by minimizing a self-defined re-projection residual error to evaluate the map-to-image registration, together with a robust association method. We validate the effectiveness of our approach by applying this method to localize a vehicle in an open dataset, achieving the RMS accuracy of 0.345 meter with reduced sensor setup and map storage compared to the state of art approaches. We also evaluate some key steps and discuss the contribution of the subsystems.