Search

Lane-level localization is critical for autonomous vehicles (AVs). However, complex urban scenarios, particularly tunnels, pose significant challenges to AVs' localization systems. In this paper, we propose a fusion localization method that integrates multiple mass-production sensors, including Global Navigation Satellite Systems (GNSSs), Inertial Measurement Units (IMUs), cameras, and high-definition (HD) maps. Firstly, we use a novel electronic horizon module to assess GNSS integrity and concurrently load the HD map data surrounding the AVs. This map data are then transformed into a visual space to match the corresponding lane lines captured by the on-board camera using an improved BiSeNet. Consequently, the matched HD map data are used to correct our localization algorithm, which is driven by an extended Kalman filter that integrates multiple sources of information, encompassing a GNSS, IMU, speedometer, camera, and HD maps. Our system is designed with redundancy to handle challenging city tunnel scenarios. To evaluate the proposed system, real-world experiments were conducted on a 36-kilometer city route that includes nine consecutive tunnels, totaling near 13 km and accounting for 35% of the entire route. The experimental results reveal that 99% of lateral localization errors are less than 0.29 m, and 90% of longitudinal localization errors are less than 3.25 m, ensuring reliable lane-level localization for AVs in challenging urban tunnel scenarios. [ABSTRACT FROM AUTHOR]

Night-time vehicle detection plays a vital role due to the high incidence of abnormal events in our daily security field. However, existing studies mainly focus on vehicle detection in autonomous driving and traffic intersection scenes, but ignore urban scenes. There are vast differences between these scenes, such as viewpoint, position, illumination, etc. In this paper, the authors present a night-time vehicle detection dataset collected from urban scenes, named Vehicle Detection in Night-Time Urban Scene (VD-NUS). The VD-NUS dataset consists of more than 100 K challenging images, comprising a total of about 500 K labelled vehicles. This paper introduces a vehicle detection framework via an active auxiliary mechanism (AAM) to reduce the annotation workload. The proposed AAM framework can actively select the informative sample for annotation by estimating its uncertainty and locational instability. Furthermore, this paper proposes a computer-assisted detection module embedded in the AAM framework to help human annotators to rapidly and accurately label the selected data. AAM outperformed the baseline method (random sampling) by up to 0.91 AP and 3.0 MR−2 on the VD-NUS dataset. [ABSTRACT FROM AUTHOR]

In autonomous driving, Frequency-Modulated Continuous-Wave (FMCW) radar has gained widespread acceptance for target detection due to its resilience and dependability under diverse weather and illumination circumstances. Although deep learning radar target identification models have seen fast improvement, there is a lack of research on their susceptibility to adversarial attacks. Various spoofing attack techniques have been suggested to target radar sensors by deliberately sending certain signals through specialized devices. In this paper, we proposed a new adversarial deep learning network for spoofing attacks in radar target detection (RTD). Multi-level adversarial attack prevention using deep learning is designed for the coherence pulse deep feature map from DAALnet and Range-Doppler (RD) map from TDDLnet. After the discrimination of the attack, optimization of hybrid deep learning (OHDL) integrated with enhanced PSO is used to predict the range and velocity of the target. Simulations are performed to evaluate the sensitivity of AOHDL for different radar environment configurations. RMSE of AOHDL is almost the same as OHDL without attack conditions and it outperforms the earlier RTD implementations. [ABSTRACT FROM AUTHOR]

In-cabin occupancy detection has become increasingly important due to incidents involving children left in vehicles under extreme temperature conditions. Frequency modulated continuous wave (FMCW) radars are widely used for non-contact monitoring and sensing applications, particularly for occupancy detection. However, the confined and metallic environment inside vehicle cabins presents significant challenges due to multipath reflections. This paper introduces a novel approach that detects the occupied space in each seat to determine occupancy, using the variance of detected points as an indicator of volume occupancy. In an experimental study involving 70 different scenarios with single and multiple subjects, we classify occupants in each seat into one of three categories: adult, baby, or empty. The proposed method achieves an overall accuracy of 96.7% using an Adaboost classifier and a miss-detection rate of 1.8% for detecting babies. This approach demonstrates superior robustness to multipath interference compared to traditional energy-based methods, offering a significant advancement in in-cabin occupancy detection technology. [ABSTRACT FROM AUTHOR]

Pedestrian tracking is an important aspect of autonomous vehicles environment perception in a vehicle running environment. The performance of the existing pedestrian tracking algorithms is limited by the complex traffic environment, the changeable appearance characteristics of pedestrians and the frequent occlusion interaction, which leads to the insufficient accuracy and stability of tracking. Therefore, this paper proposes a detector–tracker integration framework for autonomous vehicle pedestrian tracking. Firstly, a pedestrian objects detector based on the improved YOLOv7 network was established. Space-to-Depth convolution layer was adopted to improve the backbone network of YOLOv7. Then, a novel appearance feature extraction network is proposed, which integrates the convolutional structural re-parameterization idea to construct a full-scale feature extraction block, which is the optimized DeepSORT tracker. Finally, experiments were carried out on MOT17 and MOT20 public datasets and driving video sequences, and the tracking performance of the proposed framework was evaluated by comparing it with the most advanced multi-object tracking algorithms. Quantitative analysis results show that the framework has high tracking accuracy. Compared with DeepSORT, MOTA improves by 2.3% in the MOT17 dataset and MOTA improves by 4.2% in the MOT20 dataset. Through qualitative evaluation on real driving video sequences, the framework proposed in this paper is robust in a variety of climate environments, and can be effectively applied to the pedestrian tracking of autonomous vehicles. [ABSTRACT FROM AUTHOR]

The signal blockage and multipath effects of the Global Navigation Satellite System (GNSS) caused by urban canyon scenarios have brought great technical challenges to the positioning and navigation of autonomous vehicles. In this paper, an improved factor graph optimization algorithm enhanced by a resilient noise model is proposed. The measurement noise is resilient and adjusted based on an approximate Gaussian distribution-based estimation. In estimating and adjusting the noise parameters of the measurement model, the error covariance matrix of the multi-sensor fusion positioning system is dynamically optimized to improve the system accuracy. Firstly, according to the approximate Gaussian statistical property of the GNSS/odometer velocity residual sequence, the measured data are divided into an approximate Gaussian fitting region and an approximate Gaussian convergence region. Secondly, the interval is divided according to the measured data, and the corresponding variational Bayesian network and Gaussian mixture model are used to estimate the innovation online. Further, the noise covariance matrix of the adaptive factor graph-based model is dynamically optimized using the estimated noise parameters. Finally, based on low-cost inertial navigation equipment, GNSS, odometer, and vision, the algorithm is implemented and verified using a simulation platform and real-vehicle road test. The experimental results show that in a complex urban road environment, compared with the traditional factor graph fusion localization algorithm, the maximum improvement in accuracy of the proposed algorithm can reach 65.63%, 39.52%, and 42.95% for heading, position, and velocity, respectively. [ABSTRACT FROM AUTHOR]

In this paper, to track maneuvering unmanned surface vehicles (USVs) in scenarios with clutter, we propose a novel method based on the probabilistic data association (PDA) algorithm and Rao-Blackwellized particle filter (RBPF) algorithm, and we further improve the tracking performance by Max-Q criterion-based waveform selection. This work develops a maneuvering target model in the context of clutter, integrating linear and nonlinear states as well as observations with false alarms. In order to jointly tackle the mixed-state tracking problem, the PDA algorithm is integrated into the RBPF framework. This allows it to be used with the complex nonlinear and linear hybrid system and helps to minimize the state dimensions of conventional particle filtering (PF). Additionally, by utilizing Q-learning principles, we provide a Max-Q-based criterion to select the waveform parameters, which guarantees low measurement errors and efficiently handles measurement uncertainties. Our simulation results show that the PDA-RBPF algorithm, which has a more appropriate tracking mechanism, produces results that are more accurate than those of the EKF or PF algorithms alone. Furthermore, the RMSE derived by the Max-Q-based criterion is smaller and more robust than that of other selection methods, as well as yielding a fixed waveform. Our proposed mechanism, which combines the concepts of PDA-RBPF and Max-Q waveform selection, performs well in target tracking tasks and exhibits relatively good performance over some existing ones. [ABSTRACT FROM AUTHOR]

High-performance radar systems are becoming increasingly popular for accurately detecting obstacles in front of unmanned vehicles in fog, snow, rain, night and other scenarios. The use of these systems is gradually expanding, such as indicating empty space and environment detection rather than just detecting and tracking the moving targets. In this paper, based on our high-resolution radar system, a three-dimensional point cloud image algorithm is developed and implemented. An axis translation and compensation algorithm is applied to minimize the point spreading caused by the different mounting positions and the alignment error of the Global Navigation Satellite System (GNSS) and radar. After applying the algorithm, a point cloud image for a corner reflector target and a parked vehicle is created to directly compare the improved results. A recently developed radar system is mounted on the vehicle and it collects data through actual road driving. Based on this, a three-dimensional point cloud image including an axis translation and compensation algorithm is created. As a results, not only the curbstones of the road but also street trees and walls are well represented. In addition, this point cloud image is made to overlap and align with an open source web browser (QtWeb)-based navigation map image to implement the imaging algorithm and thus determine the location of the vehicle. This application algorithm can be very useful for positioning unmanned vehicles in urban area where GNSS signals cannot be received due to a large number of buildings. Furthermore, sensor fusion, in which a three-dimensional point cloud radar image appears on the camera image, is also implemented. The position alignment of the sensors is realized through intrinsic and extrinsic parameter optimization. This high-performance radar application algorithm is expected to work well for unmanned ground or aerial vehicle route planning and avoidance maneuvers in emergencies regardless of weather conditions, as it can obtain detailed information on space and obstacles not only in the front but also around them. [ABSTRACT FROM AUTHOR]

Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms. [ABSTRACT FROM AUTHOR]

Autonomous vehicles are equipped with multiple heterogeneous sensors and drive while processing data from each sensor in real time. Among the sensors, the global navigation satellite system (GNSS) is essential to the localization of the vehicle itself. However, if a GNSS-denied situation occurs while driving, the accident risk may be high due to the degradation of the vehicle positioning performance. This paper presents a cooperative positioning technique based on the lidar sensor and vehicle-to-everything (V2X) communication. The ego-vehicle continuously tracks surrounding vehicles and objects, and localizes itself using tracking information from the surroundings, especially in GNSS-denied situations. We present the effectiveness of the cooperative positioning technique by constructing a GNSS-denied case during autonomous driving. A numerical simulation using a driving simulator is included in the paper to evaluate and verify the proposed method in various scenarios. [ABSTRACT FROM AUTHOR]

Mobile robots lack a driver or a pilot and, thus, should be able to detect obstacles autonomously. This paper reviews various image-based obstacle detection techniques employed by unmanned vehicles such as Unmanned Surface Vehicles (USVs), Unmanned Aerial Vehicles (UAVs), and Micro Aerial Vehicles (MAVs). More than 110 papers from 23 high-impact computer science journals, which were published over the past 20 years, were reviewed. The techniques were divided into monocular and stereo. The former uses a single camera, while the latter makes use of images taken by two synchronised cameras. Monocular obstacle detection methods are discussed in appearance-based, motion-based, depth-based, and expansion-based categories. Monocular obstacle detection approaches have simple, fast, and straightforward computations. Thus, they are more suited for robots like MAVs and compact UAVs, which usually are small and have limited processing power. On the other hand, stereo-based methods use pair(s) of synchronised cameras to generate a real-time 3D map from the surrounding objects to locate the obstacles. Stereo-based approaches have been classified into Inverse Perspective Mapping (IPM)-based and disparity histogram-based methods. Whether aerial or terrestrial, disparity histogram-based methods suffer from common problems: computational complexity, sensitivity to illumination changes, and the need for accurate camera calibration, especially when implemented on small robots. In addition, until recently, both monocular and stereo methods relied on conventional image processing techniques and, thus, did not meet the requirements of real-time applications. Therefore, deep learning networks have been the centre of focus in recent years to develop fast and reliable obstacle detection solutions. However, we observed that despite significant progress, deep learning techniques also face difficulties in complex and unknown environments where objects of varying types and shapes are present. The review suggests that detecting narrow and small, moving obstacles and fast obstacle detection are the most challenging problem to focus on in future studies. [ABSTRACT FROM AUTHOR]

Autonomous Underwater Vehicles (AUVs) are currently one of the most intensively developing branches of marine technology. Their widespread use and versatility allow them to perform tasks that, until recently, required human resources. One problem in AUVs is inadequate navigation, which results in inaccurate positioning. Weaknesses in electronic equipment lead to errors in determining a vehicle's position during underwater missions, requiring periodic reduction of accumulated errors through the use of radio navigation systems (e.g., GNSS). However, these signals may be unavailable or deliberately distorted. Therefore, in this paper, we propose a new computer vision-based method for estimating the position of an AUV. Our method uses computer vision and deep learning techniques to generate the surroundings of the vehicle during temporary surfacing at the point where it is currently located. The next step is to compare this with the shoreline representation on the map, which is generated for a set of points that are in a specific vicinity of a point determined by dead reckoning. This method is primarily intended for low-cost vehicles without advanced navigation systems. Our results suggest that the proposed solution reduces the error in vehicle positioning to 30–60 m and can be used in incomplete shoreline representations. Further research will focus on the use of the proposed method in fully autonomous navigation systems. [ABSTRACT FROM AUTHOR]

The stable detection and tracking of high-speed vehicles on the road by using LiDAR can input accurate information for the decision-making module and improve the driving safety of smart cars. This paper proposed a novel LiDAR-based robust vehicle detection method including three parts: point cloud clustering, bounding box fitting and point cloud recognition. Firstly, aiming at the problem of clustering quality degradation caused by the uneven distribution of LiDAR point clouds and the difference in clustering radius between point cloud clusters in traditional DBSCAN (TDBSCAN) obstacle clustering algorithms, an improved DBSCAN algorithm based on distance-adaptive clustering radius (ADBSCAN) is designed, and a point cloud KD-Tree data structure is constructed to speed up the traversal of the algorithm; meanwhile, the OPTICS algorithm is introduced to enhance the performance of the proposed algorithm. Then, by adopting different fitting strategies for vehicle contour points in various states, the adaptability of the bounding box fitting algorithm is improved; Moreover, in view of the shortcomings of the poor robustness of the L-shape algorithm, the principal component analysis method (PCA) is introduced to obtain stable bounding box fitting results. Finally, considering the time-consuming and low-accuracy training of traditional machine learning algorithms, advanced PointNet in deep learning technique is built to send the point cloud within the bounding box of a high-confidence vehicle into PointNet to complete vehicle recognition. Experiments based on our autonomous driving perception platform and the KITTI dataset prove that the proposed method can stably complete vehicle target recognition and achieve a good balance between time-consuming and accuracy. [ABSTRACT FROM AUTHOR]

An important research direction in the field of traffic light recognition of autonomous systems is to accurately obtain the region of interest (ROI) of the image through the multi-sensor assisted method. Dynamic evaluation of the performance of the multi-sensor (GNSS, IMU, and odometer) fusion positioning system to obtain the optimum size of the ROI is essential for further improvement of recognition accuracy. In this paper, we propose a dynamic estimation adjustment (DEA) model construction method to optimize the ROI. First, according to the residual variance of the integrated navigation system and the vehicle velocity, we divide the innovation into an approximate Gaussian fitting region (AGFR) and a Gaussian convergence region (GCR) and estimate them using variational Bayesian gated recurrent unit (VBGRU) networks and a Gaussian mixture model (GMM), respectively, to obtain the GNSS measurement uncertainty. Then, the relationship between the GNSS measurement uncertainty and the multi-sensor aided ROI acquisition error is deduced and analyzed in detail. Further, we build a dynamic estimation adjustment model to convert the innovation of the multi-sensor integrated navigation system into the optimal ROI size of the traffic lights online. Finally, we use the YOLOv4 model to detect and recognize the traffic lights in the ROI. Based on laboratory simulation and real road tests, we verify the performance of the DEA model. The experimental results show that the proposed algorithm is more suitable for the application of autonomous vehicles in complex urban road scenarios than the existing achievements. [ABSTRACT FROM AUTHOR]

High-precision and robust localization in GNSS-denied areas is crucial for autonomous vehicles and robots. Most state-of-the-art localization methods are based on simultaneous localization and mapping (SLAM) with a camera or light detection and ranging (LiDAR). However, SLAM will suffer from drift during long-term running without loop closure or prior constraints. Lightweight, high-precision environmental maps have gradually become an indispensable part of future autonomous driving. In order to solve the problem of real-time global localization for autonomous vehicles and robots, we propose a precise and robust LiDAR localization system based on a pre-built, occupied high-definition (HD) map called the Extended QuadTree (EQT) map. It makes use of a planar quadtree for block division and a Z-sequence index structure within the block cells. Then, a four-level occupancy probability cell value model is adopted. It will save about eight times the storage space compared with Google Cartographer, and the EQT map can be extended to store other information. For efficient scan-to-map matching with our specialized EQT map, the Bursa linearized model is used in the Gauss–Newton iteration of our algorithm, which makes the calculation of partial derivatives fast. All the above improvements lead to optimal storage and efficient querying for real-time scan-to-map matching localization. Field tests in an industrial park and road environment prove that positioning accuracy of about 6–13 cm and attitude accuracy of about 0.15° were achieved using a VLP-16 LiDAR. They also show that the method proposed in this paper is significantly better than the NDT method. For the long and narrow environment of an underground mine tunnel, high-resolution maps are also helpful for accurate and robust localization. [ABSTRACT FROM AUTHOR]

Motorcycle detection and collision warning are essential features in advanced driver assistance systems (ADAS) to ensure road safety, especially in emergency situations. However, detecting motorcycles from videos captured from a car is challenging due to the varying shapes and appearances of motorcycles. In this paper, we propose an integrated and innovative remote sensing and artificial intelligence (AI) methodology for motorcycle detection and distance estimation based on visual data from a single camera installed in the back of a vehicle. Firstly, MD-TinyYOLOv4 is used for detecting motorcycles, refining the neural network through SPP (spatial pyramid pooling) feature extraction, Mish activation function, data augmentation techniques, and optimized anchor boxes for training. The proposed algorithm outperforms eight existing YOLO versions, achieving a precision of 81% at a speed of 240 fps. Secondly, a refined disparity map of each motorcycle's bounding box is estimated by training a Monodepth2 with a bilateral filter for distance estimation. The proposed fusion model (motorcycle's detection and distance from vehicle) is evaluated with depth stereo camera measurements, and the results show that 89% of warning scenes are correctly detected, with an alarm notification time of 0.022 s for each image. Outcomes indicate that the proposed integrated methodology provides an effective solution for ADAS, with promising results for real-world applications, and can be suitable for running on mobility services or embedded computing boards instead of the super expensive and powerful systems used in some high-tech unmanned vehicles. [ABSTRACT FROM AUTHOR]

LiDAR-based Multi-Object Tracking (MOT) is a critical technology employed in various autonomous systems, including self-driving vehicles and autonomous delivery robots. In this paper, a novel LiDAR-based 3D MOT approach is introduced. The proposed method was built upon the Tracking-by-Detection (TbD) paradigm and incorporated multi-level associations that exploit an object's short-term and long-term relationships with the existing tracks. Specifically, the short-term association leverages the fact that objects do not move much between consecutive frames. In contrast, the long-term association assesses the degree to which a long-term trajectory aligns with current detections. The evaluation of the matching between the current detection and the maintained trajectory was performed using a Graph Convolutional Network (GCN). Furthermore, an inactive track was maintained to address the issue of incorrect ID switching for objects that have been occluded for an extended period. The proposed method was evaluated on the KITTI benchmark MOT tracking dataset and achieved a Higher-Order Tracking Accuracy (HOTA) of 75.65%, marking a 5.66% improvement over the benchmark method AB3DMOT, while also accomplishing the number of ID switches of 39, 74 less than AB3DMOT. These results confirmed the effectiveness of the proposed approach in diverse road environments. [ABSTRACT FROM AUTHOR]

This paper demonstrates the weakness of GNSS/INS-RTK (GIR) systems in mapping challenging environments because of obstruction and deflection of satellite signals. Thus, it emphasizes that the strategy of mapping companies to commercially provide maps using expensive GIR systems does not always work robustly. This limits the scalability of autonomous vehicle deployment in many road structures and modern cities. Accordingly, different critical environments in Tokyo have been analyzed and investigated to demonstrate the effects of the road structure complexity on the GIR map quality with highlighting the relevant reasons. Therefore, this paper is intended to be a reference to prove that the data of GIR systems cannot always be considered as ground truth and the integration of SLAM technologies into the mapping modules is very necessary to enable the levels four and five of autonomous driving. [ABSTRACT FROM AUTHOR]

For emergency rescue and damage assessment after an earthquake, quick detection of seismic landslides in the affected areas is crucial. The purpose of this study is to quickly determine the extent and size of post-earthquake seismic landslides using a small amount of post-earthquake seismic landslide imagery data. This information will serve as a foundation for emergency rescue efforts, disaster estimation, and other actions. In this study, Wenchuan County, Sichuan Province, China's 2008 post-quake Unmanned Air Vehicle (UAV) remote sensing images are used as the data source. ResNet-50, ResNet-101, and Swin Transformer are used as the backbone networks of Mask R-CNN to train and identify seismic landslides in post-quake UAV images. The training samples are then augmented by data augmentation methods, and transfer learning methods are used to reduce the training time required and enhance the generalization of the model. Finally, transfer learning was used to apply the model to seismic landslide imagery from Haiti after the earthquake that was not calibrated. With Precision and F1 scores of 0.9328 and 0.9025, respectively, the results demonstrate that Swin Transformer performs better as a backbone network than the original Mask R-CNN, YOLOv5, and Faster R-CNN. In Haiti's post-earthquake images, the improved model performs significantly better than the original model in terms of accuracy and recognition. The model for identifying post-earthquake seismic landslides developed in this paper has good generalizability and transferability as well as good application potential in emergency responses to earthquake disasters, which can offer strong support for post-earthquake emergency rescue and disaster assessment. [ABSTRACT FROM AUTHOR]

In the field of autonomous driving and robotics, 3D object detection is a difficult, but important task. To improve the accuracy of detection, LiDAR, which collects the 3D point cloud of a scene, is updated constantly. But the density of the collected 3D points is low, and its distribution is unbalanced in the scene, which influences the accuracy of 3D object detectors in regards to object location and identification. Although corresponding high-resolution scene images from cameras can be used as supplemental information, poor fusion strategies can result in decreased accuracy compared with that of LiDAR-point-only detectors. Thus, to improve the detection performance for the classification, localization, and even boundary location of 3D objects, a two-stage detector with density-and-sparsity feature aggregation, called DASANet, is proposed in this paper. In the first stage, dense pseudo point clouds are generated with images from cameras and are used to obtain the initial proposals. In the second stage, two novel feature aggregation modules are designed to fuse LiDAR point information and pseudo point information, which refines the semantic and detailed representation of the feature maps. To supplement the semantic information of the highest-scale LiDAR features for object localization and classification, a triple differential information supplement (TDIS) module is presented to extract the LiDAR-pseudo differential features and enhance them in spatial, channel, and global dimensions. To increase the detailed information of the LiDAR features for object boundary location, a Siamese three-dimension coordinate attention (STCA) module is presented to extract stable LiDAR and pseudo point cloud features with a Siamese encoder and fuse these features using a three-dimension coordinate attention. Experiments using the KITTI Vision Benchmark Suite demonstrate the improved performance of our DASANet in regards to the localization and boundary location of objects. The ablation studies demonstrate the effectiveness of the TDIS and the STCA modules. [ABSTRACT FROM AUTHOR]

UAVs represent a tremendous opportunity to perform geophysical and repeated experiments, particularly in volcanic contexts. Their ability to be deployed rapidly and fly at various altitudes and the fact that they are easy to operate despite complex field conditions make them attractive for magnetic surveys. Detailed maps of the magnetic field in turn bring key constraints on the rocks' composition, thermal anomalies, intrusive systems, and crustal contrast evolution. Yet, raw magnetic field measurements require careful processing to minimize directional, positional, and crossover errors. Moreover, stitching together adjacent or overlapping surveys acquired at different times and altitudes is not a trivial task. Therefore, it is challenging in remote areas to directly evaluate the consistency of a survey and to ascertain the success of the field mission. In this paper, we present a fast algorithm allowing for a quick-look modeling of scalar magnetic intensity measurements. The approach relies on rectangular harmonic analysis (RHA). The field measurements are automatically corrected for a global main field. Then, they are projected along this main field and modeled in terms of RHA functions. The software can exploit the quality indices provided with data and a procedure is applied to mitigate the effect of outliers. Maps for the scalar and the vector anomaly fields are readily built on an interpolated regular grid leveled at a constant altitude. In order to assess the modeling and the inversion procedures, analyses are carried out with synthetic measurements derived from a high-resolution global lithospheric magnetic field model estimated on the French aeromagnetic grid and at UAV locations with some added nonrandom noise. These analyses indicate that RHA is efficient for first-order and direct mapping of the crustal magnetic field structures measured by UAVs but that it could be applied on airborne and marine magnetic intensity data covering dense and large geographical extensions. [ABSTRACT FROM AUTHOR]

Microwave radar has advantages in detection accuracy and robustness, and it is an area of active research in unmanned ground vehicles. However, the existing conventional automotive corner radar, which employs real-aperture antenna arrays, has limitations in terms of observable angle and azimuthal resolution. This paper proposes a novel 3D ArcSAR method to address this issue, which combines rotational synthetic aperture radar (SAR) and direction estimation algorithms. The method aims to reconstruct 3D images of 360° scenes and offers distinctive advantages in both azimuthal and altitudinal sensing. Nevertheless, due to the unique structural characteristics of vehicle SAR, it is limited to receiving only a single snapshot signal for 3D sensing. We propose a resolution algorithm based on ArcSAR and the iterative adaptive approach (IAA) to resolve the limitation. Furthermore, the errors in altitude angle estimation of the proposed algorithm and conventional algorithms are analyzed under various conditions, including different target spacing and signal-to-noise ratio (SNR). Finally, we design and implement a prototype of the 3D ArcSAR sensing system, which utilizes a millimeter-wave MIMO radar system and a rotating scanning mechanical system. The experimental results obtained from this prototype effectively validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Simultaneous Localization and Mapping (SLAM) forms the foundation of vehicle localization in autonomous driving. Utilizing high-precision 3D scene maps as prior information in vehicle localization greatly assists in the navigation of autonomous vehicles within large-scale 3D scene models. However, generating high-precision maps is complex and costly, posing challenges to commercialization. As a result, a global localization system that employs low-precision, city-scale 3D scene maps reconstructed by unmanned aerial vehicles (UAVs) is proposed to optimize visual positioning for vehicles. To address the discrepancies in image information caused by differing aerial and ground perspectives, this paper introduces a wall complementarity algorithm based on the geometric structure of buildings to refine the city-scale 3D scene. A 3D-to-3D feature registration algorithm is developed to determine vehicle location by integrating the optimized city-scale 3D scene with the local scene generated by an onboard stereo camera. Through simulation experiments conducted in a computer graphics (CG) simulator, the results indicate that utilizing a completed low-precision scene model enables achieving a vehicle localization accuracy with an average error of 3.91 m, which is close to the 3.27 m error obtained using the high-precision map. This validates the effectiveness of the proposed algorithm. The system demonstrates the feasibility of utilizing low-precision city-scale 3D scene maps generated by unmanned aerial vehicles (UAVs) for vehicle localization in large-scale scenes. [ABSTRACT FROM AUTHOR]

This paper presents a target tracking algorithm based on 4D millimeter-wave radar point cloud information for autonomous driving applications, which addresses the limitations of traditional 2 + 1D radar systems by using higher resolution target point cloud information that enables more accurate motion state estimation and target contour information. The proposed algorithm includes several steps, starting with the estimation of the ego vehicle's velocity information using the radial velocity information of the millimeter-wave radar point cloud. Different clustering suggestions are then obtained using a density-based clustering method, and correlation regions of the targets are obtained based on these clustering suggestions. The binary Bayesian filtering method is then used to determine whether the targets are dynamic or static targets based on their distribution characteristics. For dynamic targets, Kalman filtering is used to estimate and update the state of the target using trajectory and velocity information, while for static targets, the rolling ball method is used to estimate and update the shape contour boundary of the target. Unassociated measurements are estimated for the contour and initialized for the trajectory, and unassociated trajectory targets are selectively retained and deleted. The effectiveness of the proposed method is verified using real data. Overall, the proposed target tracking algorithm based on 4D millimeter-wave radar point cloud information has the potential to improve the accuracy and reliability of target tracking in autonomous driving applications, providing more comprehensive motion state and target contour information for better decision making. [ABSTRACT FROM AUTHOR]

Autonomous vehicles (AVs) are predicted to change transportation; however, it is still difficult to maintain robust situation awareness in a variety of driving situations. To enhance AV perception, methods to integrate sensor data from the camera, radar, and LiDAR sensors have been proposed. However, due to rigidity in their fusion implementations, current techniques are not sufficiently robust in challenging driving scenarios (such as inclement weather, poor light, and sensor obstruction). These techniques can be divided into two main groups: (i) early fusion, which is ineffective when sensor data are distorted or noisy, and (ii) late fusion, which is unable to take advantage of characteristics from numerous sensors and hence yields sub-optimal estimates. In this paper, we suggest a flexible selective sensor fusion framework that learns to recognize the present driving environment and fuses the optimum sensor combinations to enhance robustness without sacrificing efficiency to overcome the above-mentioned limitations. The proposed framework dynamically simulates early fusion, late fusion, and mixtures of both, allowing for a quick decision on the best fusion approach. The framework includes versatile modules for pre-processing heterogeneous data such as numeric, alphanumeric, image, and audio data, selecting appropriate features, and efficiently fusing the selected features. Further, versatile object detection and classification models are proposed to detect and categorize objects accurately. Advanced ensembling, gating, and filtering techniques are introduced to select the optimal object detection and classification model. Further, innovative methodologies are proposed to create an accurate context and decision rules. Widely used datasets like KITTI, nuScenes, and RADIATE are used in experimental analysis to evaluate the proposed models. The proposed model performed well in both data-level and decision-level fusion activities and also outperformed other fusion models in terms of accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Geolocation is a fundamental component of route planning and navigation for unmanned vehicles, but GNSS-based geolocation fails under denial-of-service conditions. Cross-view geo-localization (CVGL), which aims to estimate the geographic location of the ground-level camera by matching against enormous geo-tagged aerial (e.g., satellite) images, has received a lot of attention but remains extremely challenging due to the drastic appearance differences across aerial–ground views. In existing methods, global representations of different views are extracted primarily using Siamese-like architectures, but their interactive benefits are seldom taken into account. In this paper, we present a novel approach using cross-view knowledge generative techniques in combination with transformers, namely mutual generative transformer learning (MGTL), for CVGL. Specifically, by taking the initial representations produced by the backbone network, MGTL develops two separate generative sub-modules—one for aerial-aware knowledge generation from ground-view semantics and vice versa—and fully exploits the entirely mutual benefits through the attention mechanism. Moreover, to better capture the co-visual relationships between aerial and ground views, we introduce a cascaded attention masking algorithm to further boost accuracy. Extensive experiments on challenging public benchmarks, i.e., CVACT and CVUSA, demonstrate the effectiveness of the proposed method, which sets new records compared with the existing state-of-the-art models. Our code will be available upon acceptance. [ABSTRACT FROM AUTHOR]

Autonomous navigation (AN) and manoeuvring are increasingly important in distributed satellite systems (DSS) in order to avoid potential collisions with space debris and other resident space objects (RSO). In order to accomplish collision avoidance manoeuvres, tracking and characterization of RSO is crucial. At present, RSO are tracked and catalogued using ground-based observations, but space-based space surveillance (SBSS) represents a valid alternative (or complementary asset) due to its ability to offer enhanced performances in terms of sensor resolution, tracking accuracy, and weather independence. This paper proposes a particle swarm optimization (PSO) algorithm for DSS AN and manoeuvring, specifically addressing RSO tracking and collision avoidance requirements as an integral part of the overall system design. More specifically, a DSS architecture employing hyperspectral sensors for Earth observation is considered, and passive electro-optical sensors are used, in conjunction with suitable mathematical algorithms, to accomplish autonomous RSO tracking and classification. Simulation case studies are performed to investigate the tracking and system collision avoidance capabilities in both space-based and ground-based tracking scenarios. Results corroborate the effectiveness of the proposed AN technique and highlight its potential to supplement either conventional (ground-based) or SBSS tracking methods. [ABSTRACT FROM AUTHOR]

Autonomous driving is becoming a pivotal technology that can realize intelligent transportation and revolutionize the future of mobility. Various types of sensors, including perception sensors and localization sensors, are essential for high-level autonomous and intelligent vehicles (AIV). In this paper, the characteristics of different sensors are compared, and the application characteristics and requirements of AIV are analyzed in depth. These analyses indicate that: GNSS, as the unique localization sensor that can obtain an absolute position, can not only provide all-weather position and time information for internal multi-sensor fusion but also act as a standard spatiotemporal reference for all autonomous systems; Furthermore, AIVs aim to provide safety for a mass user base ranging from tens to hundreds of millions; for this, AIVs require a global wide-area and instantaneous precise positioning service with location privacy protection. Based on a "geometry-bound" description of road grade and vehicle size, it has been found that GNSS requirements in autonomous vehicles include decimeter-level positioning with the assurance of high integrity. Combined with high-integrity GNSS implementation in the civil aviation field, GNSS different technology routes, and commercial solutions, a state space representation (SSR)-based GNSS high-precision augmentation positioning solution for AIV is summarized and introduced. The solution can achieve instantaneous, precise positioning with high integrity in a wide area by utilizing passive positioning mode with location privacy protection. In addition, the research progress on key technologies in the solution and existing challenges is investigated in detail by reviewing a series of publications. [ABSTRACT FROM AUTHOR]

Autonomous driving has received enormous attention from the academic and industrial communities. However, achieving full driving autonomy is not a trivial task, because of the complex and dynamic driving environment. Perception ability is a tough challenge for autonomous driving, while 3D object detection serves as a breakthrough for providing precise and dependable 3D geometric information. Inspired by practical driving experiences of human experts, a pure visual scheme takes sufficient responsibility for safe and stable autonomous driving. In this paper, we proposed an anchor-free and keypoint-based 3D object detector with monocular vision, named Keypoint3D. We creatively leveraged 2D projected points from 3D objects' geometric centers as keypoints for object modeling. Additionally, for precise keypoints positioning, we utilized a novel self-adapting ellipse Gaussian filter (saEGF) on heatmaps, considering different objects' shapes. We tried different variations of DLA-34 backbone and proposed a semi-aggregation DLA-34 (SADLA-34) network, which pruned the redundant aggregation branch but achieved better performance. Keypoint3D regressed the yaw angle in a Euclidean space, which resulted in a closed mathematical space avoiding singularities. Numerous experiments on the KITTI dataset for a moderate level have proven that Keypoint3D achieved the best speed-accuracy trade-off with an average precision of 39.1 % at 18.9 FPS on 3D cars detection. [ABSTRACT FROM AUTHOR]

Free-space detection plays a pivotal role in autonomous vehicle applications, and its state-of-the-art algorithms are typically based on semantic segmentation of road areas. Recently, hyperspectral images have proven useful supplementary information in multi-modal segmentation for providing more texture details to the RGB representations, thus performing well in road segmentation tasks. Existing multi-modal segmentation methods assume that all the inputs are well-aligned, and then the problem is converted to fuse feature maps from different modalities. However, there exist cases where sensors cannot be well-calibrated. In this paper, we propose a novel network named multi-modal cross-attention network (MMCAN) for multi-modal free-space detection with uncalibrated hyperspectral sensors. We first introduce a cross-modality transformer using hyperspectral data to enhance RGB features, then aggregate these representations alternatively via multiple stages. This transformer promotes the spread and fusion of information between modalities that cannot be aligned at the pixel level. Furthermore, we propose a triplet gate fusion strategy, which can increase the proportion of RGB in the multiple spectral fusion processes while maintaining the specificity of each modality. The experimental results on a multi-spectral dataset demonstrate that our MMCAN model has achieved state-of-the-art performance. The method can be directly used on the pictures taken in the field without complex preprocessing. Our future goal is to adapt the algorithm to multi-object segmentation and generalize it to other multi-modal combinations. [ABSTRACT FROM AUTHOR]

LiDAR-based semantic segmentation, particularly for unstructured environments, plays a crucial role in environment perception and driving decisions for unmanned ground vehicles. Unfortunately, chaotic unstructured environments, especially the high-proportion drivable areas and large-area static obstacles therein, inevitably suffer from the problem of blurred class edges. Existing published works are prone to inaccurate edge segmentation and have difficulties dealing with the above challenge. To this end, this paper proposes a real-time edge-guided LiDAR semantic segmentation network for unstructured environments. First, the main branch is a lightweight architecture that extracts multi-level point cloud semantic features; Second, the edge segmentation module is designed to extract high-resolution edge features using cascaded edge attention blocks, and the accuracy of extracted edge features and the consistency between predicted edge and semantic segmentation results are ensured by additional supervision; Third, the edge guided fusion module fuses edge features and main branch features in a multi-scale manner and recalibrates the channel feature using channel attention, realizing the edge guidance to semantic segmentation and further improving the segmentation accuracy and adaptability of the model. Experimental results on the SemanticKITTI dataset, the Rellis-3D dataset, and on our test dataset demonstrate the effectiveness and real-time performance of the proposed network in different unstructured environments. Especially, the network has state-of-the-art performance in segmentation of drivable areas and large-area static obstacles in unstructured environments. [ABSTRACT FROM AUTHOR]

Autonomous vehicles and smart roads are not new concepts and the undergoing development to empower the vehicles for higher levels of automation has achieved initial milestones. However, the transportation industry and relevant research communities still require making considerable efforts to create smart and intelligent roads for autonomous driving. To achieve the results of such efforts, the CCAM infrastructure is a game changer and plays a key role in achieving higher levels of autonomous driving. In this paper, we present a smart infrastructure and autonomous driving capabilities enhanced by CCAM infrastructure. Meaning thereby, we lay down the technical requirements of the CCAM infrastructure: identify the right set of the sensory infrastructure, their interfacing, integration platform, and necessary communication interfaces to be interconnected with upstream and downstream solution components. Then, we parameterize the road and network infrastructures (and automated vehicles) to be advanced and evaluated during the research work, under the very distinct scenarios and conditions. For validation, we demonstrate the machine learning algorithms in mobility applications such as traffic flow and mobile communication demands. Consequently, we train multiple linear regression models and achieve accuracy of over 94% for predicting aforementioned demands on a daily basis. This research therefore equips the readers with relevant technical information required for enhancing CCAM infrastructure. It also encourages and guides the relevant research communities to implement the CCAM infrastructure towards creating smart and intelligent roads for autonomous driving. [ABSTRACT FROM AUTHOR]

This paper presents a novel technique to estimate DCBs from GPS transmitters and receivers on-board Low Earth Orbit (LEO) satellites. The technique consists of obtaining the DCBs as residuals from the difference between the ionospheric combination of the code and the associated ionospheric delay. The ionospheric delay is computed with TOMION, a background-model-free ionospheric tomographic technique based on dual-frequency GPS carrier phase data only, and solved with a Kalman filter. Thus, DCBs are also estimated epoch-wise from the LEO Precise Orbit Determination (POD) GPS receiver as a secondary product. The results for GPS satellite DCBs, obtained exclusively from the three MetOp LEO POD GPS receivers over four consecutive weeks, are in full agreement (i.e., at the level of a few tenths of ns) with those reported independently with other techniques from hundreds of ground-based receivers exclusively, by JPL and CODE analysis centers. [ABSTRACT FROM AUTHOR]

Simultaneous localization and mapping (SLAM) is the key technology for the automation of intelligent mining equipment and the digitization of the mining environment. However, the shotcrete surface and symmetrical roadway in underground coal mines make light detection and ranging (LiDAR) SLAM prone to degeneration, which leads to the failure of mobile robot localization and mapping. To address these issues, this paper proposes a robust LiDAR SLAM method which detects and compensates for the degenerated scenes by integrating LiDAR and inertial measurement unit (IMU) data. First, the disturbance model is used to detect the direction and degree of degeneration caused by insufficient line and plane feature constraints for obtaining the factor and vector of degeneration. Second, the degenerated state is divided into rotation and translation. The pose obtained by IMU pre-integration is projected to plane features and then used for local map matching to achieve two-step degenerated compensation. Finally, a globally consistent LiDAR SLAM is implemented based on sliding window factor graph optimization. The extensive experimental results show that the proposed method achieves better robustness than LeGO-LOAM and LIO-SAM. The absolute position root mean square error (RMSE) is only 0.161 m, which provides an important reference for underground autonomous localization and navigation in intelligent mining and safety inspection. [ABSTRACT FROM AUTHOR]

3D Mapping-Aided (3DMA) Global Navigation Satellite System (GNSS) is a widely used method to mitigate multipath errors. Various research has been presented which utilizes 3D building model data in conjunction with ray-tracing algorithms to compute and predict satellites' visibility conditions and compute delays caused by signal reflection. To simulate, model and potentially correct multipath errors in highly dynamic applications, such as, e.g., autonomous driving, the satellite–receiver–reflector geometry has to be known precisely in a common reference frame. Three-dimensional building models are often provided by regional public or private services and the coordinate information is usually given in a coordinate system of a map projection. Inconsistencies in the coordinate frames used to express the satellite and user coordinates, as well as the reflector surfaces, lead to falsely determined multipath errors and, thus, reduce the performance of 3DMA GNSS. This paper aims to provide the needed transformation steps to consider when integrating 3D building model data, user position, and GNSS orbit information. The impact of frame inconsistencies on the computed extra path delay is quantified based on a simulation study in a local 3D building model; they can easily amount to several meters. Differences between the extra path-delay computations in a metric system and a map projection are evaluated and corrections are proposed to both variants depending on the accuracy needs and the intended use. [ABSTRACT FROM AUTHOR]

The detection of drivable areas in off-road scenes is a challenging problem due to the presence of unstructured class boundaries, irregular features, and dust noise. Three-dimensional LiDAR data can effectively describe the terrain features, and a bird's eye view (BEV) not only shows these features, but also retains the relative size of the environment compared to the forward viewing. In this paper, a method called LRTI, which is used for detecting drivable areas based on the texture information of LiDAR reflection data, is proposed. By using an instance segmentation network to learn the texture information, the drivable areas are obtained. Furthermore, a multi-frame fusion strategy is applied to improve the reliability of the output, and a shelter's mask of a dynamic object is added to the neural network to reduce the perceptual delay caused by multi-frame fusion. Through TensorRT quantization, LRTI achieves real-time processing on the unmanned ground vehicle (UGV). The experiments on our dataset show the robustness and adaptability of LRTI to sand dust and occluded scenes. [ABSTRACT FROM AUTHOR]

Simultaneous localization and mapping (SLAM) algorithm is a prerequisite for unmanned ground vehicle (UGV) localization, path planning, and navigation, which includes two essential components: frontend odometry and backend optimization. Frontend odometry tends to amplify the cumulative error continuously, leading to ghosting and drifting on the mapping results. However, loop closure detection (LCD) can be used to address this technical issue by significantly eliminating the cumulative error. The existing LCD methods decide whether a loop exists by constructing local or global descriptors and calculating the similarity between descriptors, which attaches great importance to the design of discriminative descriptors and effective similarity measurement mechanisms. In this paper, we first propose novel multi-channel descriptors (CMCD) to alleviate the lack of point cloud single information in the discriminative power of scene description. The distance, height, and intensity information of the point cloud is encoded into three independent channels of the shadow-casting region (bin) and then compressed it into a two-dimensional global descriptor. Next, an ORB-based dynamic threshold feature extraction algorithm (DTORB) is designed using objective 2D descriptors to describe the distributions of global and local point clouds. Then, a DTORB-based similarity measurement method is designed using the rotation-invariance and visualization characteristic of descriptor features to overcome the subjective tendency of the constant threshold ORB algorithm in descriptor feature extraction. Finally, verification is performed over KITTI odometry sequences and the campus datasets of Jilin University collected by us. The experimental results demonstrate the superior performance of our method to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This paper proposes a reliable framework to map multilevel road structures in the 2D image domain called layered sub-image maps (LSM). The road is divided into a set of sub-areas providing IDs in the XY plane. Each sub-area is decomposed into several layered images using LIDAR intensity and elevation data to form a 2.5D map image. The layered elevation images are given IDs in the Z plane to represent the height of the contained road features in meter-order whereas the elevation pixels indicate the cm-order of the road slope in the range of 200 cm. The layered intensity images are then created to describe the road surface in conjunction with the number of the layered elevation images and the corresponding pixel distributions. A significant map retrieval strategy during autonomous driving has been designed based on the LSM implementation tactic and the IDs in the XYZ plane. The system's reliability has been proved by a unique localization module to localize an autonomous vehicle in a challenging multilevel environment consisting of four stacked loops with an average accuracy of 5 cm in lateral, longitudinal and altitudinal directions. [ABSTRACT FROM AUTHOR]

In order to realize the cooperative localization of multi-unmanned platforms in the GNSS-denied environment, this paper proposes a collaborative SLAM (simultaneous localization and mapping, SLAM) framework based on image feature point matching. Without GNSS, a single unmanned platform UGV and UAV (unmanned ground vehicle, UGV; unmanned aerial vehicle, UAV) equipped with vision and IMU (inertial measurement unit, IMU) sensors can exchange information through data communication to jointly build a three-dimensional visual point map, and determine the relative position of each other through visual-based position re- identification and PnP (Perspective-n-Points, PnP) methods. When any agent can receive reliable GNSS signals, GNSS positioning information will greatly improve the positioning accuracy without changing the positioning algorithm framework. In order to achieve this function, we designed a set of two-stage position estimation algorithms. In the first stage, we used the modified ORB-SLAM3 algorithm for position estimation by fusing visual and IMU information. In the second stage, we integrated GNSS positioning and cooperative positioning information using the factor graph optimization (FGO) algorithm. Our framework consists of an UGV as the central server node and three UAVs carried by the UGV, that will collaborate on space exploration missions. Finally, we simulated the influence of different visibility and lighting conditions on the framework function on the virtual simulation experiment platform built based on ROS (robot operating system, ROS) and Unity3D. The accuracy of the cooperative localization algorithm and the single platform localization algorithm was evaluated. In the two cases of GNSS-denied and GNSS-challenged, the error of co-location reduced by 15.5% and 19.7%, respectively, compared with single-platform independent positioning. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) point cloud maps are widely used in autonomous driving scenarios. These maps are usually generated by accumulating sequential LiDAR scans. When generating a map, moving objects (such as vehicles or moving pedestrians) will leave long trails on the assembled map. This is undesirable and reduces the map quality. In this paper, we propose MapCleaner, an approach that can effectively remove the moving objects from the map. MapCleaner first estimates a dense and continuous terrain surface, based on which the map point cloud is then divided into a noisy part below the terrain, the terrain, and the object part above the terrain. Next, a specifically designed moving points identification algorithm is performed on the object part to find moving objects. Experiments are performed on the SemanticKITTI dataset. Results show that the proposed MapCleaner outperforms state-of-the-art approaches on all five tested SemanticKITTI sequences. MapCleaner is a learning-free method and has few parameters to tune. It is also successfully evaluated on our own dataset collected with a different type of LiDAR. [ABSTRACT FROM AUTHOR]

Location awareness is crucial for underwater applications; without it, gathered data would be essentially useless. However, it is impossible to directly determine the location of an underwater target because GPS-reliant methods cannot be utilized in the underwater environment. To this end, the underwater target localization technique has become one of the most critical technologies in underwater applications, wherein GPS-equipped autonomous surface vehicles (ASVs) are typically used to assist with localization. It has been proved that, under the assumption of Gaussian noise, an appropriate geometry among ASVs and the underwater target can enhance localization accuracy. Unfortunately, the conclusion may not hold if outliers arise and the closed-form expression of Cramér–Rao lower bound (CRLB) cannot be established. Eventually, it becomes hard to derive the accepted geometry, particularly for the received signal strength (RSS)-based ranging scenario. Therefore, this work optimizes the trajectory of ASVs with RSS-based ranging and in the presence of outliers to optimally estimate the location of an underwater target. The D-optimality criterion is applied in conjunction with the Monte Carlo method to determine the closed-form expression of the function, which then transforms the problem into an optimized framework. Nevertheless, the framework cannot be solved in the absence of the target location. In this case, the paper presents two methodologies to overcome the issue and achieve the optimum configuration without identifying the target location. (1) A min–max strategy that assumes that the target location drops in an uncertain region for a single or two ASVs is proposed; and (2) a two-phase localization approach (TPLA) that calculates the target location at each time slot for three ASVs is developed. Finally, the optimal trajectories of ASVs are constructed by a series of waypoints based on an analytically tractable measurement model in various conditions. [ABSTRACT FROM AUTHOR]

Completing point clouds from partial terrestrial laser scannings (TLS) is a fundamental step for many 3D visual applications, such as remote sensing, digital city and autonomous driving. However, existing methods mainly followed an ordinary auto-encoder architecture with only partial point clouds as inputs, and adopted K-Nearest Neighbors (KNN) operations to extract local geometric features, which takes insufficient advantage of input point clouds and has limited ability to extract features from long-range geometric relationships, respectively. In this paper, we propose a keypoints-aligned siamese (KASiam) network for the completion of partial TLS point clouds. The network follows a novel siamese auto-encoder architecture, to learn prior geometric information of complete shapes by aligning keypoints of complete-partial pairs during the stage of training. Moreover, we propose two essential blocks cross-attention perception (CAP) and self-attention augment (SAA), which replace KNN operations with attention mechanisms and are able to establish long-range geometric relationships among points by selecting neighborhoods adaptively at the global level. Experiments are conducted on widely used benchmarks and several TLS data, which demonstrate that our method outperforms other state-of-the-art methods by a 4.72% reduction of the average Chamfer Distance of categories in PCN dataset at least, and can generate finer shapes of point clouds on partial TLS data. [ABSTRACT FROM AUTHOR]

Multi-spectral semantic segmentation has shown great advantages under poor illumination conditions, especially for remote scene understanding of autonomous vehicles, since the thermal image can provide complementary information for RGB image. However, methods to fuse the information from RGB image and thermal image are still under-explored. In this paper, we propose a simple but effective module, add–multiply fusion (AMFuse) for RGB and thermal information fusion, consisting of two simple math operations—addition and multiplication. The addition operation focuses on extracting cross-modal complementary features, while the multiplication operation concentrates on the cross-modal common features. Moreover, the attention module and atrous spatial pyramid pooling (ASPP) modules are also incorporated into our proposed AMFuse modules, to enhance the multi-scale context information. Finally, in the UNet-style encoder–decoder framework, the ResNet model is adopted as the encoder. As for the decoder part, the multi-scale information obtained from our proposed AMFuse modules is hierarchically merged layer-by-layer to restore the feature map resolution for semantic segmentation. The experiments of RGBT multi-spectral semantic segmentation and salient object detection demonstrate the effectiveness of our proposed AMFuse module for fusing the RGB and thermal information. [ABSTRACT FROM AUTHOR]

In this paper, we present a challenging stereo-inertial dataset collected onboard a sports utility vehicle (SUV) for the tasks of visual-inertial odometry (VIO), simultaneous localization and mapping (SLAM), autonomous driving, object detection, and other computer vision techniques. We recorded a large set of time-synchronized stereo image sequences (2 × 1280 × 720 @ 30 fps RGB) and corresponding inertial measurement unit (IMU) readings (400 Hz) from a Stereolabs ZED2 camera, along with centimeter-level-accurate six-degree-of-freedom ground truth (100 Hz) from a u-blox GNSS-IMU navigation device with real-time kinematic correction signals. The dataset comprises 34 sequences recorded during November 2020 in Wuhan, the largest city of Central China. Further, the dataset contains abundant unique urban scenes and features of a complex modern metropolis, which have rarely appeared in previously released benchmarks. Results from milestone VIO/SLAM algorithms reveal that methods exhibiting excellent performance on established datasets such as KITTI and EuRoC perform unsatisfactorily when moved outside the laboratory to the real world. We expect our dataset to reduce this limitation by providing more challenging and diverse scenarios to the research community. The full dataset with raw and calibrated data is publicly available along with a lightweight MATLAB/Python toolbox for preprocessing and evaluation. The dataset can be downloaded in its entirety from the uniform resource locator (URL) we provide in the main text. [ABSTRACT FROM AUTHOR]

Lane localization is one of the core tasks in an autonomous driving system. It receives the visual information collected by the camera and the lane marks and road edges information outputted from the perception module and gives lane index for the subsequent decision module. Traditional rule-based lane localization methods using navigation maps can only be effective in regular road scenarios and have poor generalization ability. High-Definition Map (HD map) was originally thought to solve the lane localization problem, but due to the regulations of the relevant departments, HD map is currently not allowed to be used in autonomous driving systems. In addition, many multi-sensor fusion methods have been proposed to solve the lane localization problem. However, due to the extremely strict safety requirements of autonomous driving systems, these well-designed solutions make it difficult to meet the requirements in terms of robustness, efficiency, and stability. To solve these problems, we innovatively define the lane localization task as a classification problem. First, to better utilize the perceptual information outputted from the perceptual model, we design an image-generating method that projects the perceptual information onto a new image and ensures that our model can learn the perceptual features wisely. Second, to better fuse the perceptual and visual information, we propose a fusion structure deep learning neural network named LLNet to address the lane localization problem in an end-to-end manner. Finally, to ensure the generalization ability, robustness, and stability of LLNet, we conduct extensive comparison experiments on a large-scale real-world dataset, with a total mileage of over 400 km. The experiments show that our approach remarkably outperforms the deep learning classification baselines. In the discussion part of this paper, we give a comprehensive and detailed elaboration for the effectiveness of various designs in our LLNet. To our knowledge, LLNet is the first lane localization method based entirely on deep learning. LLNet is added to the self-driving suite for a kind of mass production vehicle that will be available in the summer of 2022, with an expected sales volume more than 300,000. [ABSTRACT FROM AUTHOR]

The3D object detection of LiDAR point cloud data has generated widespread discussion and implementation in recent years. In this paper, we concentrate on exploring the sampling method of point-based 3D object detection in autonomous driving scenarios, a process which attempts to reduce expenditure by reaching sufficient accuracy using fewer selected points. FPS (farthest point sampling), the most used sampling method, works poorly in small sampling size cases, and, limited by the massive points, some newly proposed sampling methods using deep learning are not suitable for autonomous driving scenarios. To address these issues, we propose the learned sampling network (LSNet), a single-stage 3D object detection network containing an LS module that can sample important points through deep learning. This advanced approach can sample points with a task-specific focus while also being differentiable. Additionally, the LS module is streamlined for computational efficiency and transferability to replace more primitive sampling methods in other point-based networks. To reduce the issue of the high repetition rates of sampled points, a sampling loss algorithm was developed. The LS module was validated with the KITTI dataset and outperformed the other sampling methods, such as FPS and F-FPS (FPS based on feature distance). Finally, LSNet achieves acceptable accuracy with only 128 sampled points and shows promising results when the number of sampled points is small, yielding up to a 60% improvement against competing methods with eight sampled points. [ABSTRACT FROM AUTHOR]

The autonomous flight of an unmanned aerial vehicle refers to creating a new flight route after self-recognition and judgment when an unexpected situation occurs during the flight. The unmanned aerial vehicle can fly at a high speed of more than 60 km/h, so obstacle recognition and avoidance must be implemented in real-time. In this paper, we propose to recognize objects quickly and accurately by effectively using the H/W resources of small computers mounted on industrial unmanned air vehicles. Since the number of pixels in the image decreases after the resizing process, filtering and object resizing were performed according to the altitude, so that quick detection and avoidance could be performed. To this end, objects up to 60 m in height were classified by subdividing them at 20 m intervals, and objects unnecessary for object detection were filtered with deep learning methods. In the 40 m to 60 m sections, the average speed of recognition was increased by 38%, without compromising the accuracy of object detection. [ABSTRACT FROM AUTHOR]

Despite impressive progress, obtaining appropriate data for instance-level lane segmentation remains a significant challenge. This limitation hinders the refinement of granular lane-related applications such as lane line crossing surveillance, pavement maintenance, and management. To address this gap, we introduce a benchmark for lane instance segmentation called InstLane. To the best of our knowledge, InstLane constitutes the first publicly accessible instance-level segmentation standard for lane line detection. The complexity of InstLane emanates from the fact that the original data are procured using cameras mounted laterally, as opposed to traditional front-mounted sensors. InstLane encapsulates a range of challenging scenarios, enhancing the generalization and robustness of the lane line instance segmentation algorithms. In addition, we propose GeoLaneNet, a real-time, geometry-aware lane instance segmentation network. Within GeoLaneNet, we design a finer localization of lane proto-instances based on geometric features to counteract the prevalent omission or multiple detections in dense lane scenarios resulting from non-maximum suppression (NMS). Furthermore, we present a scheme that employs a larger receptive field to achieve profound perceptual lane structural learning, thereby improving detection accuracy. We introduce an architecture based on partial feature transformation to expedite the detection process. Comprehensive experiments on InstLane demonstrate that GeoLaneNet can achieve up to twice the speed of current State-Of-The-Artmethods, reaching 139 FPS on an RTX3090 and a mask AP of 73.55%, with a permissible trade-off in AP, while maintaining comparable accuracy. These results underscore the effectiveness, robustness, and efficiency of GeoLaneNet in autonomous driving. [ABSTRACT FROM AUTHOR]

Accurate and comprehensive 3D objects detection is important for perception systems in autonomous driving. Nevertheless, contemporary mainstream methods tend to perform more effectively on large objects in regions proximate to the LiDAR, leaving limited exploration of long-range objects and small objects. The divergent point pattern of LiDAR, which results in a reduction in point density as the distance increases, leads to a non-uniform point distribution that is ill-suited to discretized volumetric feature extraction. To address this challenge, we propose the Foreground Voxel Proposal (FVP) module, which effectively locates and generates voxels at the foreground of objects. The outputs are subsequently merged to mitigating the difference in point cloud density and completing the object shape. Furthermore, the susceptibility of small objects to occlusion results in the loss of feature space. To overcome this, we propose the Multi-Scale Feature Integration Network (MsFIN), which captures contextual information at different ranges. Subsequently, the outputs of these features are integrated through a cascade framework based on transformers in order to supplement the object features space. The extensive experimental results demonstrate that our network achieves remarkable results. Remarkably, our approach demonstrated an improvement of 8.56% AP on the SECOND baseline for the Car detection task at a distance of more than 20 m, and 9.38% AP on the Cyclist detection task. [ABSTRACT FROM AUTHOR]

Cameras and LiDAR sensors have been used in sensor fusion for robust object detection in autonomous driving. Object detection networks for autonomous driving are often trained again by adding or changing datasets aimed at robust performance. Repeat training is necessary to develop an efficient network module. Existing efficient network module development changes to hand design and requires much module design experience. For this, a neural architecture search was designed, but it takes much time and requires optimizing the design process. To solve this problem, we propose a two-stage optimization method for the offspring generation process in a neural architecture search based on reinforcement learning. In addition, we propose utilizing two split datasets to solve the fast convergence problem as the objective function of the genetic algorithm: source data (daytime, sunny) and target data (day/night, adversary weather). The proposed method is an efficient module generation method requiring less time than the NSGA-NET. We confirmed the performance improvement and the convergence speed reduction using the Dense dataset. Through experiments, it was proven that the proposed method generated an efficient module. [ABSTRACT FROM AUTHOR]