Search

Projecting the point cloud on the 2D spherical range image transforms the LiDAR semantic segmentation to a 2D segmentation task on the range image. However, the LiDAR range image is still naturally different from the regular 2D RGB image; for example, each position on the range image encodes the unique geometry information. In this paper, we propose a new projection-based LiDAR semantic segmentation pipeline that consists of a novel network structure and an efficient post-processing step. In our network structure, we design a FID (fully interpolation decoding) module that directly upsamples the multi-resolution feature maps using bilinear interpolation. Inspired by the 3D distance interpolation used in PointNet++, we argue this FID module is a 2D version distance interpolation on $(\theta, \phi)$ space. As a parameter-free decoding module, the FID largely reduces the model complexity by maintaining good performance. Besides the network structure, we empirically find that our model predictions have clear boundaries between different semantic classes. This makes us rethink whether the widely used K-nearest-neighbor post-processing is still necessary for our pipeline. Then, we realize the many-to-one mapping causes the blurring effect that some points are mapped into the same pixel and share the same label. Therefore, we propose to process those occluded points by assigning the nearest predicted label to them. This NLA (nearest label assignment) post-processing step shows a better performance than KNN with faster inference speed in the ablation study. On the SemanticKITTI dataset, our pipeline achieves the best performance among all projection-based methods with $64 \times 2048$ resolution and all point-wise solutions. With a ResNet-34 as the backbone, both the training and testing of our model can be finished on a single RTX 2080 Ti with 11G memory. The code is released., Comment: Accepted by IROS'21, code link: https://github.com/placeforyiming/IROS21-FIDNet-SemanticKITTI

Light Detection And Ranging (LiDAR) has been widely used in autonomous vehicles for perception and localization. However, the cost of a high-resolution LiDAR is still prohibitively expensive, while its low-resolution counterpart is much more affordable. Therefore, using low-resolution LiDAR for autonomous driving is an economically viable solution, but the point cloud sparsity makes it extremely challenging. In this paper, we propose a two-stage neural network framework that enables 3D object detection using a low-resolution LiDAR. Taking input from a low-resolution LiDAR point cloud and a monocular camera image, a depth completion network is employed to produce dense point cloud that is subsequently processed by a voxel-based network for 3D object detection. Evaluated with KITTI dataset for 3D object detection in Bird-Eye View (BEV), the experimental result shows that the proposed approach performs significantly better than directly applying the 16-line LiDAR point cloud for object detection. For both easy and moderate cases, our 3D vehicle detection results are close to those using 64-line high-resolution LiDARs.

LiDAR depth completion is a task that predicts depth values for every pixel on the corresponding camera frame, although only sparse LiDAR points are available. Most of the existing state-of-the-art solutions are based on deep neural networks, which need a large amount of data and heavy computations for training the models. In this letter, a novel non-learning depth completion method is proposed by exploiting the local surface geometry that is enhanced by an outlier removal algorithm. The proposed surface geometry model is inspired by the observation that most pixels with unknown depth have a nearby LiDAR point. Therefore, it is assumed those pixels share the same surface with the nearest LiDAR point, and their respective depth can be estimated as the nearest LiDAR depth value plus a residual error. The residual error is calculated by using a derived equation with several physical parameters as input, including the known camera intrinsic parameters, estimated normal vector, and offset distance on the image plane. The proposed method is further enhanced by an outlier removal algorithm that is designed to remove incorrectly mapped LiDAR points from occluded regions. On KITTI dataset, the proposed solution achieves the best error performance among all existing non-learning methods and is comparable to the best self-supervised learning method and some supervised learning methods. Moreover, since outlier points from occluded regions is a commonly existing problem, the proposed outlier removal algorithm is a general preprocessing step that is applicable to many robotic systems with both camera and LiDAR sensors., Comment: IEEE Robotics and Automation Letters (2021). Code link: https://github.com/placeforyiming/RAL_Non-Learning_DepthCompletion

We study the problem of simultaneous search for multiple targets over a multidimensional unit cube and derive fundamental resolution limits of non-adaptive querying procedures using the 20 questions estimation framework. The performance criterion that we consider is the achievable resolution, which is defined as the maximal $L_\infty$ norm between the location vector and its estimated version where the maximization is over all target location vectors. The fundamental resolution limit is defined as the minimal achievable resolution of any non-adaptive query procedure, where each query has binary yes/no answers. We drive non-asymptotic and second-order asymptotic bounds on the minimal achievable resolution, using tools from finite blocklength information theory. Specifically, in the achievability part, we relate the 20 questions problem to data transmission over a multiple access channel, use the information spectrum method by Han and borrow results from finite blocklength analysis for random access channel coding. In the converse part, we relate the 20 questions problem to data transmission over a point-to-point channel and adapt finite blocklength converse results for channel coding. Our results extend the purely first-order asymptotic analyses of Kaspi \emph{et al.} (ISIT 2015) for the one-dimensional case: we consider channels beyond the binary symmetric channel and derive non-asymptotic and second-order asymptotic bounds on the performance of optimal non-adaptive query procedures., Comment: To appear in IEEE Transactions on Information Theory. Copyright (c) 2017 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org

In this paper, we propose a novel visible light communication (VLC) assisted Perspective-fourLine algorithm (V-P4L) for practical indoor localization. The basic idea of V-P4L is to joint VLC and computer vision to achieve high accuracy regardless of LED height differences. In particular, we first exploit the space-domain information to estimate the orientation and coordinate of a single rectangular LED luminaire in the camera coordinate system based on the plane and solid geometry. Then, based on the time-domain information transmitted by VLC and the estimated luminaire information, V-P4L can estimate the position and pose of the camera by the single-view geometry theory and the linear least square (LLS) method. To further mitigate the effect of height differences among LEDs on localization accuracy, we then propose a correction algorithm of V-P4L based on the LLS method and a simple optimization method. Due to the combination of time- and space-domain information, V-P4L only requires a single luminaire for localization without limitation on the correspondences between the features and their projections. Simulation results show that for V-P4L the position error is always less than 15 cm and the orientation error is always less than 3{\deg} using popular indoor luminaires.

Autonomous vehicles rely heavily on sensors such as camera and LiDAR, which provide real-time information about their surroundings for the tasks of perception, planning and control. Typically a LiDAR can only provide sparse point cloud owing to a limited number of scanning lines. By employing depth completion, a dense depth map can be generated by assigning each camera pixel a corresponding depth value. However, the existing depth completion convolutional neural networks are very complex that requires high-end GPUs for processing, and thus they are not applicable to real-time autonomous driving. In this paper, a light-weight network is proposed for the task of LiDAR point cloud depth completion. With an astonishing 96.2% reduction in the number of parameters, it still achieves comparable performance (9.3% better in MAE but 3.9% worse in RMSE) to the state-of-the-art network. For real-time embedded platforms, depthwise separable technique is applied to both convolution and deconvolution operations and the number of parameters decreases further by a factor of 7.3, with only a small percentage increase in RMSE and MAE performance. Moreover, a system-on-chip architecture for depth completion is developed on a PYNQ-based FPGA platform that achieves real-time processing for HDL-64E LiDAR at the speed 11.1 frame per second.

In recent years, convolutional neural network has gained popularity in many engineering applications especially for computer vision. In order to achieve better performance, often more complex structures and advanced operations are incorporated into the neural networks, which results very long inference time. For time-critical tasks such as autonomous driving and virtual reality, real-time processing is fundamental. In order to reach real-time process speed, a light-weight, high-throughput CNN architecture namely RoadNet-RT is proposed for road segmentation in this paper. It achieves 90.33% MaxF score on test set of KITTI road segmentation task and 8 ms per frame when running on GTX 1080 GPU. Comparing to the state-of-the-art network, RoadNet-RT speeds up the inference time by a factor of 20 at the cost of only 6.2% accuracy loss. For hardware design optimization, several techniques such as depthwise separable convolution and non-uniformed kernel size convolution are customized designed to further reduce the processing time. The proposed CNN architecture has been successfully implemented on an FPGA ZCU102 MPSoC platform that achieves the computation capability of 83.05 GOPS. The system throughput reaches 327.9 frames per second with image size 1216x176.

LiDAR sensors have been widely used in many autonomous vehicle modalities, such as perception, mapping, and localization. This paper presents an FPGA-based deep learning platform for real-time point cloud processing targeted on autonomous vehicles. The software driver for the Velodyne LiDAR sensor is modified and moved into the on-chip processor system, while the programmable logic is designed as a customized hardware accelerator. As the state-of-art deep learning algorithm for point cloud processing, PointNet is successfully implemented on the proposed FPGA platform. Targeted on a Xilinx Zynq UltraScale+ MPSoC ZCU104 development board, the FPGA implementations of PointNet achieve the computing performance of 182.1 GOPS and 280.0 GOPS for classification and segmentation respectively. The proposed design can support an input up to 4096 points per frame. The processing time is 19.8 ms for classification and 34.6 ms for segmentation, which meets the real-time requirement for most of the existing LiDAR sensors., Comment: This paper has been accepted by ISCAS 2020

In this paper, a scalable neural network hardware architecture for image segmentation is proposed. By sharing the same computing resources, both convolution and deconvolution operations are handled by the same process element array. In addition, access to on-chip and off-chip memories is optimized to alleviate the burden introduced by partial sum. As an example, SegNet-Basic has been implemented using the proposed unified architecture by targeting on Xilinx ZC706 FPGA, which achieves the performance of 151.5 GOPS and 94.3 GOPS for convolution and deconvolution respectively. This unified convolution/deconvolution design is applicable to other CNNs with deconvolution., Comment: This paper has been accepted by ISCAS 2020

In this paper, a received signal strength assisted Perspective-three-Point positioning algorithm (R-P3P) is proposed for visible light positioning (VLP) systems. The basic idea of R-P3P is to joint visual and strength information to estimate the receiver position using 3 LEDs regardless of the LEDs' orientations. R-P3P first utilizes visual information captured by the camera to estimate the incidence angles of visible lights. Then, R-P3P calculates the candidate distances between the LEDs and the receiver based on the law of cosines and the Wu-Ritt's zero decomposition method. Based on the incidence angles, the candidate distances and the physical characteristics of the LEDs, R-P3P can select the exact distances from all the candidate distances. Finally, the linear least square (LLS) method is employed to estimate the position of the receiver. Due to the combination of visual and strength information of visible light signals, R-P3P can achieve high accuracy using 3 LEDs regardless of the LEDs' orientations. Simulation results show that R-P3P can achieve positioning accuracy within 10 cm over 70% indoor area with low complexity regardless of LEDs orientations., Comment: arXiv admin note: substantial text overlap with arXiv:2004.06294

The existing phase shifter models adopted for reconfigurable intelligent surfaces (RISs) have ignored the electromagnetic (EM) waves propagation behavior, thus cannot reveal practical effects of RIS on wireless communication systems. Based on the equivalent circuit, this paper introduces an angle-dependent phase shifter model for varactor-based RISs. To the best of our knowledge, this is the first phase shifter model which reveals that the incident angle of EM waves has influence on the reflection coefficient of RIS. In addition, the angle-reciprocity on RIS is investigated and further proved to be tenable when the reflection phase difference of adjacent RIS unit cells is invariant for an impinging EM wave and its reverse incident one. The angle-dependent characteristic of RIS is verified through full-wave simulation. According to our analysis and the simulation results, we find that the angle-reciprocity of varactor-based RIS only holds under small incident angles of both forward and reverse incident EM waves, thus limits the channel reciprocity in RIS-assisted TDD systems., Comment: Accepted by IEEE Communications Letters

In this paper, a high coverage algorithm termed enhanced camera assisted received signal strength ratio (eCA-RSSR) positioning algorithm is proposed for visible light positioning (VLP) systems. The basic idea of eCA-RSSR is to utilize visual information captured by the camera to estimate the incidence angles of visible lights first. Based on the incidence angles, eCA-RSSR utilizes the received signal strength ratio (RSSR) calculated by the photodiode (PD) to estimate the ratios of the distances between the LEDs and the receiver. Based on an Euclidean plane geometry theorem, eCA-RSSR transforms the ratios of the distances into the absolute values. In this way, eCA-RSSR only requires 3 LEDs for both orientation-free 2D and 3D positioning, implying that eCA-RSSR can achieve high coverage. Based on the absolute values of the distances, the linear least square method is employed to estimate the position of the receiver. Therefore, for the receiver having a small distance between the PD and the camera, the accuracy of eCA-RSSR does not depend on the starting values of the non-linear least square method and the complexity of eCA-RSSR is low. Furthermore, since the distance between the PD and camera can significantly affect the performance of eCA-RSSR, we further propose a compensation algorithm for eCA-RSSR based on the single-view geometry. Simulation results show that eCA-RSSR can achieve centimeter-level accuracy over 80% indoor area for both the receivers having a small and a large distance between the PD and the camera.