Search

Data augmentation is widely recognized as an effective means of bolstering model robustness. However, when applied to monocular 3D object detection, non-geometric image augmentation neglects the critical link between the image and physical space, resulting in the semantic collapse of the extended scene. To address this issue, we propose two geometric-level data augmentation operators named Geometric-Copy-Paste (Geo-CP) and Geometric-Crop-Shrink (Geo-CS). Both operators introduce geometric consistency based on the principle of perspective projection, complementing the options available for data augmentation in monocular 3D. Specifically, Geo-CP replicates local patches by reordering object depths to mitigate perspective occlusion conflicts, and Geo-CS re-crops local patches for simultaneous scaling of distance and scale to unify appearance and annotation. These operations ameliorate the problem of class imbalance in the monocular paradigm by increasing the quantity and distribution of geometrically consistent samples. Experiments demonstrate that our geometric-level augmentation operators effectively improve robustness and performance in the KITTI and Waymo monocular 3D detection benchmarks. [ABSTRACT FROM AUTHOR]

Hand-eye calibration, a fundamental task in vision-based robotic systems, is commonly equipped with collaborative robots, especially for robotic applications in small and medium-sized enterprises (SMEs). Most approaches to hand-eye calibration rely on external markers or human assistance. We proposed a novel methodology that addresses the hand-eye calibration problem using the robot base as a reference, eliminating the need for external calibration objects or human intervention. Using point clouds of the robot base, a transformation matrix from the coordinate frame of the camera to the robot base is established as “I=AXB.” To this end, we exploit learning-based 3D detection and registration algorithms to estimate the location and orientation of the robot base. The robustness and accuracy of the method are quantified by ground-truth-based evaluation, and the accuracy result is compared with other 3D vision-based calibration methods. To assess the feasibility of our methodology, we carried out experiments utilizing a low-cost structured light scanner across varying joint configurations and groups of experiments. The proposed hand-eye calibration method achieved a translation deviation of 0.930 mm and a rotation deviation of 0.265 degrees according to the experimental results. Additionally, the 3D reconstruction experiments demonstrated a rotation error of 0.994 degrees and a position error of 1.697 mm. Moreover, our method offers the potential to be completed in 1 second, which is the fastest compared to other 3D hand-eye calibration methods. We conduct indoor 3D reconstruction and robotic grasping experiments based on our hand-eye calibration method. Related code is released at . [ABSTRACT FROM AUTHOR]

With the increase in port throughput and the development of the trend of large-scale ships, selecting applicable anchor positions for ships and ensuring the rational and comprehensive utilization of anchorage areas have become a key issue in utilizing the rate of anchorage resources, ensuring the safety of ships anchoring operations and promoting the development of the shipping industry. Existing anchor position selection and detection algorithm studies are limited to a two-dimensional plane for ship anchor position selection, with few studies considering intelligent detection algorithms for safe ship anchoring water depths based on three-dimensional space, considering conditions such as wind and waves. By considering water depth conditions and the objectives of anchorage safety issues, this study proposes an intelligent detection method for ship anchor detection to find the ship's ideal anchor location in the anchorages by applying the Monte Carlo algorithm. In the processing step, in combination with the Monte Carlo random plane anchor position detection algorithm and Monte Carlo random sampling water depth detection method, the anchor position circle radius model, safe spacing model, anchoring area detection model and safe water depth model are used for examining the anchorage area for awaiting the ship in three-dimensions. To verify the accuracy of the proposed model, based on the scale of common ship types and considering the most conservative parameters, a series of simulation experiments are conducted to check whether the water depth meets the requirements and fully ensure the safety of the experimental results. The research results indicate that the detection almost cover the whole anchorage area and obtain safe water depth restrictions. This method helps to improve the efficiency of ship anchoring and makes actual anchoring operations safer. Under the premise of ensuring sufficient safe spacing between ships, the anchorage ground can accommodate more ships and simulate multi-type ship anchor position detection operations concerning various ship-type parameters to further ensure the safety of ship anchoring. [ABSTRACT FROM AUTHOR]

Objectives: A real-time and robust 3D dynamic object perception module is a key part of autonomous driving system. Methods: This paper fuses monocular camera and light detection and ranging (LiDAR) to detect 3D objects. First, we use convolutional neural network to detect 2D bounding boxes and generate 3D frustum region of interest (ROI) according to the geometric projection relation between camera and LiDAR. Then, we cluster the point cloud in the frustum ROI and fit the 3D bounding box of the objects. After detecting 3D objects, we reidentify the objects between adjacent frames by appearance features and Hungarian algorithm, and then propose a tracker management model based on a quad-state machine. Finally, a novel prediction model is proposed, which leverages lane lines to constrain vehicle trajectories. Results: The experimental results show that in the stage of target detection, the accuracy and recall of the proposed algorithm can reach 92.5% and 86.7%, respectively. The root mean square error of the proposed trajectory prediction algorithm is smaller than that of the existing algorithms on the simulation datasets including straight line, arc and spiral curves. The whole algorithm only takes approximately 25 ms, which meets the real-time requirements. Conclusions: The proposed algorithm is effective and efficient, and has a good performance in different lane lines. [ABSTRACT FROM AUTHOR]

This paper presents a low-cost pipeline surface 3D detection method based on line structured light. The method adds 3D detection to the robot at a meager cost and change. The optical flow method is used to derive the motion information of the robot to replace the motion closed loop. Finally, the depth data for the entire surface are generated automatically only from the camera and the line laser projector, without using other devices. In addition, a novel spot centroid extraction algorithm based on the color region of interest and an adaptive threshold is presented. This method can accurately detect the laser centroid in the pipeline surface. We conducted experiments with a quadruped robot and validated algorithms. The experimental results show that the proposed method achieves 3D detection on a trackless robot at a meager cost and is superior to standard depth sensors. [ABSTRACT FROM AUTHOR]

Detection based on LiDAR and camera fusion is increasingly popular for researchers in the autonomous driving domain. Compared to the camera-only and LiDAR-only methods, the fusion-based methods indeed improve the detection accuracy on public-available datasets. However, due to the complexity of the projection or fusion mechanism, few of these methods can run in real time even on an advanced desktop GPU. Thus, in this paper, we propose a new fusion detection model G-Fusion with a light and fast image view-transform module. According to our receptive field analysis of image feature maps, we directly project image features to only one voxel layer located on the ground, then fuse the LiDAR and image features by concatenation and convolution. With this delicately designed module, G-Fusion greatly boosts the state-of-the-art speed performance on the nuScenes dataset, achieving a good balance with the competitive detection scores. Meanwhile, since the precision of sensor extrinsic parameters is important for most fusion-based methods, we also deeply dig into our model’s calibration error tolerance ability and discover the failure noise condition.

With the increase in port throughput and the development of the trend of large-scale ships, selecting applicable anchor positions for ships and ensuring the rational and comprehensive utilization of anchorage areas have become a key issue in utilizing the rate of anchorage resources, ensuring the safety of ships anchoring operations and promoting the development of the shipping industry. Existing anchor position selection and detection algorithm studies are limited to a two-dimensional plane for ship anchor position selection, with few studies considering intelligent detection algorithms for safe ship anchoring water depths based on three-dimensional space, considering conditions such as wind and waves. By considering water depth conditions and the objectives of anchorage safety issues, this study proposes an intelligent detection method for ship anchor detection to find the ship’s ideal anchor location in the anchorages by applying the Monte Carlo algorithm. In the processing step, in combination with the Monte Carlo random plane anchor position detection algorithm and Monte Carlo random sampling water depth detection method, the anchor position circle radius model, safe spacing model, anchoring area detection model and safe water depth model are used for examining the anchorage area for awaiting the ship in three-dimensions. To verify the accuracy of the proposed model, based on the scale of common ship types and considering the most conservative parameters, a series of simulation experiments are conducted to check whether the water depth meets the requirements and fully ensure the safety of the experimental results. The research results indicate that the detection almost cover the whole anchorage area and obtain safe water depth restrictions. This method helps to improve the efficiency of ship anchoring and makes actual anchoring operations safer. Under the premise of ensuring sufficient safe spacing between ships, the anchorage ground can accommodate more ships and simulate multi-type ship anchor position detection operations concerning various ship-type parameters to further ensure the safety of ship anchoring.

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Instance-level perception tasks like object detection, instance segmentation, and 3D detection require many training samples to achieve satisfactory performance. The meticulous labels for these tasks are usually expensive to obtain and data augmentation is a natural choice to tackle such a problem. However, instance-level augmentation is less studied in previous research. In this paper, we present an effective, efficient and unified crop-paste mechanism to augment the training set utilizing existing instance-level annotations. Our design is derived from visual coherence and mines three inherent principles that widely exist in real-world data: (i) background coherence in local neighbor area, (ii) appearance coherence for instance placement, and (iii) instance coherence within the same category. Such methodologies are unified for various tasks including object detection, instance segmentation, and 3D detection. Extensive experiments demonstrate that our proposed approaches can successfully boost the performance of diverse frameworks on various datasets across multiple tasks, without modifying the network structure. Remarkable improvements are obtained: 5.1 mAP for object detection and 3.2 mAP for instance segmentation on COCO dataset, and 6.9 mAP for 3D detection on ScanNetV2 dataset. Our method can be easily integrated into different frameworks without affecting the training and inference efficiency. [ABSTRACT FROM AUTHOR]

In this paper, LCD in‐cell self‐capacitive sensor using 3D detection driving method for proximity 50mm is proposed. Based on self‐capacitive sensor and 3D detection driving method, proximity signal is 0.249pF in our simulation system. This paper also shows that the 50mm height slug can be detected with Novatek LCD TDDI NT36523 and PCB sensor board. [ABSTRACT FROM AUTHOR]

Abstract Sensor fusion is an important component of the perception system in autonomous driving, and the fusion of radar point cloud information and camera visual information can improve the perception capability of autonomous vehicles. However, most of the existing studies ignore the extraction of local neighborhood information and only consider shallow fusion between the two modalities based on the extracted global information, which cannot perform a deep fusion of cross-modal contextual information interaction. Meanwhile, in data preprocessing, the noise in radar data is usually only filtered by the depth information derived from image feature prediction, and such methods affect the accuracy of radar branching to generate regions of interest and cannot effectively filter out irrelevant information of radar points. This paper proposes the CenterTransFuser model that makes full use of millimeter-wave radar point cloud information and visual information to enable cross-modal fusion of the two heterogeneous information. Specifically, a new interaction called cross-transformer is explored, which cooperatively exploits cross-modal cross-multiple attention and joint cross-multiple attention to mine radar and image complementary information. Meanwhile, an adaptive depth thresholding filtering method is designed to reduce the noise of radar modality-independent information projected onto the image. The CenterTransFuser model is evaluated on the challenging nuScenes dataset, and it achieves excellent performance. Particularly, the detection accuracy is significantly improved for pedestrians, motorcycles, and bicycles, showing the superiority and effectiveness of the proposed model.

Aiming at the problems of high complexity and low detection accuracy of single-stage three-dimensional (3D) detection method, a vehicle object detection algorithm based on the Efficient Channel Attention (ECA) mechanism is proposed. This paper provides a good solution to the problems of low object recognition accuracy and high model complexity in the field of 3D object detection. First, we voxelized the original point cloud data, taking the average coordinates and intensity values as the initial features. By entering into the Voxel Feature Encoding (VFE) layer, we can extract the features of each voxel. Then, referring to the VoxelNet model, the ECA mechanism is introduced, which reduces the complexity of the model while maintaining the good performance in the model. Finally, experiments on the widely used KITTI dataset show that the algorithm performs well, and the accuracy of the proposed ECA algorithm has reached 87.75%. Compared with the current mainstream algorithm SE-SSD of object detection, the accuracy is increased by 0.21%. [ABSTRACT FROM AUTHOR]

In this study, we present a three-dimensional (3D) object detection algorithm based on monocular images by constructing an end-to-end network, that incorporates depth information. The entire network consists of three parts. The first part includes the basic object detection neural network as the main body, that uses the region proposal network to obtain the two-dimensional (2D) region proposal of the object. The second part is the depth estimation branch network, that obtains the depth information of the object pixels and calculates the corresponding 3D point cloud. In the last part, concatenated features obtained from the aforementioned two parts are fed into the fully-connected layers. Subsequently, 2D and 3D detection results are obtained. Compared with certain existing methods, the accuracy of the detection results is improved in this study. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Shanghai Jiaotong University (Science) is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Sensor fusion is an important component of the perception system in autonomous driving, and the fusion of radar point cloud information and camera visual information can improve the perception capability of autonomous vehicles. However, most of the existing studies ignore the extraction of local neighborhood information and only consider shallow fusion between the two modalities based on the extracted global information, which cannot perform a deep fusion of cross-modal contextual information interaction. Meanwhile, in data preprocessing, the noise in radar data is usually only filtered by the depth information derived from image feature prediction, and such methods affect the accuracy of radar branching to generate regions of interest and cannot effectively filter out irrelevant information of radar points. This paper proposes the CenterTransFuser model that makes full use of millimeter-wave radar point cloud information and visual information to enable cross-modal fusion of the two heterogeneous information. Specifically, a new interaction called cross-transformer is explored, which cooperatively exploits cross-modal cross-multiple attention and joint cross-multiple attention to mine radar and image complementary information. Meanwhile, an adaptive depth thresholding filtering method is designed to reduce the noise of radar modality-independent information projected onto the image. The CenterTransFuser model is evaluated on the challenging nuScenes dataset, and it achieves excellent performance. Particularly, the detection accuracy is significantly improved for pedestrians, motorcycles, and bicycles, showing the superiority and effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

The deep learning-based 3D object detection literature on monocular images has been dominated by methods that require supervision in the form of 3D bounding box annotations for training. However, obtaining sufficient 3D annotations is expensive, laborious and prone to introducing errors. To address this problem, we propose a monocular self-supervised approach towards 3D object detection relying solely on observed RGB data rather than 3D bounding boxes for training. We leverage differentiable rendering to apply visual alignment to depth maps, instance masks and point clouds for self-supervision. Furthermore, considering the complexity of autonomous driving scenes, we introduce a point cloud filter to reduce noise impact and design an automatic training set pruning strategy suitable for the self-supervised framework to further improve network performance. We provide detailed experiments on the KITTI benchmark and achieve competitive performance with existing self-supervised methods as well as some fully supervised methods. • A monocular self-supervised 3D object detection method for autonomous driving. • A point cloud filter to reduce noise impact. • A training set pruning strategy suitable for the self-supervised framework. • Performance competitive with existing self-supervised methods. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this paper, we propose a novel system named Disp R-CNN for 3D object detection from stereo images. Many recent works solve this problem by first recovering point clouds with disparity estimation and then apply a 3D detector. The disparity map is computed for the entire image, which is costly and fails to leverage category-specific prior. In contrast, we design an instance disparity estimation network (iDispNet) that predicts disparity only for pixels on objects of interest and learns a category-specific shape prior for more accurate disparity estimation. To address the challenge from scarcity of disparity annotation in training, we propose to use a statistical shape model to generate dense disparity pseudo-ground-truth without the need of LiDAR point clouds, which makes our system more widely applicable. Experiments on the KITTI dataset show that, when LiDAR ground-truth is not used at training time, Disp R-CNN outperforms previous state-of-the-art methods based on stereo input by 20 percent in terms of average precision for all categories. The code and pseudo-ground-truth data are available at the project page: https://github.com/zju3dv/disprcnn. [ABSTRACT FROM AUTHOR]

Voxel-based structures in 3D detection have achieved rapid advancement due to their superior capability for feature extraction. However, the accuracy is usually low because the point cloud is divided into a grid. In order to overcome the above problems and improve detection accuracy, we propose a flexible two-stage 3D object detection architecture, which adopts two branches to refine generated proposals, aggregating voxel features and raw point features simultaneously. We also design a new gating mechanism to achieve fusion features from different levels. In addition, we propose a novel feature aggregation module to reduce the semantic gap between the features of the two types. First, a transformer based on raw points is employed as an encoder to aggregate the contextual information. Then, the point-based channel-wise self-attention mechanism serves as a decoder to aggregate the global features. Experiment results on the KITTI 3D dataset and Waymo Open datest demonstrate that our approach outperforms the state-of-the-art methods and exhibits excellent scalability. [ABSTRACT FROM AUTHOR]

Deep neural networks are widely used for understanding 3D point clouds. At each point convolution layer, features are computed from local neighbourhoods of 3D points and combined for subsequent processing in order to extract semantic information. Existing methods adopt the same individual point neighborhoods throughout the network layers, defined by the same metric on the fixed input point coordinates. This common practice is easy to implement but not necessarily optimal. Ideally, local neighborhoods should be different at different layers, as more latent information is extracted at deeper layers. We propose a novel end-to-end approach to learn different non-rigid transformations of the input point cloud so that optimal local neighborhoods can be adopted at each layer. We propose both linear (affine) and non-linear (projective and deformable) spatial transformers for 3D point clouds. With spatial transformers on the ShapeNet part segmentation dataset, the network achieves higher accuracy for all categories, with 8 percent gain on earphones and rockets in particular. Our method also outperforms the state-of-the-art on other point cloud tasks such as classification, detection, and semantic segmentation. Visualizations show that spatial transformers can learn features more efficiently by dynamically altering local neighborhoods according to the geometry and semantics of 3D shapes in spite of their within-category variations. [ABSTRACT FROM AUTHOR]

Accurate and fast 3D object detection from point clouds is a key task in autonomous driving. Existing one-stage 3D object detection methods can achieve real-time performance, however, they are dominated by anchor-based detectors which are inefficient and require additional post-processing. In this paper, we eliminate anchors and model an object as a single point—the center point of its bounding box. Based on the center point, we propose an anchor-free CenterNet3D network that performs 3D object detection without anchors. Our CenterNet3D uses keypoint estimation to find center points and directly regresses 3D bounding boxes. However, because inherent sparsity of point clouds, 3D object center points are likely to be in empty space which makes it difficult to estimate accurate boundaries. To solve this issue, we propose an extra corner attention module to enforce the CNN backbone to pay more attention to object boundaries. Besides, considering that one-stage detectors suffer from the discordance between the predicted bounding boxes and corresponding classification confidences, we develop an efficient keypoint-sensitive warping operation to align the confidences to the predicted bounding boxes. Our proposed CenterNet3D is non-maximum suppression free which makes it more efficient and simpler. We evaluate CenterNet3D on the widely used KITTI dataset and more challenging nuScenes dataset. Our method outperforms all state-of-the-art anchor-based one-stage methods and has comparable performance to two-stage methods as well. It has an inference speed of 20 FPS and achieves the best speed and accuracy trade-off. Our source code will be released at https://github.com/wangguojun2018/CenterNet3d. [ABSTRACT FROM AUTHOR]

Varying density of point clouds increases the difficulty of 3D detection. In this paper, we present a context-aware dynamic network (CADNet) to capture the variance of density by considering both point context and semantic context. Point-level contexts are generated from original point clouds to enlarge the effective receptive filed. They are extracted around the voxelized pillars based on our extended voxelization method and processed with the context encoder in parallel with the pillar features. With a large perception range, we are able to capture the variance of features for potential objects and generate attentive spatial guidance to help adjust the strengths for different regions. In the region proposal network, considering the limited representation ability of traditional convolution where same kernels are shared among different samples and positions, we propose a decomposable dynamic convolutional layer to adapt to the variance of input features by learning from the local semantic context. It adaptively generates the position-dependent coefficients for multiple fixed kernels and combines them to convolve with local features. Based on our dynamic convolution, we design a dual-path convolution block to further improve the representation ability. We conduct experiments on KITTI dataset and the proposed CADNet has achieved superior performance of 3D detection outperforming SECOND and PointPillars by a large margin at the speed of 30 FPS. [ABSTRACT FROM AUTHOR]

This paper proposed a two-stage Voxel-based 3D Object detector which named GVnet. Voxel-based method mainly relies on sampling and Grouping point in voxel and the feature map generated by subsequent 3D CNN to control the quality of detection. Moreover, traditional voxel feature encoder (VFE) methods cannot adjust the quality of feature map through reasonable sampling. Therefore, the method we propose is an improvement to the existing VFE. The specific operations are: First calculate the corresponding Gaussian distribution of the original point cloud data, and then sampling any number of points by controlling the confidence value to improve the performance of voxel encoder and further improve the quality of the feature map output by the 3D CNN. In addition, a voxel ROI pooling method is proposed in stage two. In ROI Pooling, the receptive field in the original space and the corresponding raw point are obtained through the mapping relationship between feature and ROI, then change the raw point to adjust the receptive field to improve the performance of classification and regression. Finally, the experimental results on the KITTI, nuScenes and Waymo dataset show that the performance of GVnet under most of the evaluation indexes is better than the current detection methods, at the cost of only a small amount of inference time. [ABSTRACT FROM AUTHOR]

The incidence of facial fractures is on the rise globally, yet limited studies are addressing the diverse forms of facial fractures present in 3D images. In particular, due to the nature of the facial fracture, the direction in which the bone fractures vary, and there is no clear outline, it is difficult to determine the exact location of the fracture in 2D images. Thus, 3D image analysis is required to find the exact fracture area, but it needs heavy computational complexity and expensive pixel-wise labeling for supervised learning. In this study, we tackle the problem of reducing the computational burden and increasing the accuracy of fracture localization by using a weakly-supervised object localization without pixel-wise labeling in a 3D image space. We propose a Very Fast, High-Resolution Aggregation 3D Detection CAM (VFHA-CAM) model, which can detect various facial fractures. To better detect tiny fractures, our model uses high-resolution feature maps and employs Ablation CAM to find an exact fracture location without pixel-wise labeling, where we use a rough fracture image detected with 3D box-wise labeling. To this end, we extract important features and use only essential features to reduce the computational complexity in 3D image space. Experimental findings demonstrate that VFHA-CAM surpasses state-of-the-art 2D detection methods by up to 20% in sensitivity/person and specificity/person, achieving sensitivity/person and specificity/person scores of 87% and 85%, respectively. In addition, Our VFHA-CAM reduces location analysis time to 76 s without performance degradation compared to a simple Ablation CAM method that takes more than 20 min. This study introduces a novel weakly-supervised object localization approach for bone fracture detection in 3D facial images. The proposed method employs a 3D detection model, which helps detect various forms of facial bone fractures accurately. The CAM algorithm adopted for fracture area segmentation within a 3D fracture detection box is key in quickly informing medical staff of the exact location of a facial bone fracture in a weakly-supervised object localization. In addition, we provide 3D visualization so that even non-experts unfamiliar with 3D CT images can identify the fracture status and location. • 3D facial fractures rising, challenging 2D imaging; 3D needed for accurate localization. • Proposed VFHA-CAM detects fractures using weakly-supervised learning without pixel labels. • VFHA-CAM improves detection accuracy by 20% and reduces analysis time to 76 seconds. • VFHA-CAM is applicable to various medical 3D imaging tasks without pixel-wise labels. [ABSTRACT FROM AUTHOR]

The BEV-CFKT proposed in this paper leverages knowledge transfer through transformers for LiDAR-Camera fusion in the Bird's-Eye-View (BEV) space, aiming to achieve accurate and robust 3D object detection. BEV-CFKT comprises three main components, which include the generation of BEV features from images and point clouds, cross-modality interaction, and hybrid object queries using a monocular detection head. By unifying features from both point clouds and images into the BEV space, we simplify modal interaction, facilitate knowledge transfer, and extract richer structural and semantic information from multimodal data. This effectively enhances the network's performance. To further improve detection performance, BEV-CFKT incorporates a temporal fusion module. Additionally, a hybrid object queries module based on a monocular detection head accelerates the convergence of our model. We demonstrate the effectiveness of our approach through an extensive set of experiments. [ABSTRACT FROM AUTHOR]

3D object detection and recognition are crucial tasks for many spatiotemporal processing applications, such as computer-aided diagnosis and autonomous driving. Although prevalent 3D Convolution Nets (ConvNets) have continued to improve the accuracy and sensitivity, excessive computing resources are required. In this paper, we propose Leaky Integrate and Fire Networks (LIF-Nets) for 3D detection and recognition tasks. LIF-Nets have rich inter-frame sensing capability brought from membrane potentials, and low power event-driven mechanism, which make them excel in 3D processing and save computational cost at the same time. We also develop ResLIF Blocks to solve the degradation problem of deep LIF-Nets, and employ U-LIF structure to improve the feature representation capability. As a result, we carry out experiments on the LUng Nodule Analysis 2016 (LUNA16) public dataset for chest CT automated analysis and conclude that the LIF-Nets achieve 94.6% detection sensitivity at 8 False Positives per scan and 94.14% classification accuracy while the LIF-detection net reduces 65.45% multiplication operations, 65.12% addition operations, and 65.32% network parameters. The results show that LIF-Nets have extraordinary time-efficient and energy-saving performance while achieving comparable accuracy.

Agricultural robots must navigate challenging dynamic and semi-structured environments. Recently, environmental modelling using LiDAR-based SLAM has shown promise in providing highly accurate geometry. However, how this chaotic environmental information can be used to achieve effective robot automation in the agricultural sector remains unexplored. In this study, we propose a novel semantic mapping and navigation framework for achieving robotic autonomy in orchards. It consists of two main components: a semantic processing module and a navigation module. First, we present a novel 3D detection network architecture, 3D-ODN, which can accurately process object instance information from point clouds. Second, we develop a framework to construct the visibility map by incorporating semantic information and terrain analysis. By combining these two critical components, our framework is evaluated in a number of key horticultural production scenarios, including a robotic system for in-situ phenotyping and daily monitoring, and a selective harvesting system in apple orchards. The experimental results show that our method can ensure high accuracy in understanding the environment and enable reliable robot autonomy in agricultural environments. • Create a robust 3D Object Detection Network (3D-ODN) to process 3D point cloud from SLAM. • Develop a novel semantic mapping framework involving the 3D-ODN and terrain analysis. • Demonstrate the proposed perception and semantic mapping framework on mobile robots in orchards. [ABSTRACT FROM AUTHOR]

Most autonomous vehicles build their perception systems on expensive sensors, such as LIDAR, RADAR, and high-precision Global Positioning System (GPS). However, cameras can provide richer sensing at a considerably lower cost, this makes them a more appealing alternative. A driving assistance system (DAS) based on monocular vision has gradually become a research hotspot, and inter-vehicle distance estimation based on monocular vision is an important technology in DAS. There are still constrains in the existing methods for estimating the inter-vehicle distance based on monocular vision, such as low accuracy when distance is larger, unstable accuracy for different types vehicles, and significantly poor performance on distance estimation for severely occluded vehicles. To improve the accuracy and robustness of ranging results, this study proposes a monocular vision end-to-end inter-vehicle distance estimation method based on 3D detection. The actual area of the rare view of the vehicle and the corresponding projection area in the image are obtained by 3D detection method. An area-distance geometric model is then established on the basis of the camera projection principle to recover distance. Our method shows its potential in complex traffic scenarios by testing the test set data provided on the real-world computer vision benchmark, KITTI. The experimental results have superior performance than the existing published methods. Moreover, the accuracy of occluded vehicle ranging results can reach approximately $98\%$ , while the accuracy deviation between vehicles with different visual angles is less than $2\%$. [ABSTRACT FROM AUTHOR]

Three-dimensional information perception from point clouds is of vital importance for improving the ability of machines to understand the world, especially for autonomous driving and unmanned aerial vehicles. Data annotation for point clouds is one of the most challenging and costly tasks. In this paper, we propose a closed-loop and virtual–real interactive point cloud generation and model-upgrading framework called Parallel Point Clouds (PPCs). To our best knowledge, this is the first time that the training model has been changed from an open-loop to a closed-loop mechanism. The feedback from the evaluation results is used to update the training dataset, benefiting from the flexibility of artificial scenes. Under the framework, a point-based LiDAR simulation model is proposed, which greatly simplifies the scanning operation. Besides, a group-based placing method is put forward to integrate hybrid point clouds, via locating candidate positions for virtual objects in real scenes. Taking advantage of the CAD models and mobile LiDAR devices, two hybrid point cloud datasets, i.e., ShapeKITTI and MobilePointClouds, are built for 3D detection tasks. With almost zero labor cost on data annotation for newly added objects, the models (PointPillars) trained with ShapeKITTI and MobilePointClouds achieved 78.6% and 60.0% of the average precision of the model trained with real data on 3D detection, respectively.

Recent one-stage 3D detection methods generate anchor boxes with various sizes and orientations in the ground plane, then determine whether these anchor boxes contain any region of interest and adjust the edges of them for accurate object bounding boxes. The anchor-based algorithm calculates the classification and regression label for each anchor box during the training process, which is inefficient and complicated. We propose a one-stage, anchor-free 3D vehicle detection algorithm based on LiDAR point clouds. The object position is encoded as a set of keypoints in the bird’s-eye view (BEV) of point clouds. We apply the voxel/pillar feature extractor and convolutional blocks to map an unstructured point cloud to a single-channel 2D heatmap. The vehicle’s Z-axis position, dimension, and orientation angle are regressed as additional attributes of the keypoints. Our method combines SmoothL1 loss and IoU (Intersection over Union) loss, and we apply (cosθ,sinθ) as angle regression labels, which achieve high average orientation similarity (AOS) without any direction classification tricks. During the target assignment and bounding box decoding process, our framework completely avoids any calculations related to anchor boxes. Our framework is end-to-end training and stands at the same performance level as the other one-stage anchor-based detectors.

We present an algorithm for estimating the pose of a rigid object in real-time under challenging conditions. Our method effectively handles poorly textured objects in cluttered, changing environments, even when their appearance is corrupted by large occlusions, and it relies on grayscale images to handle metallic environments on which depth cameras would fail. As a result, our method is suitable for practical Augmented Reality applications including industrial environments. At the core of our approach is a novel representation for the 3D pose of object parts: We predict the 3D pose of each part in the form of the 2D projections of a few control points. The advantages of this representation is three-fold: We can predict the 3D pose of the object even when only one part is visible; when several parts are visible, we can easily combine them to compute a better pose of the object; the 3D pose we obtain is usually very accurate, even when only few parts are visible. We show how to use this representation in a robust 3D tracking framework. In addition to extensive comparisons with the state-of-the-art, we demonstrate our method on a practical Augmented Reality application for maintenance assistance in the ATLAS particle detector at CERN. [ABSTRACT FROM PUBLISHER]

Copyright of Telecommunication Engineering is the property of Telecommunication Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The blades are an important part of gas turbines and internal defects, such as cracks and holes, can occur in some gas turbine blades. Due to the harsh working environment, these defects have a great influence on the safe operation of the gas turbine in practical applications. In this paper, a new method based on a digital radiography (DR) system is adopted to detect defects in gas turbine blades. Considering the limited size of the fiat-panel detector in a DR system and the uneven thickness distribution of the irregular-shaped gas turbine blades, a region segmentation-based inspection method is presented in this paper. That is, the gas turbine blade is divided into a few regions using lead wires and the DR system is then used to inspect these sub-regions. Considering the high cost and low efficiency limitations of high-power computed tomography (Cf) and traditional film-based radiographic testing (RT), certain image processing methods are adopted to accurately automate the extraction and analysis processes. A cubic polynomial interpolation background subtraction method is adopted for the accurate extraction of defects. In addition, a finite element method is presented to obtain the 3D parameter (the volume) of a defect from its 2D radiographic image. Experimental results show that the proposed method can efficiently estimate the 2D and 3D geometric parameters of defects in gas turbine blades at a low cost. The method is particularly suitable for offline X-ray inspection of gas turbine blades. [ABSTRACT FROM AUTHOR]

Copyright of China Sciencepaper is the property of China Sciencepaper and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

3D object detector performance has gradually improved, but it still faces huge challenges for accurate detection from point clouds due to incomplete shapes and sparse point clouds. To solve this problem, recent two-stage approaches usually divide the region of interest (RoI) into regular grids and aggregate the internal features of the RoI within a certain radius for refinement proposals. However, for objects with sparse points, only the features inside the RoI cannot satisfy the need for accurate detection. In view of this objective fact, we present a new two-stage 3D object detection framework from point clouds to 3D voxels inside and outside RoI, named Deformable Pyramid R-CNN. Specially, we firstly propose the Voxel Feature Pyramid, which dynamically selects multi-layers of 3D features based on the sparsity of non-empty voxels within the RoI for proposal refinement. Secondly, a new Deformable Voxel RoI Pooling is introduced to abstract rich context information from the voxels of interest outside RoI, to better align semantic information by deformation grid points set with multiple receptive fields for accurate detection. Our method achieves progressive performance in both KITTI Dataset and Waymo Open Dataset. Especially on KITTI Dataset, our method outperforms PV-RCNN by 0.47%, 1.63%, 1.34% on v a l set e a s y , m o d e r a t e and h a r d levels for car detection, respectively. Code will be available at https://github.com/june2678/DP-RCNN. • We propose Voxel Feature Pyramid to select 3D features via the sparsity of non-empty voxels. • We introduce Deformable Voxel RoI Pooling to abstract context information from outside RoI. • Experiments demonstrate Deformable Pyramid R-CNN achieves progressive performance on Datasets. [ABSTRACT FROM AUTHOR]

A modified U-Net neural network was used to evaluate the laser sharpening quality of diamond grinding wheels, and the laser sharpening parameters were optimized. The three-dimensional (3D) detection algorithm is researched, and a 3D detection algorithm for the diamond wheel surface matching the two-dimensional (2D) image and the 3D point cloud was proposed. The recognized 2D grain image is filtered to remove edge grains and connected grains, correct the 3D point cloud by eliminating the effect of curvature, and match grain pixels. The laser sharpening experiment of the bronze-bonded diamond wheel was carried out by the orthogonal experiment method, and the quality evaluation of the laser sharpening pictures of the wheel obtained by the experiment was carried out. The embedding depth of the abrasive grains was obtained from the 2D area of the abrasive grains, and the evaluation index of abrasive grain height-depth distribution was proposed. The laser sharpening experiments was carried out to obtain grinding wheels with different sharpening qualities, and the grinding tests were carried out. The effectiveness of the sharpening evaluation index was verified by the amount of grinding force when grinding the workpiece and the surface roughness of the workpiece after grinding. The optimal dressing process parameters were obtained as the average power of 35 W, the repetition frequency of 100 kHz, the rotational speed of 300 r/min, and the scanning speed of 3.6 mm/min. [Display omitted] • Three-dimensional (3D) detection algorithm is researched. • A 3D detection algorithm for diamond wheel surface matching 2D image and 3D point cloud was proposed. • Laser sharpening experiment of bronze-bonded diamond wheel was carried out. • Evaluation index of abrasive grain height-depth distribution was proposed. • Effectiveness of sharpening evaluation index was verified and optimal dressing process parameters were obtained. [ABSTRACT FROM AUTHOR]