PointResNet: A grape bunches point cloud semantic segmentation model based on feature enhancement and improved PointNet++.

As a type of representative bunch-type fruit, the collision-free and undamaged harvesting of grapes is of great significance. To obtain accurate 3D spatial semantic information, this paper proposes a method for multi-feature enhanced semantic segmentation model based on Mask R-CNN and PointResNet(improved PointNet++). Firstly, a depth camera is used to obtain RGBD images. The RGB images are then inputted into the Mask-RCNN network for fast detection of grape bunches. The color and depth information are fused and transformed into point cloud data, followed by the estimation of normal vectors. Finally, the nine-dimensional point cloud, which include spatial location, color information, and surface structure information, are inputted into the PointResNet network to achieve semantic segmentation of grape bunches, peduncles, and leaves. This process obtains the extraction of spatial semantic information from the surrounding area of the bunches. The experimental results show that by incorporating normal vector and color features, the overall accuracy of point cloud segmentation increases to 96.5%, with a mean accuracy of 90.3%. This represents a significant improvement of 7.9% and 16.6% compared to using only positional features. The results demonstrate that the model method presented in this paper can effectively provide precise 3D semantic information to the robot while ensuring both speed and accuracy. This lays the groundwork for subsequent collision-free and undamaged picking. • 2D detection locates grape bunches, extracting regions of interest. • 3D point cloud segmentation identifies grape parts—bunches, peduncles, leaves. • Improved point cloud segmentation network adapts to grape scenes. • Enhanced features for better accuracy. • Experiments show superior recognition using 3D point clouds. [ABSTRACT FROM AUTHOR]