Search

• Detect and locate fruits based on deep learning is the key to the success of fruit picking. • UAV fuzzy image preprocessing is the basic condition for longan detection. • Reduce the number of calculations and parameters is vital for target detection task. • The positioning strategy of suitable cutting points on fruit branches is elaborated in detail. • The performance of picking platform is evaluated by performing picking experiments. In agriculture, fruit picking robots on the ground have difficulty adapting to the terrain conditions of mountain orchards and cannot pick longan fruit from tall longan trees. In this paper, aiming to allow picking of longan fruit by unmanned aerial vehicles (UAVs), a deep learning-based scheme to quickly and accurately detect and locate suitable picking points on fruit branches is proposed. The scheme includes a UAV fuzzy image preprocessing method, longan detection based on a convolutional neural network (CNN), red, green, blue and depth (RGB-D) information fusion and an accurate target location strategy. First, the UAV is equipped with an Intel Realsense D455 camera, which collects longan images from the front for training and testing the model. Second, the lightweight MobileNet backbone network is used to improve the performance of the You Only Look Once version 4 (YOLOv4) model in feature extraction. The results for the test set show that compared with the classical feature pyramid network (FPN), YOLOv3 and YOLOv4 models, this model reduces the computation, parameters and detection time of the model. Compared with MobileNet single-shot multibox detector (MobileNet-SSD) and YOLOv4-tiny, this model exhibits improved detection accuracy. Third, according to the target detection result map, a strategy is formulated to accurately determine the suitable picking point on the main branch of the result. Finally, the performance of the improved model and picking platform in the harvest scene is evaluated by performing picking experiments in a longan orchard. In summary, we fully exploit the advantages of the combination of UAVs, RGB-D cameras and CNNs to improve the speed and accuracy of target detection and location for longan picking by UAVs based on vision. [ABSTRACT FROM AUTHOR]