Search

• An improved YOLOV7-Litchi for litchi detection in complex environments was proposed. • A dataset of litchi fruits in complex orchard environments was established. • The proposed method achieves satisfactory results in lightweight, speed and accuracy. The identification and detection of small fruit targets combined with mechanized harvesting can effectively solve the problems of high labor intensity, high labor cost and low harvesting efficiency in modern litchi harvesting. The orchard environment background of litchi fruits is relatively complex, which makes it difficult for current object detection algorithms to identify small targets and densely distributed litchi fruits. To improve the detection of litchi fruit in complex environments, this paper proposed an improved YOLOv7-Litchi detection algorithm. Compared with the YOLOv7 network, the improved YOLOv7-Litchi integrates ELAN-L and ELAN-A modules based on a lightweight ELAN on a backbone network and replaces the ELAN modules, which make the network structure lightweight. The CNeB module in ConvNeXt was added to the backbone network, and the inverted bottleneck structure and larger convolution kernel size improved the performance of the network by reducing information loss and amplifying the global interactive representation. In addition, a lightweight CBAM module integrating channel attention and spatial attention was added and embedded into the ELAN-A module to better utilize and extract features. Finally, the addition of the transformer module CoT with multihead self-attention enhanced the visual representation of the neural network by tightly combining the contextual information of the input matrix. The experimental results showed that the YOLOv7-Litchi network proposed in this paper provides good detection ability for litchi fruits characterized by a dense distribution and mutual occlusion in complex backgrounds, with a recall rate of 95.90 %, an accuracy of 94.60 %, an mAP of 98.60 %, and an average detection time of 8.6 ms. These results represent improvements of 0.9 %, 1.6 %, 0.6 % and 2.4 ms compared with the recall, accuracy, mAP and average detection time of the YOLOv7 model, respectively. The proposed algorithm has the advantages of high precision, fast detection speed and strong robustness, thus providing a theoretical basis for the mechanized picking of litchis. [ABSTRACT FROM AUTHOR]

Automatic fruit blossom detection plays a crucial role in agricultural intelligence to predict fruit yield. Existing deep learning methods for vision-based fruit blossom detection oftentimes rely on large labeled samples and are tailor for a single category of fruit blossoms, limiting their flexibility and adaptability in real applications. In this paper, a fast and data-efficient framework is proposed to achieve multi-class fruit blossom detection using few labeled samples, which not only enhances the flexibility of model training by reducing the need for extensive samples but also extends the model's applicability to a wider range of blossom categories. Specifically, the proposed framework incorporates the paradigm of few-shot object detection into a lightweight two-stage detector named CenterNet2. To improve the recall of blossom proposals, we introduce a location guidance module (LGM) to highlight foreground regions in query images that bear resemblance to the blossom reference provided by support images. Moreover, a contrastive learning scheme (CLS) is introduced to further distinguish fruit blossom categories with similar appearances, enhancing classification accuracy. Experimental results demonstrated that our framework achieved comparable performance to existing state-of-the-art models, with a mean average precision of 74.33% over 12 categories of fruit blossoms and a frame per second rate of 47. In summary, our proposed framework can reduce data dependency and enhance flexibility of existing deep learning detectors, making it a promising solution for automatic monitoring in real-world agricultural applications. • The proposed framework can detect multi-class fruit blossoms with few labeled samples. • A location guidance module is introduced to guide the framework to produce more accurate blossom proposals. • A contrastive learning scheme is introduced to distinguish between different fruit blossoms with similar appearances. • The proposed framework achieves comparable performance to existing state-of-the-art models and faster inference speed. • The proposed framework can reduce data dependency and enhance flexibility of existing deep learning detectors. [ABSTRACT FROM AUTHOR]

• Proposing an end-to-end noise-tolerant feature fusion network to integrate outdoor RGB-D information. • Designing an attention-based fusion module to eliminate the negative influence of depth noise and make full use of the complementary multi-modal features. • Learning object position from multi-scale features from color images and fusion modules, further improving the network noise immunity. • Achieving accurate fruit detection in outdoor scenes with a real-time speed. In the process of farm automation, fruit detection is the basis and guarantee for yield prediction, automatic picking, and other orchard operations. RGB images can only obtain the two-dimensional information of the scene, which is not sufficient to effectively distinguish fruits that are dense growth and occlusion by branches and leaves. With the development of depth sensors, using RGB-D images with more complementary information can boost the performance of fruit detection. However, due to the nature of sensors and scene configurations, the quality of outdoor depth images is poor, posing a challenge when fusing RGB-D features. Therefore, this paper proposes an end-to-end RGB-D object detection network, termed as noise-tolerant feature fusion network (NT-FFN), to utilize the outdoor multi-modal data properly and improve the detection accuracy. Specifically, the NT-FFN first uses two structurally identical feature extractors to extract single-modal (color and depth) features, which is the base of the subsequent feature fusion. Then, to avoid introducing too much depth noise and focus the perception on the important part of the features, an attention-based fusion module is designed to adaptively fuse the multi-modal features. Finally, multi-scale features from the color images and the fusion modules are used to predict object position, which not only improves the network's ability to detect multi-scale objects but also further enhances the noise immunity of the network. In addition, this paper constructs an RGB-D citrus fruit dataset, which contributes to comprehensively evaluating the proposed network. Evaluation metrics on the dataset show that the NT-FFN achieves an AP50 of 95.4% with a real-time speed, which outperforms single-modal methods, common multi-modal fusion strategies, and advanced multi-modal detection methods. The proposed NT-FFN also achieves excellent detection results in other fruit detection tasks, which verifies its generalization ability. This study provides the possibility and foundation for performing multi-modal information fusion in outdoor fruit detection. [ABSTRACT FROM AUTHOR]

• Complete and accurate counting of fruit on the tree. • High performance single stage deep Convolution Neural Network YOLOX used. • Superior detection rate (≥99% when R ≤ 1.2 m) was obtained. • Excellent robustness in complex orchard scenarios. Fruits counting is important in management of orchard and plantation since better decision for labor and logistic can be made based on complete and accurate counting of fruits. Computer vision-based fruits counting has been research focus as it's an automatic way for recognition of dense fruit on the branch. However, complete fruits counting of a whole tree hasn't hardly been studied. And there is a lack of robust and accurate fruits counting method in complex orchard scenarios, like covering, shadow, clustering in image. In this paper, a panoramic method for fruit complete yield counting based on deep learning object detection is proposed, and was validated on a holly tree with dense fruits. Firstly, images were taken surrounding the fruit trees using UAV, and SIFT based images matching were performed to form a complete panoramic unfolding map of the fruit tree surface. Then, a YOLOX object detection network was built and trained with novel samples augmentation and composition strategies. Finally, fruits counting YOLOX was performed on the panorama to count the whole plant fruits number. The accuracy and effectiveness of this method were tested at different scales and scenarios. The results show that: (1) high-quality panoramic images can be built for an accurate fruit number counting. (2) The Statistical Rate (SR) between detected number and actual number is as high as SR > 96% when the ring shot parameter of Holly tree is R ≤ 1.2 m, SR > 95% when R ≤ 1.6 m. The Detection Rate between detected number and captured number in the panorama image is over 99% when R ≤ 1.2 m and over 97% when R ≤ 2.0 m. The result is superior to previous researches. (3) it has good robustness against shading, covering, incomplete contour. Comparisons between the proposed method and other methods has been done and the result show the proposed method is the most effective in fruits counting. Moreover, we proposed and verified the positive effects of Gaussian convolution kernel and γ-component control on fruit detection rate. The YOLOX-based fruit counting method can be extended to a wide range of fruits, like apples, lychee and so. Moreover, YOLOX has excellent inferencing efficiency which makes it a good potential for real-time application in orchard and plantation management. [ABSTRACT FROM AUTHOR]

• An improved active obstacle separation method for strawberry-harvesting robots. • A faster vision system that used deep learning for object detection and tracking. • The method combined push and drag-push to precisely separate the obstacles. • The separation was controlled in a hybrid visual loop based on the vision system. • The closed visual loop and continuous "look-and-move" made the system more precise. Selectively picking a target fruit surrounded by obstacles remains a challenge for fruit harvesting robots. This paper presents improvements to the active obstacle separation method for strawberry picking in clusters. A faster and more accurate vision system was developed that combined two neural networks and color thresholding for real-time detection, tracking and localization of strawberries. We propose an improved active obstacle separation method that used a push and a drag-push operation to separate the obstacles from the target in three stages. The push and drag vectors were simplified and precisely calculated based on the exact locations of obstacles. Also, different from many systems that only "looked" once for the entire picking process, the new system used a hybrid vision-based control method. In stage 1, the push operation was controlled by a simple closed-loop vision at two key points. In stages 2 and 3, the vision system re-perceived the environment to update the target and obstacle information before each round of drag-push movements. Field evaluation showed that the proposed method was more precise to separate the obstacles without reducing the speed, increasing the whole process success rate to 62.4% in clusters on the "Murano" strawberry cultivator that was 36.8% higher than the previous work. [ABSTRACT FROM AUTHOR]