Search

Realizing the integration of intelligent fruit picking and grading for apple harvesting robots is an inevitable requirement for the future development of smart agriculture and precision agriculture. Therefore, an apple maximum diameter estimation model based on RGB-D camera fusion depth information was proposed in the study. Firstly, the maximum diameter parameters of Red Fuji apples were collected, and the results were statistically analyzed. Then, based on the Intel RealSense D435 RGB-D depth camera and LabelImg software, the depth information of apples and the two-dimensional size information of fruit images were obtained. Furthermore, the relationship between fruit depth information, two-dimensional size information of fruit images, and the maximum diameter of apples was explored. Based on Origin software, multiple regression analysis and nonlinear surface fitting were used to analyze the correlation between fruit depth, diagonal length of fruit bounding rectangle, and maximum diameter. A model for estimating the maximum diameter of apples was constructed. Finally, the constructed maximum diameter estimation model was experimentally validated and evaluated for imitation apples in the laboratory and fruits on the Red Fuji fruit trees in modern apple orchards. The experimental results showed that the average maximum relative error of the constructed model in the laboratory imitation apple validation set was ±4.1%, the correlation coefficient (R2) of the estimated model was 0.98613, and the root mean square error (RMSE) was 3.21 mm. The average maximum diameter estimation relative error on the modern orchard Red Fuji apple validation set was ±3.77%, the correlation coefficient (R2) of the estimation model was 0.84, and the root mean square error (RMSE) was 3.95 mm. The proposed model can provide theoretical basis and technical support for the selective apple-picking operation of intelligent robots based on apple size grading. [ABSTRACT FROM AUTHOR]

The article proposes a real‐time apple picking method based on an improved YOLOv5. This method accurately recognizes different apple targets on fruit trees for robots and helps them adjust their position to avoid obstructions during fruit picking. Firstly, the original BottleneckCSP module in the YOLOv5 backbone network is enhanced to extract deeper features from images while maintaining lightweight. Secondly, the ECA module is embedded into the improved backbone network to better extract features of different apple targets. Lastly, the initial anchor frame size of the network is adjusted to avoid recognizing apples in distant planting rows. The results demonstrate that this improved model achieves high accuracy rates and recall rates for recognizing various types of apple picking methods with an average recognition time of 0.025s per image. Compared with other models tested on six types of apple picking methods, including the original YOLOv5 model as well as YOLOv3 and EfficientDet‐D0 algorithms, our improved model shows significant improvements in mAP by 1.95%, 17.6%, and 12.7% respectively. This method provides technical support for robots' picking hands to actively avoid obstructions caused by branches during fruit harvesting, effectively reducing apple loss. [ABSTRACT FROM AUTHOR]

Decreased availability and rising cost in labor poses a serious threat to the long‐term profitability and sustainability of the apple industry in the United States and many other countries. Harvest automation is thus urgently needed. In this paper, we present the unified system design and field evaluation of a new apple harvesting robot. The robot is mainly composed of a specially designed perception component, a four‐degree‐of‐freedom manipulator, an improved vacuum‐based soft end‐effector, and a dropping/catching component to receive and transport picked fruits. Software algorithms are developed to enable synergistic coordination of the hardware components for efficient, automated harvesting of apples in challenging orchard environments. Specifically, by integrating modified triangulation and image processing and analysis algorithms, a novel perception strategy is developed to achieve robust apple detection and precise localization. Improved planning and control algorithms are developed to guide the robot to the target positions. The performance of the robotic system was evaluated through field tests in two apple orchards with different tree architectures and foliage conditions. In the orchard where trees were young and well‐pruned, the robot achieved 82.4% successful harvesting rate. In a second, older orchard with dense, clustered branches and foliage, the robot had 65.2% successful rate. The average cycle time to harvest a fruit was approximately 6 s, which included software algorithm processing and hardware execution. Moreover, through an in‐depth analysis of the obtained results, limitations and planned future works are discussed. [ABSTRACT FROM AUTHOR]

Aiming to accurately identify apple targets and achieve segmentation and the extraction of branch and trunk areas of apple trees, providing visual guidance for a picking robot to actively adjust its posture to avoid branch trunks for obstacle avoidance fruit picking, the spindle-shaped fruit trees, which are widely planted in standard modern apple orchards, were focused on, and an algorithm for apple tree fruit detection and branch segmentation for picking robots was proposed based on an improved YOLOv8s model design. Firstly, image data of spindle-shaped fruit trees in modern apple orchards were collected, and annotations of object detection and pixel-level segmentation were conducted on the data. Training set data were then augmented to improve the generalization performance of the apple detection and branch segmentation algorithm. Secondly, the original YOLOv8s network architecture's design was improved by embedding the SE module visual attention mechanism after the C2f module of the YOLOv8s Backbone network architecture. Finally, the dynamic snake convolution module was embedded into the Neck structure of the YOLOv8s network architecture to better extract feature information of different apple targets and tree branches. The experimental results showed that the proposed improved algorithm can effectively recognize apple targets in images and segment tree branches and trunks. For apple recognition, the precision was 99.6%, the recall was 96.8%, and the mAP value was 98.3%. The mAP value for branch and trunk segmentation was 81.6%. The proposed improved YOLOv8s algorithm design was compared with the original YOLOv8s, YOLOv8n, and YOLOv5s algorithms for the recognition of apple targets and segmentation of tree branches and trunks on test set images. The experimental results showed that compared with the other three algorithms, the proposed algorithm increased the mAP for apple recognition by 1.5%, 2.3%, and 6%, respectively. The mAP for tree branch and trunk segmentation was increased by 3.7%, 15.4%, and 24.4%, respectively. The proposed detection and segmentation algorithm for apple tree fruits, branches, and trunks is of great significance for ensuring the success rate of robot harvesting, which can provide technical support for the development of an intelligent apple harvesting robot. [ABSTRACT FROM AUTHOR]

Copyright of Revista Brasileira de Engenharia Agricola e Ambiental - Agriambi is the property of Revista Brasileira de Engenharia Agricola e Ambiental and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

To address the issue of automated apple harvesting in orchards, we propose a YOLOv5-RACF algorithm for identifying apples and calculating apple diameters. This algorithm employs the robot operating dystem (ROS) to control the robot's locomotion system, Lidar mapping, and navigation, as well as the robotic arm's posture and grasping operations, achieving automated apple harvesting and placement. The tests were conducted in an actual orchard environment. The algorithm model achieved an average apple detection accuracy (mAP@0.5) of 98.748% and a (mAP@0.5:0.95) of 90.02%. The time to calculate the diameter of one apple was 0.13 s, with a measurement accuracy within an error range of 1–3 mm. The robot takes an average of 9 s to pick an apple and return to the initial pose. These results demonstrate the system's efficiency and reliability in real agricultural environments. [ABSTRACT FROM AUTHOR]

Accurately recognizing apples in complex environments is essential for automating apple picking operations, particularly under challenging natural conditions such as cloudy, snowy, foggy, and rainy weather, as well as low-light situations. To overcome the challenges of reduced apple target detection accuracy due to branch occlusion, apple overlap, and variations between near and far field scales, we propose the Rep-ViG-Apple algorithm, an advanced version of the YOLO model. The Rep-ViG-Apple algorithm features a sophisticated architecture designed to enhance apple detection performance in difficult conditions. To improve feature extraction for occluded and overlapped apple targets, we developed the inverted residual multi-scale structural reparameterized feature extraction block (RepIRD Block) within the backbone network. We also integrated the sparse graph attention mechanism (SVGA) to capture global feature information, concentrate attention on apples, and reduce interference from complex environmental features. Moreover, we designed a feature extraction network with a CNN-GCN architecture, termed Rep-Vision-GCN. This network combines the local multi-scale feature extraction capabilities of a convolutional neural network (CNN) with the global modeling strengths of a graph convolutional network (GCN), enhancing the extraction of apple features. The RepConvsBlock module, embedded in the neck network, forms the Rep-FPN-PAN feature fusion network, which improves the recognition of apple targets across various scales, both near and far. Furthermore, we implemented a channel pruning algorithm based on LAMP scores to balance computational efficiency with model accuracy. Experimental results demonstrate that the Rep-ViG-Apple algorithm achieves precision, recall, and average accuracy of 92.5%, 85.0%, and 93.3%, respectively, marking improvements of 1.5%, 1.5%, and 2.0% over YOLOv8n. Additionally, the Rep-ViG-Apple model benefits from a 22% reduction in size, enhancing its efficiency and suitability for deployment in resource-constrained environments while maintaining high accuracy. [ABSTRACT FROM AUTHOR]

The cultivation model for spindle-shaped apple trees is widely used in modern standard apple orchards worldwide and represents the direction of modern apple industry development. However, without an effective obstacle avoidance path, the robotic arm is prone to collision with obstacles such as fruit tree branches during the picking process, which may damage fruits and branches and even affect the healthy growth of fruit trees. To address the above issues, a three-dimensional path -planning algorithm for full-field fruit obstacle avoidance harvesting for spindle-shaped fruit trees, which are widely planted in modern apple orchards, is proposed in this study. Firstly, based on three typical tree structures of spindle-shaped apple trees (free spindle, high spindle, and slender spindle), a three-dimensional spatial model of fruit tree branches was established. Secondly, based on the grid environment representation method, an obstacle map of the apple tree model was established. Then, the initial pheromones were improved by non-uniform distribution on the basis of the original ant colony algorithm. Furthermore, the updating rules of pheromones were improved, and a biomimetic optimization mechanism was integrated with the beetle antenna algorithm to improve the speed and stability of path searching. Finally, the planned path was smoothed using a cubic B-spline curve to make the path smoother and avoid unnecessary pauses or turns during the harvesting process of the robotic arm. Based on the proposed improved ACO algorithm (ant colony optimization algorithm), obstacle avoidance 3D path planning simulation experiments were conducted for three types of spindle-shaped apple trees. The results showed that the success rates of obstacle avoidance path planning were higher than 96%, 86%, and 92% for free-spindle-shaped, high-spindle-shaped, and slender-spindle-shaped trees, respectively. Compared with traditional ant colony algorithms, the average planning time was decreased by 49.38%, 46.33%, and 51.03%, respectively. The proposed improved algorithm can effectively achieve three-dimensional path planning for obstacle avoidance picking, thereby providing technical support for the development of intelligent apple picking robots. [ABSTRACT FROM AUTHOR]

The growing development of the agricultural e-commerce industry is promoting the Digital transformation of agricultural production and logistics. Aiming at the problem of labor shortage in apple picking, this paper proposes a performance control method for robot apple picking based on a nonlinear wide convolutional neural network (NLWCNN) and multi-objective cooperative distance transformation algorithm (MOCDTA). This method utilizes deep learning algorithms and visual sensing technology to enable robots to accurately recognize apples and perform autonomous picking operations based on their maturity and position. We have designed a machine learning-based algorithm that enables robots to accurately recognize and distinguish apples of different maturity levels through training on a large amount of sample data. At the same time, we also combined the motion control algorithm of the robot to enable efficient picking operations based on the position and maturity information of the apples. The experimental results indicate that the NLWCNN method can significantly improve the efficiency and accuracy of apple picking, ensure the safety and traceability of agricultural products, increase consumer trust, and promote agricultural product transactions. [ABSTRACT FROM AUTHOR]

Apple is easily damaged in the process of the mechanical harvesting, which reduces the fruit's quality. It is of great significance to study the damage principle of apple in the transport process of picking platform. In this study, the apple compression test was carried out. The compression and drop process of the fruit was analyzed by the finite element analysis (FEA). The experimental platform of apple harvesting was designed, the conveying process of apple was analyzed. The results of compression finite element analysis showed that when the compression force is greater than 15.0 N, both radial compression and axial compression will be damaged. The results of drop finite element analysis showed that when the drop direction is axial, the maximum contact stress of the peel and kernel decreases with the increase of drop angle, the maximum contact stress of the pulp increases first and then decreases. When the drop direction is radial, the maximum contact stress between pulp and kernel decreases with the increase of drop angle, the maximum contact stress of the peel first decreases and then increases. The simulation results of the harvesting platform transportation showed that the damage rate of apples is less than 10 % when the sub-conveyor belt speed is 0.02 m–0.04 m/s. This study can provide theoretical guidance for the design of the harvesting test platform and the reduction of the damage of apples during transportation. [ABSTRACT FROM AUTHOR]

The rapid development of artificial intelligence and remote sensing technologies is indispensable for modern agriculture. In orchard environments, challenges such as varying light conditions and shading complicate the tasks of intelligent picking robots. To enhance the recognition accuracy and efficiency of apple-picking robots, this study aimed to achieve high detection accuracy in complex orchard environments while reducing model computation and time consumption. This study utilized the CenterNet neural network as the detection framework, introducing gray-centered RGB color space vertical decomposition maps and employing grouped convolutions and depth-separable convolutions to design a lightweight feature extraction network, Light-Weight Net, comprising eight bottleneck structures. Based on the recognition results, the 3D coordinates of the picking point were determined within the camera coordinate system by using the transformation relationship between the image's physical coordinate system and the camera coordinate system, along with depth map distance information of the depth map. Experimental results obtained using a testbed with an orchard-picking robot indicated that the proposed model achieved an average precision (AP) of 96.80% on the test set, with real-time performance of 18.91 frames per second (FPS) and a model size of only 17.56 MB. In addition, the root-mean-square error of positioning accuracy in the orchard test was 4.405 mm, satisfying the high-precision positioning requirements of the picking robot vision system in complex orchard environments. [ABSTRACT FROM AUTHOR]

The quality of apple picking affects the sales of apples, and the grasping force of the end effector of an apple picking robot is very important for apple picking. It is easy to cause apple damage due to excessive contact force, or when the contact force is too small to grasp the apple. However, the current research lacks an analysis of the minimum stable grasping force of apples. Therefore, in order to realize the stable grasping of apples by the end-effector of a picking robot and reduce fruit damage, this study first analyzes the grasping stability of the end-effector based on the force closure theory, and comprehensively considers the force closure constraints, nonlinear friction cone constraints and the introduction of torque constraints. Next, the constraint conditions are processed using an obstacle function, and a penalty factor is introduced to construct an optimization model of the contact force distribution of the end-effector. Then, the improved Newton method is used to grasp and solve the contact force distribution model. Under the premise of selecting the penalty factor, the optimal contact force of grasping an apple is determined using a method of numerical example simulation analysis, and the validity of the solution is verified. In order to verify the reliability of the contact force distribution optimization model, the practical significance of the method for apple grasping is verified in an actual grasping experiment. The actual experiment shows that the method can provide the minimum stable grasping force to the end-effector to achieve stable grasping. [ABSTRACT FROM AUTHOR]

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations.

Intelligent apple-picking robots can significantly improve the efficiency of apple picking, and the realization of fast and accurate recognition and localization of apples is the prerequisite and foundation for the operation of picking robots. Existing apple recognition and localization methods primarily focus on object detection and semantic segmentation techniques. However, these methods often suffer from localization errors when facing occlusion and overlapping issues. Furthermore, the few instance segmentation methods are also inefficient and heavily dependent on detection results. Therefore, this paper proposes an apple recognition and localization method based on RGB-D and an improved SOLOv2 instance segmentation approach. To improve the efficiency of the instance segmentation network, the EfficientNetV2 is employed as the feature extraction network, known for its high parameter efficiency. To enhance segmentation accuracy when apples are occluded or overlapping, a lightweight spatial attention module is proposed. This module improves the model position sensitivity so that positional features can differentiate between overlapping objects when their semantic features are similar. To accurately determine the apple-picking points, an RGB-D-based apple localization method is introduced. Through comparative experimental analysis, the improved SOLOv2 instance segmentation method has demonstrated remarkable performance. Compared to SOLOv2, the F1 score, mAP, and mIoU on the apple instance segmentation dataset have increased by 2.4, 3.6, and 3.8%, respectively. Additionally, the model's Params and FLOPs have decreased by 1.94M and 31 GFLOPs, respectively. A total of 60 samples were gathered for the analysis of localization errors. The findings indicate that the proposed method achieves high precision in localization, with errors in the X, Y, and Z axes ranging from 0 to 3.95mm, 0 to 5.16mm, and 0 to 1mm, respectively. [ABSTRACT FROM AUTHOR]

Accurate localization of apples is the key factor that determines a successful harvesting cycle in the automation of apple harvesting for unmanned operations. In this regard, accurate depth sensing or positional information of apples is required for harvesting apples based on robotic systems, which is challenging in outdoor environments because of uneven light variations when using 3D cameras for the localization of apples. Therefore, this research attempted to overcome the effect of light variations for the 3D cameras during outdoor apple harvesting operations. Thus, integrated single-point laser sensors for the localization of apples using a state-of-the-art model, the EfficientDet object detection algorithm with an mAP@0.5 of 0.775 were used in this study. In the experiments, a RealSense D455f RGB-D camera was integrated with a single-point laser ranging sensor utilized to obtain precise apple localization coordinates for implementation in a harvesting robot. The single-point laser range sensor was attached to two servo motors capable of moving the center position of the detected apples based on the detection ID generated by the DeepSORT (online real-time tracking) algorithm. The experiments were conducted under indoor and outdoor conditions in a spindle-type apple orchard artificial architecture by mounting the combined sensor system behind a four-wheel tractor. The localization coordinates were compared between the RGB-D camera depth values and the combined sensor system under different light conditions. The results show that the root-mean-square error (RMSE) values of the RGB-D camera depth and integrated sensor mechanism varied from 3.91 to 8.36 cm and from 1.62 to 2.13 cm under 476~600 lx to 1023~1100 × 100 lx light conditions, respectively. The integrated sensor system can be used for an apple harvesting robotic manipulator with a positional accuracy of ±2 cm, except for some apples that were occluded due to leaves and branches. Further research will be carried out using changes in the position of the integrated system for recognition of the affected apples for harvesting operations. [ABSTRACT FROM AUTHOR]

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations. [ABSTRACT FROM AUTHOR]

In the realm of automated apple picking operations, the real-time monitoring of apple maturity and diameter characteristics is of paramount importance. Given the constraints associated with feature detection of apples in automated harvesting, this study proposes a machine vision-based methodology for the accurate identification of Fuji apples' maturity and diameter. Firstly, maturity level detection employed an improved YOLOv5s object detection model. The feature fusion section of the YOLOv5s network was optimized by introducing the cross-level partial network module VoVGSCSP and lightweight convolution GSConv. This optimization aimed to improve the model's multiscale feature information fusion ability while accelerating inference speed and reducing parameter count. Within the enhanced feature fusion network, a dual attention mechanism combining channel and spatial attention (GAM) was introduced to refine the color and texture feature information of apples and to increase spatial position feature weights. In terms of diameter determination, the contours of apples are obtained by integrating the dual features of color and depth images within the target boxes acquired using the maturity detection model. Subsequently, the actual area of the apple contour is determined by calculating the conversion relationship between pixel area and real area at the current depth value, thereby obtaining the diameter of the apples. Experimental results showed that the improved YOLOv5s model achieved an average maturity level detection precision of 98.7%. Particularly noteworthy was the detection accuracy for low maturity apples, reaching 97.4%, surpassing Faster R-CNN, Mask R-CNN, YOLOv7, and YOLOv5s models by 6.6%, 5.5%, 10.1%, and 11.0% with a real-time detection frame rate of 155 FPS. Diameter detection achieved a success rate of 93.3% with a real-time detection frame rate of 56 FPS and an average diameter deviation of 0.878 mm for 10 apple targets across three trials. Finally, the proposed method achieved an average precision of 98.7% for online detection of apple maturity level and 93.3% for fruit diameter features. The overall real-time inference speed was approximately 56 frames per second. These findings indicated that the method met the requirements of real-time mechanical harvesting operations, offering practical importance for the advancement of the apple industry. [ABSTRACT FROM AUTHOR]

The key technology of automated apple harvesting is detecting apples quickly and accurately. The traditional detection methods of apple detection are often slow and inaccurate in unstructured orchards. Therefore, this article proposes an improved YOLOv5s-GBR model for orchard apple detection under complex natural conditions. First, the researchers collected photos of apples in their natural environments from different angles; then, we enhanced the dataset by changing the brightness, rotating the images, and adding noise. In the YOLOv5s network, the following modules were introduced to improve its performance: First, the YOLOv5s model's backbone network was swapped out for the GhostNetV2 module. The goal of this improvement was to lessen the computational burden on the YOLOv5s algorithm while increasing the detection speed. Second, the bi-level routing spatial attention module (BRSAM), which combines spatial attention (SA) with bi-level routing attention (BRA), was used in this study. By strengthening the model's capacity to extract important characteristics from the target, its generality and robustness were enhanced. Lastly, this research replaced the original bounding box loss function with a repulsion loss function to detect overlapping targets. This model performs better in detection, especially in situations involving occluded and overlapping targets. According to the test results, the YOLOv5s-GBR model improved the average precision by 4.1% and recall by 4.0% compared to those of the original YOLOv5s model, with an impressive detection accuracy of 98.20% at a frame rate of only 101.2 fps. The improved algorithm increases the recognition accuracy by 12.7%, 10.6%, 5.9%, 2.7%, 1.9%, 0.8%, 2.6%, and 5.3% compared to those of YOLOv5-lite-s, YOLOv5-lite-e, yolov4-tiny, YOLOv5m, YOLOv5l, YOLOv8s, Faster R-CNN, and SSD, respectively, and the YOLOv5s-GBR model can be used to accurately recognize overlapping or occluded apples, which can be subsequently deployed in picked robots to meet the realistic demand of real-time apple detection. [ABSTRACT FROM AUTHOR]

The use of automation technology in agriculture has become particularly important as global agriculture is challenged by labor shortages and efficiency gains. The automated process for harvesting apples, an important agricultural product, relies on efficient and accurate detection and localization technology to ensure the quality and quantity of production. Adverse lighting conditions can significantly reduce the accuracy of fruit detection and localization in automated apple harvesting. Based on deep-learning techniques, this study aims to develop an accurate fruit detection and localization method under adverse light conditions. This paper explores the LE-YOLO model for accurate and robust apple detection and localization. The traditional YOLOv5 network was enhanced by adding an image enhancement module and an attention mechanism. Additionally, the loss function was improved to enhance detection performance. Secondly, the enhanced network was integrated with a binocular camera to achieve precise apple localization even under adverse lighting conditions. This was accomplished by calculating the 3D coordinates of feature points using the binocular localization principle. Finally, detection and localization experiments were conducted on the established dataset of apples under adverse lighting conditions. The experimental results indicate that LE-YOLO achieves higher accuracy in detection and localization compared to other target detection models. This demonstrates that LE-YOLO is more competitive in apple detection and localization under adverse light conditions. Compared to traditional manual and general automated harvesting, our method enables automated work under various adverse light conditions, significantly improving harvesting efficiency, reducing labor costs, and providing a feasible solution for automation in the field of apple harvesting. [ABSTRACT FROM AUTHOR]

Robots picking apples cause vibration of other fruit on the same branch and fruit may even fall off. This not only affects subsequent picking, but also causes fruit loss. In this research, the branch-fruit system was taken as the research object, and a mixed-mode cohesive contact model was used to simulate the tangential and directional mechanical behaviour of fruit detachment. The branch with apples was simplified as a cantilever beam model, and a multi-body finite element model of a branch with apples was established. The average error of the simulation results was 6.81%. The simulation results demonstrate that the type I picking sequence (picking fruit beginning farther from the trunk) shortened the settling time of the branch with apples than the type II picking sequence (picking fruit beginning closer to the trunk). Field experiments showed that the maximum accelerations of fruit detachment were respectively 3.826 ± 1.859 g , 2.971 ± 1.374 g , 2.246 ± 0.757 g (g = 9.8 m s−2) under three picking patterns (pulling, pull-twisting, and pull-bending). Linear correlations were found between the vibration displacement and settling time of fruit under different picking patterns and sequences. The response index (RI) is defined to comprehensively evaluate the degree of disturbance to other fruits on the same branch during the fruit-picking process. The test results reveal that the use of the type I sequence and pull-bending pattern is better for reducing the overall disturbance to other apples on the same branch. This study provides a technical and theoretical reference for designing the picking pattern and planning the picking sequence of robotic arms. • Proposing a multi-body finite element model for a branch with apples. • Using a mixed-mode cohesive contact model to simulate fruit detachment behavior. • The response index (RI) measures disturbance to adjacent fruits during picking. • Different picking patterns and sequences disturb unpicked fruit differently. [ABSTRACT FROM AUTHOR]

In order to improve the efficiency of robots picking apples in challenging orchard environments, a method for precisely detecting apples and planning the picking sequence is proposed. Firstly, the EfficientFormer network serves as the foundation for YOLOV5, which uses the EF-YOLOV5s network to locate apples in difficult situations. Meanwhile, the soft non-maximum suppression algorithm is adopted to achieve accurate identification of overlapping apples. Secondly, the adjacently identified apples are automatically divided into different picking clusters by the improved density-based spatial clustering of applications with noise. Finally, the order of apple harvest is determined to guide the robot to complete the rapid picking, according to the weight of the Gauss distance weight combined with the significance level. In the experiment, the average precision of this method is 98.84%, which is 4.3% higher than that of YOLOV5s. Meanwhile, the average picking success rate and picking time are 94.8% and 2.86 seconds, respectively. Compared with sequential and random planning, the picking success rate of the proposed method is increased by 6.8% and 13.1%, respectively. The research proves that this method can accurately detect apples in complex environments and improve picking efficiency, which can provide technical support for harvesting robots. [ABSTRACT FROM AUTHOR]

During the picking process of the apple harvesting robot, the attitude of the end effector holding the apple and the movement method of separating the apple directly affect the success rate of picking. In order to improve the stability of the picking process, reduce the gripping force, and avoid apple dislodgement and damage, this work studies the new apple-picking pattern of the flexible three-fingered end-effector based on the analysis of the existing apple-picking pattern. First, two new three-finger grasping postures for wrapping the apple horizontally and vertically on the inside of the fingers are proposed, and a new method of separating the stem with a circular-pull-down motion of the end-effector picking the apple is designed. Then, the pressure on the apple under different picking patterns was analyzed, and a branch–stem–apple simulation model was established. Combining the constraint conditions such as the angle between the apple stem and the vertical direction, the movement speed, the root impulse, and so on, the optimal angle of apple circular movement and the force required to realize the movement are obtained through dynamic simulation experiments. Finally, the experiments of apple picking patterns were carried out with the flexible three-fingered end-effector. The experiment shows that the best angle for apple picking is 15°~20° using the circular-pull-down movement separation method. In terms of average grasping force peaks and pressures, the combination of the vertical holding posture of the inner finger and the circular-pull-down movement separation method is the best picking pattern. In this pattern, the average peak exerts force on the inner side of a single finger is about 8.52 N, and the pressure is about 20.9 KPa. [ABSTRACT FROM AUTHOR]

The real-time and accurate recognition of fruits and vegetables is crucial for the intelligent control of fruit and vegetable robots. In this research, apple picking is selected. This study proposed a lightweight, coupon-product, neural-net terminal YOLO algorithm for apple image recognition. Compared with the YOLO series algorithm, the tiny algorithm shows a strong relationship with the calculation speed. In traditional red fruit detection, the recognition time is generally several seconds, which is unacceptable in the real-time system. In this research, a total of 2000 apple images from different environments are used as a dataset for training and testing. The YOLOv4-tiny model is detailed, instructed, and used for the identification. The indicators, such as F1Score (0.92) and mAP (95.5% average), are analyzed by calculating the loss rate, accuracy rate (96.21%), and recall rate (95.47%). Finally, the algorithm shows good accuracy and high speed (no more than 5 ms) in online real-time detection. [ABSTRACT FROM AUTHOR]

The book review discusses "Folk Horror on Film: Return of the British Repressed," edited by Louis Bayman and K. J. Donnelly. The review highlights the recent increase in writing on the topic of folk horror, with several edited collections published in 2023. The editors aim to define the term "folk horror" and explore it in sociological terms, focusing specifically on British film examples. The book is divided into three sections, each examining different aspects of folk horror. While the contributors do not reach a consensus on the definition of folk horror, the book provides valuable insights and contributes to ongoing scholarly discussions on the subject. [Extracted from the article]

To address the issue of automated apple harvesting in orchards, we propose a YOLOv5-RACF algorithm for identifying apples and calculating apple diameters. This algorithm employs the robot operating dystem (ROS) to control the robot’s locomotion system, Lidar mapping, and navigation, as well as the robotic arm’s posture and grasping operations, achieving automated apple harvesting and placement. The tests were conducted in an actual orchard environment. The algorithm model achieved an average apple detection accuracy (mAP@0.5) of 98.748% and a (mAP@0.5:0.95) of 90.02%. The time to calculate the diameter of one apple was 0.13 s, with a measurement accuracy within an error range of 1–3 mm. The robot takes an average of 9 s to pick an apple and return to the initial pose. These results demonstrate the system’s efficiency and reliability in real agricultural environments.

Introduction: With continuously increasing labor costs, an urgent need for automated apple- Qpicking equipment has emerged in the agricultural sector. Prior to apple harvesting, it is imperative that the equipment not only accurately locates the apples, but also discerns the graspability of the fruit. While numerous studies on apple detection have been conducted, the challenges related to determining apple graspability remain unresolved. Methods: This study introduces a method for detecting multi-occluded apples based on an enhanced YOLOv5s model, with the aim of identifying the type of apple occlusion in complex orchard environments and determining apple graspability. Using bootstrap your own atent(BYOL) and knowledge transfer(KT) strategies, we effectively enhance the classification accuracy for multi-occluded apples while reducing data production costs. A selective kernel (SK) module is also incorporated, enabling the network model to more precisely identify various apple occlusion types. To evaluate the performance of our network model, we define three key metrics: APGA, APTUGA, and APUGA, representing the average detection accuracy for graspable, temporarily ungraspable, and ungraspable apples, respectively. Results: Experimental results indicate that the improved YOLOv5s model performs exceptionally well, achieving detection accuracies of 94.78%, 93.86%, and 94.98% for APGA, APTUGA, and APUGA, respectively. Discussion: Compared to current lightweight network models such as YOLOX-s and YOLOv7s, our proposed method demonstrates significant advantages across multiple evaluation metrics. In future research, we intend to integrate fruit posture and occlusion detection to f]urther enhance the visual perception capabilities of apple-picking equipment. [ABSTRACT FROM AUTHOR]

Automatic apple harvesting robots have received much research attention in recent years to lower harvesting costs. A fundamental problem for harvesting robots is how to quickly and accurately detect branches to avoid collisions with limited hardware resources. In this paper, we propose a lightweight, high-accurate and real-time semantic segmentation network, Bilateral U-shape Network (BUNet), to segment apple tree branches. The BUNet consists mainly of a U-shaped detail branch and a U-shaped semantic branch, the former for capturing spatial details and the latter for supplementing semantic information. These two U-shape branches complement each other, keeping the high accuracy of the Encoder-decoder Backbone while maintaining the efficiency and effectiveness of the Two-pathway Backbone. In addition, a Simplified Attention Fusion Module (SAFM) is proposed to effectively fuse different levels of information from two branches for pixel-by-pixel prediction. Experimental results show (on our own constructed dataset) that BUNet achieves the highest Intersection over Union (IoU) and F1-score of 75.96% and 86.34%, respectively, with minimum parameters of 0.93M and 11.94G Floating-point of Operations (FLOPs) in branch segmentation. Meanwhile, BUNet achieves a speed of 110.32 Frames Per Second (FPS) with input image size of 1280 × 720 pixels. These results confirm that the proposed method can effectively detect the branches and can, therefore, be used to plan an obstacle avoidance path for harvesting robots. [ABSTRACT FROM AUTHOR]

In the functioning of the hand–eye collaboration of an apple picking robot, the accuracy of the hand–eye relationship is a key factor affecting the efficiency and accuracy of the robot's operation. In order to enhance the low accuracy of traditional hand–eye calibration methods, linear and nonlinear solving methods based on mathematical tools such as quaternions are commonly adopted. To solve the loss of accuracy in decoupling during the linearization solution and to reduce the cumulative error that occurs during nonlinear solutions, a hand–eye calibration method, based on the ICP algorithm, is proposed in this paper. The method initializes the ICP matching algorithm with a solution derived from Tsai–Lenz, and substitutes it for iterative computation, thereby ascertaining a precise hand–eye conversion relationship by optimizing the error threshold and iteration count in the ICP matching process. Experimental results demonstrate that the ICP-based hand–eye calibration optimization algorithm not only circumvents the issues pertaining to accuracy loss and significant errors during solving, but also enhances the rotation accuracy by 13.6% and the translation accuracy by 2.47% compared with the work presented by Tsai–Lenz. [ABSTRACT FROM AUTHOR]

In recent years, with the gradual expansion of the planting area for Chinese crab apple trees, the traditional method of picking crab apples has become inadequate due to its large workload, low efficiency, and high cost. To address this issue, this paper presents the design of a trunk-type vibration picker specifically for crab apple picking. The design process began with a modal analysis and a harmonic response analysis of a crab apple tree using Ansys Workbench. Through these analyses, the optimal excitation frequency for the crab apple tree was determined to be 12.7 Hz. Subsequently, the picker was designed, and its key components underwent modal and stress analyses. A comparison revealed no resonance between the picker and the key components, ensuring the feasibility of the picker. To further evaluate the performance of the picker, a rigid–flexible coupling analysis was conducted using Adams. This analysis determined the acceleration response curves of the lateral branch points of the crab apple tree at various excitation frequencies: 9.7 Hz, 12.7 Hz, 15.7 Hz, and 18.7 Hz. Finally, field trials were carried out to validate the reliability of the data obtained from Adams. The results showed that the lateral branch point acceleration increased with higher excitation frequencies. However, when the optimal excitation frequency of 12.7 Hz was reached, the lateral branch point acceleration decreased. This indicated that the most effective vibration frequency for picking crab apples was 12.7 Hz. [ABSTRACT FROM AUTHOR]

In recent years, the problem of "rural labor shortage" in China has become increasingly serious, with a large number of young laborers going out to work, leading to an increasing amount of idle rural land. With the intensification of population aging and the reduction of agricultural labor force in China, resulting in an urgent demand for agricultural robots. With the rapid development of agricultural machinery and automation technology, agricultural robots have been continuously developing. They can better adapt to the development of biotechnology in agriculture, and traditional harvesting methods may undergo significant changes, with an increased focus on the cultivation of crops. Therefore, this paper introduces a new encoding scheme on the basis of traditional genetic algorithm (GA) and proposes an improved double encoding GA. This new encoding scheme is used on the crossover link, whereas the path node sequence encoding scheme is still used on the mutation link. The selection operation is placed after the mutation, and the merging sorting and elitist selection are performed on the parent population, crossover population, and mutation population before selection, thereby accelerating the convergence speed. On the basis of the improved GA, the three-dimensional path of the apple harvesting robot is designed and planned, with the addition of adaptive adjustment function during the progress. The experimental simulation results show that the three-dimensional path planning of the apple harvesting mobile robot based on the improved GA can minimize the number of paths and loops and well meet the operational requirements of the harvesting robot. [ABSTRACT FROM AUTHOR]

For the standardized apple orchards in China, which are mainly dwarfed and densely planted, firstly, according to the spatial distribution characteristics of fruits within the tree canopy, a clamping-pull-off apple picking robot was developed by analyzing the parameters of apple cultivation and picking methods, in order to replace the manual harvesting operation. Then, the Denavit-Hartenberg (DH) method was used to establish the kinematic Equations of the apple-picking robot, the forward and inverse kinematic calculations were carried out, and the Monte Carlo method was used to analyze the workspace of the robot. Through the robot picking task planning and the simulation of the trajectory of the robotic arm, the scheme of the robot's picking strategy was given, and MATLAB software was applied to simulate the motion trajectory as well as to verify the feasibility of the trajectory planning scheme and the picking strategy. Finally, an apple-picking test bed was set up, the corresponding picking control system program was designed, and 45 apples were selected for picking tests. The results showed that during the robot's picking process, the average time for picking each fruit was 7.59 seconds, the fruit recognition success rate was 86.67%, and the picking damage rate was 5.13%. [ABSTRACT FROM AUTHOR]

The end effector plays a critical role in fruit picking by a robotic fruit harvesting system. A newly developed vacuum-based robotic apple harvesting system, using the twist-and-pull fruit picking method, has shown promise in picking fruit from clusters and navigating through the tree canopies. The robot's original thin foam end effector failed to achieve acceptable picking performance because it was unable to conform to fruit of different sizes and thus could not generate sufficient suction forces needed to detach fruit. This research was therefore aimed at developing new end effectors to greatly enhance the robot's fruit picking performance. Field manual pulling and twisting experiments for three varieties of apples were conducted, and the critical pulling or suction force and twisting torque needed to detach 95% of apples were determined to be 28.3 N and 0.257 N-m (equivalent to a pulling force of 21.0 N for the current robot's configuration), respectively. Three new silicone-based end effectors of different geometries (denoted as "Straight", "Bellow" and "Curved") were designed and fabricated, and they were evaluated in lab and field experiments. Results showed that the three new end effectors performed significantly better than the original, unconformable end effector based on multiple performance metrics, including vacuum pressure, overall picking success rate, picking rate by the rotation mechanism, and fruit attachment orientation. The "Straight" end effector performed consistently better than the other two new end effectors; it had 87% overall picking success rate, 65% success rate by the rotation mechanism alone, and 89% success rate when the middle section or cheek of fruit attached to the end effector. With further improvements, the "Straight" end effector should meet the apple picking performance requirement for the new vacuum-based robotic harvesting system. [ABSTRACT FROM AUTHOR]

To address the issues of low efficiency and high damage rates during apple harvesting and packing, a parameter optimization experiment was conducted on a low-damage packing device for an apple harvesting platform based on Adams 2019 software. The aim was to reduce the mechanical damage to apples during the packing process. Firstly, kinematics and energetics analyses of the apple packing process were performed, and a mathematical model for damage energy was established to identify the main factors and their ranges that influence the mechanical damage to apples. Secondly, using the fruit damage rate and packing efficiency as the evaluation criteria, a second-order orthogonal rotating regression experiment was conducted with the inclination angle of the fruit conveying tube, the inner wall radius of the fruit conveying tube, and the length of the fruit conveying tube as the experimental factors. Regression mathematical models were established to assess the relationship between the evaluation criteria and the experimental factors. Finally, the impact of each experimental factor on the evaluation criteria was analyzed to determine the optimal structural parameters for the low-damage packing device of the apple harvesting platform, and validation experiments were conducted. The results showed that when the inclination angle of the fruit conveying tube was 47°, the inner wall radius of the fruit conveying tube was 84 mm and the length of the fruit conveying tube was 0.12 m, the average fruit damage rate was minimized at 7.2%, and the average packing efficiency was maximized at 1925 kg/h. These results meet the requirements for apple harvesting operations, and the research findings can serve as a reference for the structural design and packing operation parameter optimization of apple harvesting platforms. [ABSTRACT FROM AUTHOR]

The detection algorithm of the apple-picking robot contains a complex network structure and huge parameter volume, which seriously limits the inference speed. To enable automatic apple picking in complex unstructured environments based on embedded platforms, we propose a lightweight YOLOv5-CS model for apple detection based on YOLOv5n. Firstly, we introduced the lightweight C3-light module to replace C3 to enhance the extraction of spatial features and boots the running speed. Then, we incorporated SimAM, a parameter-free attention module, into the neck layer to improve the model's accuracy. The results showed that the size and inference speed of YOLOv5-CS were 6.25 MB and 0.014 s, which were 4 5 and 1.2 times that of the YOLOv5n model, respectively. The number of floating-point operations (FLOPs) were reduced by 15.56%, and the average precision (AP) reached 99.1%. Finally, we conducted extensive experiments, and the results showed that the YOLOv5-CS outperformed mainstream networks in terms of AP, speed, and model size. Thus, our real-time YOLOv5-CS model detects apples in complex orchard environments efficiently and provides technical support for visual recognition systems for intelligent apple-picking devices. [ABSTRACT FROM AUTHOR]

The vision-based fruit recognition and localization system is the basis for the automatic operation of agricultural harvesting robots. Existing detection models are often constrained by high complexity and slow inference speed, which do not meet the real-time requirements of harvesting robots. Here, a method for apple object detection and localization is proposed to address the above problems. First, an improved YOLOX network is designed to detect the target region, with a multi-branch topology in the training phase and a single-branch structure in the inference phase. The spatial pyramid pooling layer (SPP) with serial structure is used to expand the receptive field of the backbone network and ensure a fixed output. Second, the RGB-D camera is used to obtain the aligned depth image and to calculate the depth value of the desired point. Finally, the three-dimensional coordinates of apple-picking points are obtained by combining two-dimensional coordinates in the RGB image and depth value. Experimental results show that the proposed method has high accuracy and real-time performance: F1 is 93%, mean average precision (mAP) is 94.09%, detection speed can reach 167.43 F/s, and the positioning errors in X, Y, and Z directions are less than 7 mm, 7 mm, and 5 mm, respectively. [ABSTRACT FROM AUTHOR]

An apple-picking robot is now the most widely accepted method in the substitution of low-efficiency and high-cost labor-intensive apple harvesting. Although most current research on apple-picking robots works well in the laboratory, most of them are unworkable in an orchard environment due to unsatisfied apple positioning performance. In general, an accurate, fast, and widely used apple positioning method for an apple-picking robot remains lacking. Some positioning methods with detection-based deep learning reached an acceptable performance in some orchards. However, apples occluded by apples, leaves, and branches are ignored in these methods with detection-based deep learning. Therefore, an apple binocular positioning method based on a Mask Region Convolutional Neural Network (Mask R-CNN, an instance segmentation network) was developed to achieve better apple positioning. A binocular camera (Bumblebee XB3) was adapted to capture binocular images of apples. After that, a Mask R-CNN was applied to implement instance segmentation of apple binocular images. Then, template matching with a parallel polar line constraint was applied for the stereo matching of apples. Finally, four feature point pairs of apples from binocular images were selected to calculate disparity and depth. The trained Mask R-CNN reached a detection and segmentation intersection over union (IoU) of 80.11% and 84.39%, respectively. The coefficient of variation (CoV) and positioning accuracy (PA) of binocular positioning were 5.28 mm and 99.49%, respectively. The research developed a new method to fulfill binocular positioning with a segmentation-based neural network. [ABSTRACT FROM AUTHOR]

Copyright of Packaging & Food Machinery is the property of Packaging & Food Machinery Magazine and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

To solve the problem that the robot often collides with the obstacles such as branches around the fruit during picking due to its inability to adapt to the fruit growing environment, this paper proposes an apple-picking robot picking path planning algorithm based on the improved PSO. The main contents of the algorithm are: firstly, the fruit and its surrounding branches are extracted from the 3D point cloud data, and the picking direction of the fruit is calculated; then the point cloud on the surface of the fruit and branches is used to establish the spatial model of obstacles; finally, an improved particle swarm optimization (PSO) algorithm is proposed to plan the obstacle avoidance trajectory of the end-effector in space, which can dynamically adjust the velocity weights according to the trend of the particle fitness value and the position of the particle swarm center of mass. The experimental results show that the improved PSO has faster convergence speed than the standard PSO, and the path planning method proposed in this paper improves the fruit-picking success rate to 85.93% and reduces the picking cycle to 12 s. This algorithm can effectively reduce the collision between the manipulator and branches during apple picking and improve the picking success rate and picking efficiency. [ABSTRACT FROM AUTHOR]

The use of automation technology in agriculture has become particularly important as global agriculture is challenged by labor shortages and efficiency gains. The automated process for harvesting apples, an important agricultural product, relies on efficient and accurate detection and localization technology to ensure the quality and quantity of production. Adverse lighting conditions can significantly reduce the accuracy of fruit detection and localization in automated apple harvesting. Based on deep-learning techniques, this study aims to develop an accurate fruit detection and localization method under adverse light conditions. This paper explores the LE-YOLO model for accurate and robust apple detection and localization. The traditional YOLOv5 network was enhanced by adding an image enhancement module and an attention mechanism. Additionally, the loss function was improved to enhance detection performance. Secondly, the enhanced network was integrated with a binocular camera to achieve precise apple localization even under adverse lighting conditions. This was accomplished by calculating the 3D coordinates of feature points using the binocular localization principle. Finally, detection and localization experiments were conducted on the established dataset of apples under adverse lighting conditions. The experimental results indicate that LE-YOLO achieves higher accuracy in detection and localization compared to other target detection models. This demonstrates that LE-YOLO is more competitive in apple detection and localization under adverse light conditions. Compared to traditional manual and general automated harvesting, our method enables automated work under various adverse light conditions, significantly improving harvesting efficiency, reducing labor costs, and providing a feasible solution for automation in the field of apple harvesting.