Search

• This study proposed detection and localization methods based on deep learning and active sensing for harvesting robots in real-world environments with occlusion and varying lighting conditions. • To solve the picking problem caused by the peduncle's random tilting, this study proposed a method to calculate the peduncle's inclination angle for controlling the end-effector to make the corresponding rotation. • Experimental results indicated the error between the estimated occlusion ratio and the genuine value is 16% in average, and the active sensing method improved the confidence score in occluded situations by over 50%. And the overall successful picking rate of 300 trials is 90%. • The proposed active methods have a 33% increase in precision and a 43% increase in efficiency compared to constant methods. The predominance of branch and leaf shade in agricultural environments presents a barrier for accurate target recognition. Particularly for picking robots, precise localization of the picking object is essential. For this purpose, this paper proposes detection and localization methods based on deep learning and active sensing for harvesting robots in real-world environments with occlusion and varying lighting conditions. Using a deep learning network, the detection method firstly extracts the peduncle and fruit regions; the fruit region is then used to calculate the occlusion rate and the offset distance of the peduncle relative to the fruit. With such information, the robot arm adjusts the camera's field of view to perform multiple recognitions until the confidence is satisfied. Furthermore, to solve the picking problem caused by the peduncle's random tilting, this paper proposes a method to calculate the peduncle's tilt angle for controlling the end-effector to make the corresponding angle rotation. The robot arm and its end-effector are directed to complete the harvesting with the picking point location and tilt angle. In this study, data collection, detection and picking tests were implemented in the field, the results indicated that the method obtained an average successful picking rate of 90% after 300 trials, the error between the estimated occlusion ratio and the genuine value is 16% in average, and the active sensing method has improved the confidence score in occluded situations by over 50%. The proposed active methods have a 33% increase in precision and a 43% increase in efficiency compared to constant methods. [ABSTRACT FROM AUTHOR]

• 3-degrees of freedom (dof) linear multi-arm robotic fruit harvester models were used to harvest the fruits on high density pear and peach trees. • Pick cycle times of multi-arm robotic fruit harvesters (PCTM's) were low when the arms were arranged in height split configuration compared to length split, height split matrix, and length split matrix configurations. • Harvesting the tree sides separately resulted in lower PCTM's than harvesting both sides of the trees at once with a harvester having the same number of arms while traversing in an orchard row. • Lower pick cycle times (PCT's) that could compete with human PCT's can be achieved using linear multi-arm harvesters. In some cases, it has been shown that fruits on trees with SNAP (Simple, Narrow, Accessible, and Productive) architectures are reachable by robot arms using three linear degrees of freedom; hence, high fruit-picking efficiencies can be achieved with simpler arms. This paper uses digitized fruit position data to compute the fruit pick cycle times (PCT) of robotic fruit harvesters with multiple arms arranged in grid configurations, i.e., operating in disjoint rectangular work cells independently of each other. The effects of the robot joints' maximum linear acceleration and maximum linear velocity on PCTs were studied. As V max increased, the PCT followed a negative exponent power law (diminishing return) for any given A max. Similarly, for a constant V max , the improvement of PCT as A max increased slowed down at higher acceleration values. Also, the PCTs were computed when each arm work cell was designed based on equal fruit load or equal size criteria, using four workspace partitioning schemes (height split, length split, height split matrix, and length split matrix). Equal fruit-load configurations resulted in load balancing and exhibited lower PCTs; the height-split configuration was the best among the four different partitioning schemes, possibly because it compensated better for the fruit distribution's non-uniformity along the trees' height. Finally, the PCTs were computed while harvesting one side of an orchard row or both sides concurrently. Harvesting sides separately resulted in lower PCTs (greater speed) due to non-uniform fruit distributions on the different sides of trees. The insights gained in this paper can inform the design of harvesting robots utilizing arrays of 3-dof linear arms. [ABSTRACT FROM AUTHOR]

Robot harvesting has emerged as an urgent need for apple industry due to a sharp decline in agricultural labor. In the field of harvesting robots, the use of multiple robotic arms to improve operational efficiency and promote industrial applications has gained significant attention. Despite the significant progress made in multi-arm harvesting robots in recent years, their widespread application in orchard production is hindered by insufficient efficiency of operation and the accuracy of fruit positioning. This paper focuses on the precise perception and multi-arm collaborative control issues of harvesting robots, and proposes a multi-arm apple harvesting robot system. Firstly, the paper introduces the hardware and software integration method and kinematic configuration of the robot, and presents its workspace division and asynchronous sequential operation mode. Secondly, the paper proposes a stereo vision fruit recognition and localization algorithm based on multi-task deep learning to enhance the accuracy of apple fruit positioning and a method of combining multiple perspectives to acquire a global fruit map is introduced. Finally, the paper presents a multi-arm task planning method based on the Markov game framework to optimize the target harvesting order of each arm and improve the collaboration efficiency. The effectiveness of the robot and its perception and control methods are verified through multiple field experiments in orchards. The field trials showed that the proposed vision system reduces the median locating error of the robot system by up to 44.43%; the proposed task planning algorithm can reduce the average cycle time by 33.3% compared to the heuristic-based algorithm, and time taken for optimizing task planning ranged from 1.14 s to 1.21 s; and the robot's harvest success rate varied from 71.28% to 80.45%, and the average cycle time ranged from 5.8 s to 6.7 s. • A visual recognition and localization algorithm by multiple stereo cameras. • A point-cloud-processing pipeline by a frustum-based method. • A task planning scheme based on multi-agent reinforcement learning framework. • A highly integrated four-arm apple harvesting robotic system. [ABSTRACT FROM AUTHOR]

• Dynamic modeling of hybrid robots has been presented for agricultural purposes. • An innovative method has been used to divide the system into several open chains. • Dynamic couplings between all sub-structures have been considered. • All the steps for systematic derivation are presented in computational algorithms. • A relatively complicated hybrid robotic system with 15 D.O.F has been simulated. Harvesting of agricultural crops, especially when the ripe products have to be picked one by one, has always been a major concern of farmers and growers. Consequently, the use of robots and the mechanization of harvesting operations have been considered as the effective means of tackling such challenges. In the present research, the use of hybrid robots (which combine open-loop and closed-loop manipulators) has been proposed for this purpose; because these robots help reduce the size of the workforce and thus slash the harvesting costs, and they also boost the speed and accuracy of harvesting operations. However, using this type of robots without paying attention to some important factors (e.g., using motors of optimal power at the actuated joints of a robot and choosing appropriate lengths for robot links) can have a substantial adverse effect on the manufacturing cost, working space and the effective functioning of such robots. So the main focus of this paper is on the kinematic and dynamic analysis of hybrid robotic systems. The main advantage of this research over similar works is the generality by which it describes the mechanical behavior of these robots; so that by using the recursive algorithms presented in this paper, all kinds of robots in the hybrid class can be analyzed. However, due to the complexity in the analysis of such systems, a hybrid robot consisting of three closed loops and one open loop was considered in this paper as a sample case study, so that we could explain, visually and graphically, all the steps involved in deriving its motion equations. The simulation results for this robotic system demonstrate the efficacy of the proposed method in the kinematic and dynamic analysis of hybrid robots, as efficient and functional crop harvesting equipment. [ABSTRACT FROM AUTHOR]

This paper deals with a vegetable production-planning problem in which a farmer has to simultaneously determine the crop rotations and harvest schedules for a number of vegetables in his farmlands. The crop rotation decisions include the determination of crop sequence, cropping times, and cropping size in each farmland. The harvest decisions include the collection times and harvesting size for each planted crop. By taking into account the growing season and technical-ecological constraints, this paper formulates a mixed integer linear programming model to develop vegetable production decisions for organic farmers. Due to the complexity of the model, we proposed a two-phase heuristic approach to solve the model. We applied the proposed model to analyze an organic farm in Taiwan. Computational results show that the proposed heuristic is superior to CPLEX in terms of solution quality and computational times for large scale problems. In addition, some sensitivity analyses were also conducted to evaluate the impact of parameters in the model. [ABSTRACT FROM AUTHOR]

The capability of robotic fruit-harvesting systems to accurately assess the ripeness of fruits is crucial for fulfilling the diverse standards of the market and the preferences of consumers. Existing studies involving fruit ripeness estimation mainly focus on detecting of ripe strawberries as one class or classifying of ripeness into several stages, such as overripe, ripe, and unripe. Current harvesting robots also lack the ability to determine the ripeness of the back of fruit with respect to the robots. This paper proposes a lightweight convolutional neural network (CNN) regression model for ripeness quantification based on the internal image sensing system of the gripper with full-view coverage of the fruit for strawberry-harvesting robots. A gripper internal sensing system was developed using two RGB cameras that could provide full-view fruit coverage for a more accurate estimation of fruit ripeness. Four base CNN networks capable of feature learning were used for feature extraction, followed by the utilization of newly added dense layers that produced a regressed value to represent the strawberry ripeness. However, the base networks were cumbersome and relatively slow due to their complex structures. To simplify this, a new MiniNet with fewer convolutional layers was proposed to reduce the model size and inference time. All models were trained via two loss functions, mean square error and Huber loss. The results showed that the models trained via Huber loss performed better. An Xception model trained on Huber loss showed the best performance with a mean absolute error of 4.0% and an average inference time of 42.5 ms. Of all the models, the new MiniNet was the most lightweight and fastest model while maintaining high performances (an mean absolute error of 4.8% and an inference time of 6.5 ms for Huber loss trained model). The proposed method may be also applicable to other fruit-harvesting systems. • Gripper internal sensing system allows multi-view comprehensive ripeness assessment. • Regression models can estimate ripeness levels effectively, with a minimal MAE of 4.0%. • The newly developed MiniNet performed well with a 4.8% MAE, and is faster and lighter. [ABSTRACT FROM AUTHOR]

• We presented the Deep-ToMaToS, capable of recognizing harvesting information. • We constructed a custom photo-realistic simulator for building a large-scale dataset. • We presented a harvest motion based on the Peck-and-Pull manipulation method. This paper presents the development of an autonomous harvesting robot system for tomato, a representative crop cultivated in the facility horticulture smart farm. The automated harvesting work using a robotic system is very challenging because of the appearance, environmental features, such as the atypical directions of the peduncles or their growing form in a bunch. Also, the robot system should enable the harvesting of the target fruit only without damaging other fruits, stems, and branches. Hence, this paper presents a deep learning network pipeline, Deep-ToMaToS, capable of three-level maturity classification and 6D pose (3D translation + 3D rotation) estimation of the target fruit simultaneously. Due to the difficulties encountered in building a large-scale dataset to train and test the deep learning model for the 6D pose estimation in the real world, we presented an automatic data collection scheme based on a photo-realistic 3D simulator environment. The robotic harvesting system includes a harvesting motion control algorithm based on the result of the 6D pose estimation. The overall process of the motion control phase is described along with the decision way of the appropriate final posture of the harvesting module mounted at the end-effector of the robot manipulator via removal of invalid motions getting out of the valid workspace or redundant motions. We conducted experiments on the 6D pose estimation based on the Deep-ToMaToS and the harvesting motion control in virtual and real smart farm environments. The experimental results showed a 6D pose estimation accuracy of 96 % based on the ADD_S metric, and the proposed harvesting motion control algorithm achieves the harvesting success rate of 84.5 % on average. The experimental results reveal that the harvesting robot system has significant potential to extend to harvesting works for other fruits and crops. [ABSTRACT FROM AUTHOR]

• Systematically summarize the hardware platform technology of different robot vision systems and commercial application. • Analyze the development of target recognition and positioning technology from two aspects: sensors and algorithms. • Discuss the challenges encountered by robots in harvesting operations, and describe the future trends. Fruit harvesting robot operations in unstructured agricultural environments have been facing both accuracy and speed challenges, and vision-based control plays an important role in autonomous harvesting. Visual recognition and harvest positioning technology determine whether the target information can be obtained quickly and accurately and whether fruit picking can be carried out efficiently and orderly. It includes not only the hardware platform technology of the vision system, but also different sensor technologies and the optimization of recognition and positioning algorithms. This paper reviews the application of vision recognition and harvesting localization technologies in fruit harvesting robots, starting with an overview of hardware platform technologies for different vision systems of robots, followed by an analysis of the development of target recognition and positioning technologies in terms of sensor hardware and algorithm design, respectively. In addition, the review discusses the difficulties and challenges encountered by robots in harvesting operations and describes the future trends of vision localization technologies. [ABSTRACT FROM AUTHOR]

• A path planning method is proposed based on 3D CNN and rapidly-exploring random tree. • The method improves both planning efficiency and manipulability of manipulator. • Avoid harvest failures due to large volume by prioritizing fruit recovery paths. The environment of tomato bunches is complex, and the fruit volume is relatively large, so manipulator picking-harvest motion planning should consider not only how to pick, but also how to avoid obstacles after picking tomato bunches and harvesting them from the complex environment, which puts forward strict requirements for picking-harvest motion performance. However, existing sampling-based algorithms still have problems of blind expansion and low efficiency. To solve these problems, a heuristic tomato-bunch harvest manipulator path planning method based on a 3D-CNN (convolution neural network)-based position posture map (PPM) and rapidly-exploring random tree (RRT) is proposed in this paper. This method comprehensively considers the sampling effectiveness of the whole process of picking and harvesting, search speed, path cost and manipulability of the manipulator, combining a sampling-based path planning algorithm and deep learning framework. The method utilizes CNN and FCN (Fully Connected Neural Networks) for generating a heuristic cost map where each voxel has a cost-to-go value toward the target and an optimal target posture value to guide the expansion tree. In addition, the sampling space of obstacle avoidance planning is reduced by a spatial partition method, and the difficulty caused by the increase in the volume of the end-effector clamping fruit is solved by prioritizing the planning of the fruit harvest path and then planning the picking path. On the basis of the above research, a large number of harvest experiments have verified the effectiveness of the proposed algorithm. Research and experimental results show that, based on the PPM-RRT algorithm, the picking-harvest time of a single tomato bunch is 13 s, and the success rate is close to 100%. Compared with state-of-the-art sampling algorithms, the planning time is reduced by more than 70%, the path length is reduced by more than 24%, and the manipulability of the manipulator is increased by 38.9%. The proposed path planning method is effective in complex environments. [ABSTRACT FROM AUTHOR]

• Loss monitoring systems for maize crops was designed. • Novel counting algorithm improved sensor accuracy. • Loss monitoring system has good applicability in different working situations. • Provides technical support for intelligent control of harvester. Maize is one of the world's major food crops, in the process of mechanized maize harvesting, the irrational setting of the harvester's working parameters will cause excessive kernel loss, which will have a bad impact on the quality and quantity of the maize crop yield. Real-time monitoring of loss during harvesting is the basis for dynamic adjustment of working parameters. In order to obtain the amount of kernel loss during the harvesting process and provide real-time feedback to the operator on the current kernel loss data, so that can adjust the working parameters in time to reduce the subsequent yield loss, this study proposed a real-time kernel loss monitoring system applicable to maize crops. Aiming at the limitations of the traditional monitoring system in terms of accuracy, the signal processing circuit of the monitoring sensor was improved and a loss kernel counting algorithm based on signal interval duration was proposed, which corrected the monitoring error of the traditional measurement method in the environment of high-frequency material impact. Finally, the performance of the designed monitoring system was verified under different working environments, and the results show that the monitoring system has good performance and the monitoring accuracy can reach more than 92% under different kernel flow rates and different sensor installation positions. In addition, the monitoring system has high adaptability when it comes to kernels with different moisture contents, and the accuracy of identifying lost kernels in mixtures can reach 95%. Compared with the traditional monitoring system, the system proposed in this study has higher monitoring accuracy. The research results of this paper are of great significance to the intelligent control system of maize combine harvester, and provide technical support for real-time regulation of working parameters based on kernel loss, so as to reduce grain loss and ensure grain yield safety. [ABSTRACT FROM AUTHOR]

Highlights • Developed a strawberry harvesting robot with a cable-driven gripper. • The gripper is robust to the localisation errors introduced by the vision module. • The subsystems have been integrated into a complete system. • The system has been evaluated in the farm and the failure cases have been analysed. • The picking cycle time is 7.49 s with a success rate of 53.57% in the farm setting. Abstract This paper presents the development and evaluation of a robot for harvesting strawberries (Fragaria × ananassa) grown on table-tops in polytunnels. The robot is comprised of a newly-designed gripper mounted on an industrial arm which in turn is mounted on a mobile base along with an RGB-D camera. The novel cable-driven gripper can open fingers to "swallow" a target. Since it is designed to target the fruit not the stem, it just requires the fruit location for picking. Moreover, equipped with internal sensors, the gripper can sense and correct for positional errors, and is robust to the localisation errors introduced by the vision module. Another important feature of the gripper is the internal container that is used to collect berries during picking. Since the manipulator does not need to go back and forth between each berry and a separate punnet, picking time is reduced significantly. The vision system uses colour thresholding combined with screening of the object area and the depth range to select ripe and reachable strawberries, which is fast for processing. These components are integrated into a complete system whose performance is analysed starting with the four main failure cases of the vision system: undetected, duplicate detections, inaccurate localisation and segmentation failure. The integration enables the robot to harvest continuously by moving the platform with a joystick. Field experiments show the average cycle time of continuous single strawberry picking is 7.5 s and 10.6 s when including all procedures. Furthermore, the robot is able to pick isolated strawberries with a close-to-perfect success rate (96.8%). However, in farm settings, the average picking success rate is 53.6%, and 59.0% when including "success with damage", testing on the strawberry cultivar of "FAVORI". The failure cases are analysed and most failures are found when picking strawberries in clusters, in which both the detection algorithm and the gripper struggles to separate the berries. [ABSTRACT FROM AUTHOR]

Highlights • We present a planning model for strategic planting and harvesting of vegetable crops. • The model takes considers market prices, climatic conditions and location. • We present a clustering method to simplify the development of supply chain planning models. • The models consider protected agriculture according to market, farm location and climatic conditions. • We present a case study of Mexican fresh vegetables for the export market. Abstract The availability of new information technologies, as well as changes in agricultural markets and their underlying distribution and logistics channels, present new challenges and opportunities to small and medium size farmers. This is particularly true for farmers in emerging economies that very often grow traditional crops whose profits, as observed by small farmers, seem to continuously shrink. The availability of protected agriculture and information technologies promise to open new opportunities, particularly in the fresh produce supply chain. In this paper, we introduce an integrated supply chain planning tool for fresh vegetables that takes into consideration the characteristics and resources of three specific states in Mexico, to make recommendations in terms of the crops to be planted, the timing of planting and harvesting, and what markets to target such that the farmers' profits are maximized. Also relevant is the selection of the appropriate agriculture technology level (e.g. open field, shade structures, or greenhouse) within each region analyzed. A case study based on fresh produce exporting regions of Mexico is presented. The results of the case study show that the judicious use of protected agriculture technology can render significant additional profits for farmers. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a procedure and the results of an optimal design of the kinematic structure of a manipulator to be used for autonomous cucumber harvesting in greenhouses. The design objective included the time needed to perform a collision-free motion from an initial position to the target position as well as a dexterity measure to allow for motion corrections in the neighborhood of the fruits. The optimisation problem was solved using the DIRECT algorithm implemented in the Tomlab package. A four link PPRR type manipulator was found to be most suitable. For cucumber harvesting four degrees-of-freedom, i.e. three translations and one rotation around the vertical axis, are sufficient. The PPRR manipulator described in this paper meets this requirement. Although computationally expensive, the methodology used in this research was found to be powerful and offered an objective way to evaluate and optimise the kinematic structure of a robot to be used for cucumber harvesting. [Copyright &y& Elsevier]

This paper presents a human-centered approach for an efficient cucumber-harvesting robot system. Specifically, harvest ordering, visual servoing, and end-effector-based manipulation functionalities were integrated to realize efficient and stable harvesting. The proposed approach involved determining the optimal harvest ordering, guiding the end-effector to the cucumber pedicel through visual servoing, and designing an end-effector to effectively harvest long cucumbers. The performance of the system was evaluated through preliminary and field experiments. The results of the preliminary experiments showed that harvest ordering decreased the harvesting time and travel length and increased the battery efficiency. The visual servoing was robust, and pedicels could be rapidly detected at a speed of 16–23 FPS through computer vision technologies. The pedicel could be accurately positioned within the cutting area of the end-effector. Furthermore, the proposed end-effector could effectively cut thin cucumber pedicels (3–6 mm), with a 100.0% success rate. Field experiments were conducted at three sites in Korea: Green Monsters, Sangju smart-valley, and Fresh-farm. The harvest success rate at the three sites ranged from 50.9% to 60.0%, with an overall value of 56.6%. The overall average harvest time of 56.0 s. The positional accuracy of the system was within the optimal range of 0–30 mm. Furthermore, the primary causes of harvest failure were analyzed, and future research directions to improve the performance of harvest robots were discussed. [Display omitted] • The proposed human-centered approach can mimic human harvesting methods. • Optimizing the harvest order helps minimize the travel length and operation time. • The proposed system allows rapid detection and tracking of pedicel. • The proposed end-effector increases the success rate when cutting cucumber pedicels. • Preliminary and field experiments demonstrate the effectiveness of the system. [ABSTRACT FROM AUTHOR]

The estimation of the grapes' maturity in the field using non-destructive techniques is of high interest for the high-valued vinified grapes, particularly towards the development of fully automated agrobots that perform selective harvesting operations. Whereas infrared spectroscopy has been employed using point spectrometers in the laboratory and in the field, imaging spectrometers have mainly been tested in controlled laboratory conditions due to issues with varying illumination. In this paper, the application of the autoencoder framework is proposed, which is employed to transform the raw recorded spectra, regardless of illumination conditions, into standardized reflectance spectra; thus addressing the inherent difficulties which hamper the direct application of hyperspectral imaging in the field. To validate the methodology, the sugar content ( ∘ Brix) of four grape varieties, namely Chardonnay, Malagouzia, Sauvignon-Blanc, and Syrah, is estimated. Two different autoencoder architectures are examined: deep fully-connected (DAE) and deep convolutional autoencoders (DCAE), while the estimation of sugar content takes place using as input both from the encoded (latent) space and from the autoencoders' output, i.e., the transformed standardized spectra. The use of multiple spectral pre-treatments is further examined to enhance the accuracy of prediction. Despite that DAE and DCAE showcase comparable similarity metrics, DCAE statistically outperforms DAE when using both the encoded space and the autoencoders' output, attesting to the suitability of the convolutional autoencoder framework. On the other hand, there is no statistical significant difference when employing multiple input pre-treatments. The accuracy of estimation (mean RMSE 1.83 ∘ Brix, R 2 0.70, RPIQ 2.43) is comparable to other studies that directly work with standardized reflectance spectra in laboratory conditions. • In situ hyperspectral imaging faces the varying illumination problem of the targets. • The autoencoder framework is proposed to address this issue and different architectures are investigated. • The presented solution can sufficiently transform raw spectra into standardized reference spectra. • The proposed framework is applied for grape ripeness estimation with fair results. [ABSTRACT FROM AUTHOR]

• A method for estimating citrus 3D pose via a single RGB image is developed. • An image annotation tool to label the fruit 3D pose in a 2D image is designed. • The fruit navel point is detected and the direction of fruit rotation is predicted. • The demonstrated performance shows that it is helpful for non-destructive harvesting. Automated fruit harvesting is promising research in the development of agricultural modernization. However, the complex and non-structural orchard environment is extremely challenging. In order to meet the needs of different end-effectors and to improve the success rate of automatic fruit harvesting, it is critical to perform fruit pose estimation before picking operations. In this study, a citrus pose estimation method through a single RGB image is introduced. The rotation of the citrus pose is defined as a vector that passes through the center of the fruit, which is perpendicular to the plane where the fruit navel point is located. Simply speaking, a multi-task learning model named FPENet is proposed to simultaneously locate the fruit navel point and predict the fruit rotation vector. And a hyperparameter is introduced in the loss function to achieve the simultaneous convergence of multiple tasks. In addition, this paper designs a 2D image annotation tool and constructs a citrus pose dataset, which contributes to model training and also the algorithm evaluation. In the experiment, we evaluate and analyze each module of the proposed network structure, and verify its performance on a harvesting robot. The experimental results show that the FPENet achieves an 88.92 AP score on fruit navel point detection, and 11.13 ° on the average error of the rotation vector. Over 90% of rotation vectors have an angular error of less than 22.5 °. The harvesting success rate is 79.79%. This study offers a new idea for fruit pose estimation and provides the possibility and foundation for estimating fruit pose with a 2D image input. [ABSTRACT FROM AUTHOR]

• Spatiotemporal characteristics of trans-regional operations were first revealed. • Around 57.1 % of the harvesters completed 80.09 % of the total harvested area. • Flow and reliance of each region on harvesters were analyzed and described. • 6 provinces, 82 cities and 588 counties showed a high reliance on nonlocal harvesters. Trans-regional operation of agricultural machinery is an essential socialized service that can enhance the utilization rate of agricultural machinery and accelerate the progress of grain harvesting. In this paper, we quantitatively analyze the trans-regional operations of the harvesters using massive GNSS (Global Navigation Satellite System) data in China and reveal their spatiotemporal characteristics from the province, city and district perspectives. Trajectory data were collected from 25,763 harvesters during wheat harvesting season between May and June 2021 using the agricultural machinery operation big data system. Using the field road segmentation method for GNSS trajectory and the trans-regional operation judgment method, we calculated the quantity, driving distance, harvesting duration, and harvested area. Following this, we obtained the corresponding figures for each region, including the quantity, harvested area, harvesting duration, and distance of harvesters. To further analyze the flow and reliance on harvesters, we examined these factors at the province, city, and district scales for each respective region.. Our analysis revealed that approximately 57.1% of the harvesters were involved in trans-district operation, covering approximately 80.09 % of the total harvested area by all harvesters. Furthermore, we identified 6 provinces, 82 cities and 588 districts with a significant reliance on nonlocal harvesters. The findings provide valuable insights for the government, agricultural machinery enterprises, and agricultural machinery operators to optimize decision-making and configuration schemes to enhance the overall operational efficiency of harvesters. [ABSTRACT FROM AUTHOR]

• A self-developed grasping force system which enables damage-free grasping monitors grasping force in real time. • A self-developed posture system keeps the end-effector in a safe workspace and ensures the security of picking. • The end-effector is modeled with a human reference and has relatively independent knuckles. • All knuckles are driven by ropes, and the end-effector is adaptable to different sizes and shapes of pears. More and more fruit harvesting robots have been used in fresh fruit harvesting. Because many fruits are vulnerable, achieving damage-free picking is a challenge for most harvesting robots. Controlling the grasping force of the end-effector is the key to achieving damage-free picking. In this paper, a rope-driven adaptive end-effector for damage-free pear-fruit picking is developed with reference to the human hand, including the human grasping method and the robotics model. A self-developed grasping force system is fitted to the inside of the fingers for real-time control of the end-effector grasping force. A self-developed posture system is installed at the finger knuckles to ensure that the posture of the fingers is in a safe working state. The number of fingers of the end-effector is three, with each finger consisting of two knuckles. Each knuckle is driven by a servo through a rope. Finally, the experiments show that the end-effector grasps stably. And it could adapt to different sizes and shapes of pear fruits. The average grasping time is 0.605 s. The grasping success rate of all fruits is 100 %, and the grasping damage rate is 0 %. This end-effector effectively solves the problem of other end-effectors failing to grasp or causing damage to the fruits. The development of the end-effector positively affects picking operations in agriculture. [ABSTRACT FROM AUTHOR]

• A new multi-task prediction algoritom TPMv2 was developed. • It estimates 3D pose of tomato bunch based on 2D- and 3D- Bounding Box and Keypoints prediction. • TPMv2 can predict Keypoints on partially visible plant organ from coarse data. The automatic harvesting of tomatoes has been achieved for many years in the laboratory. The new research topic is harvesting the tomato more flexibly and nondestructively at any tomato bunch pose according to the agronomic demands. Although the tomato pose can be predicted by keypoints detection, the poor data quality of commercial RGBD cameras, occlusion between plant organs, various tomato poses, and unstructured working environments pose some challenges to the tomato bunch pose detection. Therefore, our research proposed an improved version of the Tomato Pose Method (TPM), namely TPMv2, which is a two-stage end-to-end multi-task network. This network provides comprehensive information on the tomato bunch, including the positions and poses of the stem, peduncle, and fruits, by predicting the two-dimensional bounding box (2D BBox), three-dimensional bounding box (3D BBox), two-dimensional key point (2D Kpt), and three-dimensional key point (3D Kpt). Aiming at the problems of occlusion and poor-quality point cloud, this paper specially designs a key point network (KPN) for tomatoes, where a keypoints processing pipeline was innovatively proposed, improving the accuracy of key point positioning and reducing abnormal prediction effectively. TPMv2 makes it possible to detect tomato bunch pose precisely with an economical camera, avoiding dangerous situations caused by abnormal prediction. The precision of 2D BBox and 3D BBox reached 0.9372 and 0.8700, and the Percentage of correct Keypoints (PCK) of 2D Kpt and 3D Kpt reached 0.8882 and 0.7836. About 78.36 % of 3D Kpts' positioning errors are less than 20 mm, sufficient to describe a correct pose trend based on the 3D Kpt, benefiting the manipulator to plan a more reasonable trajectory for non-destructive harvesting. [ABSTRACT FROM AUTHOR]

• An array grain loss sensor for corn harvester was designed and built. • The software corresponding to the hardware was developed. • An experimental platform was built and experiments was conducted to calibration the sensor. • Experiments was conducted to verify the accuracy of the sensor. It is wildly known that the grain direct harvest can improve the corn harvest efficiency effectively. However, in the process of grain direct harvest, incomplete threshing and cleaning loss will lead to the loss of corn grain. Among them, the grain loss caused by the cleaning process can be controlled and reduced by adjusting the opening extent of the cleaning sieve, fan speed and other parameters. In traditional operation process, the operator had to adjust the cleaning mechanism based on their experience, which directly leads to low efficiency as well as high rate of grain loss. The grain loss rate sensor proposed in this paper provides the feedback parameters to control the above actuators, which makes the intelligent cleaning of the harvester possible. By comparing the existing research results, the PVDF piezoelectric film was used as the sensitive material. A sensor test platform and a real machine test platform were built and the corresponding software system of the sensor was designed and developed. The software system includes data acquisition module, filtering algorithm, wave shaped module, counting module, data processing module and wireless transmission module. Moreover, an algorithm was proposed to calibrate the error caused by misidentification of the grains and residues. After the testing and debugging, the final identification error of the laboratory tests was decreased from 12% to 3% and the error of the real machine test was under 6%, which is acceptable. The research results of this paper provided the basis for the development of intelligent control system of grain harvesting machinery, and it is of great significance to reduce the cleaning loss rate of harvesting machinery. [ABSTRACT FROM AUTHOR]

• Localization with the growth direction may enhance the plucking accuracy. • Planning motion sequence improves the efficiency of harvesting tea. • Field experiments validate the effectiveness of the proposed harvesting system. The critical issue of the robotic harvesting high-quality tea is to realize tea shoot detection, plucking point localization, and motion planning. In addition, the accuracy and efficiency of the robotic plucking of high-quality tea in the field are essential. Therefore, a robotic harvesting system is proposed in this paper by combining deep learning, point cloud processing, and spatial path planning. First, the deep learning method and the compressed YOLOv3 network are used to quickly and accurately identify tea shoots. Second, an efficient point cloud processing-based 3D localization algorithm for high-quality tea plucking points was proposed. The genetic algorithm is then used to shorten the end-effector's motion path by optimizing the plucking sequences. Eventually, a harvester robot with a parallel manipulator was developed to conduct field plucking experiments and evaluate the effectiveness of the proposed harvesting system. All experimental results demonstrate that the success rates of detection, localization, and motion plucking are 85.16 %, 78.90 %, and 80.23 %, respectively. Furthermore, the overall process harvesting success rate is 53.91 %, and the average plucking time for a single shoot is 2.233 s. Therefore, the proposed harvesting approach can provide technical support for the precise and rapid harvesting of high-quality tea. [ABSTRACT FROM AUTHOR]

• The realization of commercial large-scale application of agricultural mobile platform is inseparable from its autonomous mobile capability. • The autonomous mobility is affected by the mobile structure and the navigation system. • The application of in facility agriculture is relatively mature, and the application in field is a challenging. • It is more mature for commercial application in agricultural harvesting tasks. • Nonstructural effects caused by plant shading and changes in topography and water flow have greater significance. With the gradual disappearance of the demographic dividend and the labor shortage brought by the aging population, it is necessary and urgent to achieve a high degree of automation in agricultural scenarios. And with the development of science and technology, it has made certain progress. The autonomous mobility of platform plays a very important role in this progress. It expands the point-to-point agricultural activities of a single plant into the entire agriculture scene. The autonomous mobility of platforms is driven by the considerable multiple scenarios with different requirements and the need for an adaptable and reliable system. It contains mobile structure and motion control, which complement each other. This paper gives a detailed introduction and summarizes the state-of-the-art agricultural mobile platforms, sensor-based autonomous positioning and navigation methods, as well as their practical applications in different specific agricultural scenarios, especially harvesting task. It accounts for the absolute amount of labor in the overall agricultural task. Unlike, traditional industrial and urban, the agricultural scenarios have more unstructured complex environment. High autonomous mobility of the agricultural mobile platform, which requires the installation of more complex and comprehensive sensors sensitively perceive scene information and more intelligent positioning and navigation strategy, enable the mobile platform to securely, flexible perform agricultural tasks. Therefore, special attention is paid to the navigation strategy and mobile cruise strategy based on various sensor, to improve the autonomy of agricultural mobile platforms. It's potential challenges and future are prospected based on the development trend of agricultural mobile platforms. [ABSTRACT FROM AUTHOR]

• A novel ex-situ harvesting mode in soil was proposed for white asparagus harvesting. • The novel pull-cutting end-effector with rigid-flexible coupling was designed. • The theoretical soil resistances of end-effector in key subprocess were analyzed. • The two-way coupled MBD-DEM analysis was utilized to the harvesting process. • The harvesting was validated with success rate 90% and single cycle time 3.2s. The existing selective harvest of white asparagus originally imitates manual operation, belongs to in-situ harvesting mode, and improves harvesting speed by reverse motion compensation resulting in complicated mechanical and control. Asparagus spear grows underground with limited harvesting space, brittle and easily damaged. The design of robotic end-effector for efficient selective harvest is still a challenge. Inspired by 'fishing with nets in moving boat', this paper firstly proposed a novel ex-situ harvesting mode with sequential subprocess of move-penetrating, pull-cutting, lifting-out, and eject-throwing and the rigid-flexible coupling pull-cutting end-effector (PCEE) was designed. Secondly, the model of PCEE-soil-spear was established, and the two-way coupled MBD-DEM (Multi-Body Dynamics and Discrete Element Method) analysis was utilized to the harvesting process to clarify the interactive forces and element motions. Finally, the harvesting test bench with designed PCEE was built and tested harvesting performance. The experimental results showed that the maximum soil resistances in the moving and penetrating directions during the move-penetrating subprocess and in the moving direction during the pull-cutting subprocess were 445 N, 530 N and 624 N, and the relative errors between the test and simulation were 5.3 %, 11.7 % and 15.2 %, respectively, which validated the MBD-DEM coupling model. The harvest success rate of proposed PCEE was 90 % and the average single cycle time was 3.2 s, which demonstrated its effectiveness and feasibility for the selective harvesting robots of white asparagus. [ABSTRACT FROM AUTHOR]

• Rice feeding state prediction by threshing cylinder unbalanced vibration time-frequency signal. • Harmonic response of the threshing drum under undergoing unbalanced vibration was analyzed. • Spectral kurtosis based on short-time Fourier transform was calculated in field threshing state. • Effect of rice stalk winding vibration was analyzed using signal envelope and spectral kurtosis. When the cylinder of rice combine harvester is threshing, the interaction between the fed rice stalk and the threshing teeth makes the cylinder subject to a large vibration shock. Undergoing the influence of this shock, the vibration response of threshing cylinder is non-stationary. In view of the above problems, this paper aims to accurately characterize the harvesting state of the threshing cylinder and extract the impact components caused by the winding of rice stalks. First, the constrained modes and main response harmonics of the threshing cylinder during operation are solved by modal harmonic response analysis. The time-frequency domain analysis was carried out on the threshing cylinder vibration signal undergoing field working conditions. Combined with spectral envelope and spectral kurtosis analysis methods, the main shock frequencies during the threshing process of the cylinder are analyzed. The results show that the main response frequency of the threshing cylinder under no-load state is the response frequency of 14 Hz, but the main frequency component of the threshing cylinder during the harvesting process is 80 Hz. Comparing the results of harmonic response analysis, it can be concluded that the winding of rice stalks will affect the vibration response of the threshing cylinder. Rice threshing state prediction with harmonic response provides a new method for the follow-up threshing cylinder condition monitoring and fault diagnosis. [ABSTRACT FROM AUTHOR]

• HRC in agriculture focus on safety, ergonomics, awareness, and productivity. • Most of the HRC work in agriculture has been devoted to harvesting and spraying tasks. • A roadmap identifying specific R&D directions required to improve performance and to implement HRC in agriculture in practice. • Developing HRC systems in which the human is retained in the loop can simplify the overall system and reduce its costs. Despite intensive R&D on agricultural robotics in the past decade, implementation of agricultural robotics remains limited due to the complex unstructured agricultural environment that confines performance. A possible way to increase performance is by human-robot collaboration (HRC) approaches for agricultural tasks and applications. The first contribution of this study comprises a quantitative review of the literature on HRC systems, detailing 27 distinct projects performed over the past 21 years. The review revealed that the important and innovative topics being investigated in HRC in agriculture focus on safety, ergonomics, awareness, and productivity and that most of the HRC work in agriculture has been devoted to harvesting and spraying tasks. The second contribution of this study constitutes a roadmap identifying specific R&D directions required to improve performance and implement HRC in agriculture in practice. Specifically, we define aspects of HRC systems for efficient collaboration and functionality, and we also delineate performance measures and experimentation procedures. The paper concludes with constraints, future challenges, and prospects for HRC in the context of smart agriculture. [ABSTRACT FROM AUTHOR]

This paper presents a novel pose estimation algorithm for stem position detection in Japanese green pepper automatic harvesting. When the available visual cues do not provide sufficient information to the harvesting robot, information about the pose of the fruit in space is necessary for accurate stem position detection. In the proposed method the orientation of a fruit in space is obtained by fitting a model to surface points of the fruit. These surface points are acquired using a Lidar type laser range finder, and the point matching is performed using a coherent point drift algorithm with two model transformation methods, rigid and affine. The performance of the proposed method was evaluated both under laboratory conditions and in a greenhouse. In the laboratory test, the mean total error for the affine transformation was less than 25 mm in 42 of 49 positions, less than 20 mm in 28 of 49 positions and less than 15 mm in 19 of 49 positions. For the rigid transformation, the same error was less than 25 mm in 39 of 49 positions, less than 20 mm in 31 of 49 positions and less than 15 mm in 11 of 49 positions. The total error of the affine transformation was found to be proportional to the inclination angle, as the mean error was 11 mm, 15 mm, and 23 mm for inclination angles of 15, 30 and 45 degrees, respectively. No relationship was found between the mean total error and the inclination angle for the rigid transformation, as the calculated mean total error was 20 mm, 18 mm, and 20 mm for inclination angles of 15, 30 and 45 degrees, respectively . In the greenhouse test, the stem was calculated to be within the cutting range for 81 of 107 instances for affine transformation and for 66 of 107 for rigid transformation. These results suggest that the proposed method is suitable for stem position detection in the automatic harvesting of green pepper, and could be adjusted for use with other fruits and vegetables. [ABSTRACT FROM AUTHOR]

• The mechanical properties of jujube branches are calculated from the micro end stress. • Through the difference analysis, the relative error between the experimental and model analysis results is analyzed. • The resonance frequency is obtained by frequency sweep test. • The best parameters are obtained through interactive experiments. To explore the vibration energy transmission characteristics of mechanical harvesting of jujube and improve the effectiveness of vibration harvesting, the vibration response characteristics of small and densely planted Jun jujube branches were experimentally studied and evaluated. According to the agronomic requirements of mechanical harvesting of jujube, first, this paper analyzes the vibration harvesting mechanism of jujube, establishes the transverse bending vibration mechanical model of jujube branches, and obtains the sixth-order natural frequency of jujube branches through vibration frequency sweep experiments. Based on the slope scoring principle and response surface method, the impact of different excitation parameters on the jujube branch vibration effect evaluation index P z was characterized through field experiments, and an optimized vibration effect score was obtained. The results showed that the low-order natural frequencies of small and densely planted jujube branches were mainly concentrated in the range of 6 ∼ 22 Hz, and the vibration response was the most intense at 17.5, 18.0 and 22.5 Hz, which was more conducive to vibration harvesting. By optimizing the relationship between the excitation parameters and evaluation index P z , a better jujube branch vibration effect score of P z = 85.63 is obtained, which provides a reference for the design and development of high-efficiency jujube vibration harvesting devices. [ABSTRACT FROM AUTHOR]

[Display omitted] • Proposed a harvesting motion planning algorithm with a fast and high success rate. • Verified the algorithm by 2D and 3D simulations, in the lab and field environments. • Saved 40–60% of the path planning time by the algorithm. • Increased the path planning success rate to 95% in unstructured environment. Aiming at the problems of low success rate and long planning time of harvesting robots in unstructured and complex environments, a harvesting motion planning algorithm based on pre-harvesting guide points (Pre-harvesting Guide Informed—RRT*, PGI-RRT*) is proposed. This algorithm is on the basis of progressively optimal informed RRT * algorithm. The third point between the starting point and the target point is selected as the pre-harvesting guide point. The four random trees are generated between the initial point, the target point and the guide point at the same time to quickly obtain the initial path. In the PGI-RRT* algorithm, P probability sampling is used to replace random sampling to improve the blindness of sampling. New nodes are generated by the dynamic step size. To do so can improve the speed and flexibility of the PGI-RRT* algorithm to explore the unknown space, and improve the convergence speed of the optimal path. 2D simulation experiments show that, compared with Informed-RRT*, the PGI-RRT* algorithm can reduce the time to find the initial path by about 75–86 %, and improve the success rate by 18–32 %. 3D simulation experiments and experiments in laboratory and field environments show that the PGI-RRT* algorithm proposed in this paper can significantly reduce the planning time and improve the planning success rate. Compared to the 9 algorithms added to the pre-harvesting guide points for double path planning in the OMPL, the PGI-RRT* algorithm can reduce the path planning time by 40–60 %, and improve the success rate of path planning to 95 %, and shorten the execution time of the manipulator by about 8 s. [ABSTRACT FROM AUTHOR]

Coffee plants can present structural problems during semi-mechanized and mechanized harvesting such as excessive defoliation and breaking branches. For this reason, studies of mechanical response of a coffee plant can help the development of more optimized machines. The main objective of this study is the modeling of a coffee plant for its mechanical behavior evaluation by using the Finite Element Analysis. The presented paper includes modeling, numerical simulations and experimental tests from a Coffea arabica L. plants. Firstly, it was necessary to create a coffee tree geometry based on pieces of a real tree using 3D scanning process. The coffee tree geometry together with experimental data provided materials properties of the wood plant which are used for displacements prediction via Finite Element Analysis. In order to validate the methodology, simulated results were compared to a real plant behavior under static load. Results presented consistent values from the three-dimensional modeling of a coffee plant which demonstrated the potentiality for new applications. [ABSTRACT FROM AUTHOR]

The forest industry is a capital intensive business and therefore high efficiency in management and forest operations is a must. Maine has millions of acres of forest stands with tree diameters smaller than 30 cm. The harvest productivity in such stands is low compared to stands with larger diameter trees. A recent harvest productivity study in Maine identified operational constraints for whole tree harvest systems, but efforts to improve active operations would be expensive and time consuming. A common practice to reduce costs and time consumption is to develop simulation models and implement new ideas within them. We developed a production efficiency model that leverages an agent-based modeling approach. The model is based on the interaction of two common forest machines (grapple skidder and stroke delimber) and incorporates empirical cycle time estimates from research in Maine. Three scenarios have been developed to investigate baseline conditions, and two GPS and GIS aided communication improvements. The goal of this paper is to document a new agent based model and to investigate the effect of hardwood component on machine idle time and productivity. Results showed that system productivity was affected by skidding distance, bunch spacing, and removal intensity. An increase in hardwood component led to a decrease in stroke delimber idle time but did not affect grapple skidder idle time. Further, hardwood component did not affect system productivity, and none of the three single-skidder scenarios tested performed any better than another. We validated the model by conducting a sensitivity analysis to confirm previous research results. The modeled waiting times are well within the range of observed values and therefore suggest that this model is accurate and well calibrated. Our conclusions are that when operating under average harvesting conditions there is no loss in productivity due to a change in hardwood component and that a stroke delimber idle time of 40% or more is unavoidable unless the stroke delimber can work independently. Future applications of this model may target specific production forestry conditions. [ABSTRACT FROM AUTHOR]

• An RGB-D camera is mounted on the UAV to collect RGB and depth images. • Combine object detection with semantic segmentation. • A method combining CNN and RGB-D image information. • The 3D localization of branch picking points based on deep learning. • The performance of the whole scheme was evaluated by orchard experiments. Longan is a famous speciality fruit and cultivated medicinal plant that has important edible and medicinal value; how to improve productivity in harvest is an important issue. At present, longan is mainly planted in hilly areas. For complex site conditions and tall trees, the ground harvesting machineries cannot work normally. In this study, aiming at harvesting longan fruit using unmanned aerial vehicles, a method combining an improved YOLOv5s, improved DeepLabv3+ model and depth image information is proposed, which is used for the three-dimensional (3D) positioning of branch picking points in complex natural environments. First, the improved YOLOv5s model is used to quickly detect longan fruit skewers and the main fruit branches from a complex orchard environment. The correct main fruit branch is obtained according to its relative position relationship and is extracted as the input to the semantic segmentation model. Second, using the improved DeepLabv3+ model, the image extracted in the previous step is semantically segmented to obtain the 2D coordinate information of the main longan fruit branches. Finally, combined with the growth characteristics of a longan fruit string, RGB-D information fusion is carried out on the main fruit branches in 3D space to obtain the central axis and pose information of the main fruit branches, and the 3D coordinates of the picking points are calculated, which provides destination information for a longan harvesting drone. To verify the effectiveness of the proposed method, an experiment for identifying and locating the main fruit branches and picking points was carried out in a longan orchard. The experimental results show that the longan string fruit and main fruit branch detection accuracy is 85.50%, and the main fruit branch semantic segmentation accuracy is 94.52%. The whole algorithm takes 0.58 s in the actual scene and can quickly and accurately locate the picking points. In summary, this paper fully exploits the advantages of the combination of a convolutional neural network and RGB-D image information, further improving the efficiency of longan harvesting drones in accurately positioning picking points in 3D space. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents an approach to count mango fruit from daytime images of individual trees for the purpose of a machine vision based estimation of mango crop yield. Images of mango trees were acquired over a three day period, 3weeks before commercial harvest occurred. The fruit load of each of fifteen trees was manually counted, and these trees were imaged on four sides. Correlation between tree counts and manual image counts was strong (R 2 =0.91 for two sides). A further 555 trees were imaged on one side only. For these images, pixels were segmented into fruit and background pixels using colour segmentation in the RGB and YCbCr colour ranges and a texture segmentation based on adjacent pixel variability. Resultant blobs were counted to obtain a per image mango count. Across a set of 555 images (with mean±standard deviation of fruit per tree of 32.3±14.3), a linear regression, (y =0.582x −0.20, R 2 =0.74, bias adjusted root mean square error of prediction=7.7) was achieved on the machine vision count relative to the image count. The algorithm decreased in effectiveness as the number of fruit on the tree increased, and when imaging conditions involved direct sunlight. Approaches to reduce the impact of fruit load and lighting conditions are discussed. [Copyright &y& Elsevier]

Abstract: In this paper, we report the results of an exploration study of the feasibility of using computer vision to conduct accurate color rating of sweet cherry in outdoor orchard environments. Pre-harvest color rating of cherry is important to growers in determining the optimal harvest time. Currently, the in-field rating relies heavily on the manual comparison between the color of cherry fruits and standard color charts, which is both labor intensive and subjective. It is not uncommon to have one or two grades of deviation. A computer vision-based color rating system was developed in an attempt to provide an automatic and objective way to achieve more consistent and accurate color ratings. This system successfully used a camera flash to reduce the effects of two major obstacles in outdoor color rating: (1) inconsistent ambient light; and (2) glaring reflections on cherry skin. To mimic the manual color rating practice that is widely accepted by sweet cherry growers today, a task-oriented image processing algorithm was developed to remove the glaring reflections and to classify the color of cherries into seven levels. Field tests showed that the overall accuracy of the rating exceeded 85% based on 660 samples from three field tests under natural, outdoor lighting conditions. The tests validated the feasibility of using a computer vision system to achieve accurate and objective color ratings of sweet cherry under outdoor natural light conditions for actual in-orchard use. [Copyright &y& Elsevier]

Abstract: Vibrations are a well known potential cause of health diseases and therefore constitute a main concern for the safety of workers in a large number of activities. In order to prevent health hazards, national and international institutions have issued laws and directives which establish recommended limits to the workers’ exposure to vibrations during operations. Consequently, if the amount of adsorbed vibration exceeds the allowable daily limits the worker has to stop his job. Recent wireless sensing and communicating technologies can effectively be employed for such purpose, allowing to develop monitoring customized devices at affordable cost which could be easily employed during the workers’ activity. In such context, the present research proposes an innovative system aimed at estimating the hand–arm exposure to vibration according to the Standard EN ISO 5349-1:2004. In particular the proposed system is based on Micro Electro-Mechanical Systems (MEMS) technology and involves the design of a compact wearable unit to be attached to the waist of the operator and a fixed station for data storage and analysis. The paper reports the design of such a system and an experimental test performed on hand held shakers commonly employed for olive harvesting. The results highlight the opportunities offered by these emerging technologies in developing intelligent devices to be included in the worker’s equipment which might significantly modify the current approach to safety management in the next future. [Copyright &y& Elsevier]

Abstract: The special cultivation features of white asparagus pose new challenges to the research and development of the row guidance control. This paper presents a row tracking system of a two-layer structure for an autonomous robot developed to harvest white asparagus. At the low level, two independent speed control loops are suggested to ensure the actual revolutions of the drive motors to follow their demanded values. A cascade control structure, consisting of an inner orientation error controller and an outer lateral offset controller, is proposed for the high level to drive the robot to track the desired trajectory. The most important advantage of the cascade structure is that the inner loop directly regulates the most significant error (the orientation error), which minimizes the impact of the external disturbances upon the outer loop. Moreover, the cascade scheme allows the orientation angle to be limited within a given range. That ensures a collision-free tracking. The controllers are designed based on the traditional proportional-integral-derivative (PID) algorithms. The control parameters are selected by using root locus analysis, which guarantees the system stability. The effectiveness of the proposed control regime is evaluated by numerical simulations and validated by experiments. The experimental results demonstrate that with the suggested row guidance strategy a satisfactory tracking accuracy of ±0.5cm is achieved. [Copyright &y& Elsevier]

Abstract: Agricultural field operations, such as seeding, harvesting, spraying, and fertilizing, are typically carried out by multi-machinery systems comprising one or more self-propelled or tractor-pulled units, and one or more service – mainly transport – units. The operation of a service unit must be carefully planned because its execution efficiency can significantly affect the productivity of the whole system. In this paper, an algorithmic approach for the generation of optimal in-field paths to be followed by service units, was developed and demonstrated. Both stationary and on-the-go unloading operations are supported. The approach was based on an abstraction of a field as a two-dimensional grid, with each grid cell representing obstacle, free, initial, or goal region. By defining also the action spaces of the grid states (service unit moves one cell up, down, left, or right), the arcs and nodes of a discrete transition graph are created and by using a graph search algorithm the optimal path is generated. A number of simulated experiments were performed in order to demonstrate the paths that result from invoking the above-mentioned planning method. The algorithmic path generation can provide the basis for a navigation tool dedicated to service units, which can increase the overall field efficiency of the operations executed by cooperating machines and reduce the adverse effects of the heavy machine traffic on the soil. The low computational requirements of the proposed method make it feasible as an implementation of the optimal planning for large scale operations executed by cooperative machines. [ABSTRACT FROM AUTHOR]

Abstract: Maturity index (MI) is a key determinant of pea softness and ultimately retail value. Pea seed development goes through the optimal market stage for human consumption about a week before harvest. MI increases rapidly during the last 3–4 days prior to the optimal harvest which is when there is a need for better forecasting capability. Extensive field sampling is currently used to track MI in each of the individual paddocks, though it has limited ability to predict MI more than a day ahead. We developed an Artificial Neural Network (ANN) model that complements field sampling by forecasting the MI trend several days ahead. It was built using historical harvest information along with weather and climate forecasts. We implement and evaluate the ANN in a large pea growing region in Tasmania, Australia, and this paper highlights key results. The ANN produced an average error of 31.8MI units when forecasting MI at harvest with a 7-day lead time versus the current manual method which produced an average error of 36.6MI units for a lead time of 2 days. This means the model provides the ability to not only harvest peas closer to their ideal MI but also plan harvesting and transport logistics with a much greater lead time. [Copyright &y& Elsevier]

Abstract: In recent years many applications have been proposed for Wireless Sensor Networks (WSN). One of these is precision agriculture, where WSN can play an important part in the handling and management of water resources for irrigation, in understanding the changes in the crops to assess the optimum point for harvesting, in estimating fertilizer requirements and to predict crop performance more accurately. This paper describes our experience during the introduction and deployment of an experimental sensor network at an ecological horticultural enterprise in the semiarid region of Murcia. It presents the topology of the deployed network using four types of nodes (Soil Mote, Environmental Mote, Water Mote and Gateway Mote), some of them connected to different sensors distributed in the field. These sensors can measure various soil characteristics such us temperature, volumetric moisture content and salinity. For each node, the overall architecture, hardware and software components are described. The system also includes a real-time monitoring application operating from a computer placed in the central offices of the farm. The testing of the system was done in two phases: the first in the laboratory, to validate the functional requirements of the developed devices, the networking solution and the mote''s power management; the second on the farm, to asses the functional performance of the devices, such as range, robustness and flexibility. The system was successfully implemented on a crop of ecological cabbage (Brassica oleracea). The result was a low cost, highly reliable and simple infrastructure for the collection of agronomical data over a distributed area in horticultural environments. [Copyright &y& Elsevier]

• A predictive robot scheduler module is integrated in the simulator based on a proposed model. • Dynamic predictive scheduling was modeled for teams of robotic harvest aids during manual harvesting. • Near-optimal scheduling was implemented to provide efficiency upper-bounds for any predictive scheduling incorporating uncertainty in the predictive requests. • Scheduling performance was studied as a function of the number of robots – for a given crew size – with robot speed as a parameter. • The effect of the earliness of the availability of the predictions on the predictive scheduling performance was studied. Manual harvesting of fresh-market crops like strawberries is very labor-intensive. Apart from picking fruits, pickers spend significant amounts of time carrying full trays to a collection station at the edge of the field. Small teams of harvest-aid robots that help large picking crews by transporting empty and full trays can increase harvest efficiency by reducing pickers' non-productive walking times. However, robot sharing among the crew may introduce non-productive waiting delays between the time a tray becomes full and when a robot arrives to collect it. Reactive robot scheduling cannot eliminate mean waiting times because pickers must wait for a robot to travel the distance from the collection station to them. Predictive scheduling is better suited to this task, because if the time and location that a pickers' tray will fill are known to the scheduler in advance, a robot can start moving toward that location before the tray becomes full; hence, waiting times due to robot travel can be reduced or eliminated. In this paper, dynamic predictive scheduling was modeled for teams of robots carrying trays during manual harvesting. The times and locations of the tray-transport requests were assumed to be known exactly (deterministic predictions). Near-optimal scheduling was implemented to provide efficiency upper-bounds for any predictive scheduling algorithms that incorporate uncertainty in the predictive requests. Robot-aided harvesting was simulated using manual-harvest data collected from a commercial picking crew. Scheduling performance was studied as a function of the number of robots – for a given crew size – with robot speed as a parameter. Additionally, the effect of the earliness of the availability of the predictions on performance was studied. Experimental results showed that both reactive and predictive scheduling did not improve the mean non-productive time significantly relative to manual harvesting, when only four robots were used. Actually, deploying fewer than four robots led to worse non-productive time. However, introducing five to eight robots decreased mean non-productive time drastically, and when ten or more robots were used, non-productive time was reduced by 64.6% (reactive scheduling) and up to 93.7% (predictive scheduling) with respect to all-manual non-productive time. The efficiency increases were 15% and 24%, respectively. It was also verified that reactive dispatching always performed worse than deterministic predictive scheduling. Also, when the robot-to-picker ratio was larger than approximately 1:3, the waiting time and efficiency plateaued, i.e., did not improve further, regardless of how early the prediction was available to the scheduler. The reason is that the mean waiting time is lower bounded by the sum of mean travel time and tray exchange time, which are both constant. Although the above results represent upper-bounds for performance – since predictions were perfect - they indicate that tray-transport robots acting as harvest aids can increase harvesting efficiency significantly when scheduled properly. [ABSTRACT FROM AUTHOR]

• RGB-D cameras are widely used in fruit detection and localization. • RGB-D cameras with different depth measurement principles are conclude. • Researches based on RGB-D sensors for fruit detection and localization are summarized. • Major challenges for the application of RGB-D sensor systems on orchard are discussed. Fruit detection and localization are essential for future agronomic management of fruit crops such as yield prediction, yield mapping and automated harvesting. However, to perform robust and efficient fruit detection and localization in orchard is a challenging task under variable illumination, low-resolutions and heavy occlusion by neighboring fruits, foliage, or branches. Therefore, researches of fruit detection and localization by getting more information of objects are essential. RGB-D (Red, Green, Blue -Depth) cameras are promising sensors and widely used in fruit detection and localization given that they provide depth information and infrared information in addition to RGB information. After presenting a discussion on the advantages and disadvantages of RGB-D cameras with different depth measurement principles and application fields, this paper reviews various types of RGB-D sensor systems and image processing methods used for fruit detection and localization in the field. Finally, major challenges for the successful application of RGB-D camera-based machine vision system, and potential future directions for the research and development in this area are discussed. [ABSTRACT FROM AUTHOR]

• A rice grain impurity sensor in grain-bin of combine harvester was designed. • Structure parameters of grain sampling box and reciprocating motion of choke plate were analyzed. • Image sensing was improved through light source illumination and installation. • The decision tree algorithm was applicated to recognize rice grains and stalks. The cleaning of rice on a combine harvester is a complex process, which leads to differences in impurity ratio of harvested grain, and the impurity ratio is one of the key criteria for the assessment of performance of a combine harvester. Combine operators usually optimize parameter settings only once for harvesting each crop because of time pressure, and therefore differences in site- and temporal-specific conditions are neglected. In this paper, to offer combine operators the opportunity to make better management decision, a machine vision method for grain impurity monitoring of a rice combine harvester in real time was proposed, and the classification of kernel and impurity particles using decision tree algorithm was presented. To obtain images of high quality during harvesting, the structure of the sampling device depending on the working properties in grain bin was designed, the illumination and installation of the light source were optimized, and finally lateral lighting system was constructed. To monitor and recognize grains and impurities, the morphological features of the particles extracted from the images were acquired. The selected 6 features (A1-A6), including area, perimeter, maximal feret diameter, elongation factor, compactness factor and Heywood circularity factor, were fed to the decision tree algorithm for classification. Output of the algorithm, a visualized tree, was used to classify the particles labeled in the binary image. The decision tree provided a classification accuracy of about 76% for the given training data set extracted from the captured images. From the experimental results, it is suggested that the method of monitoring the impurity ratio of harvested grains based on decision tree algorithm using image processing can be recommended as the basis of parameter optimization of combine harvesters. [ABSTRACT FROM AUTHOR]

• Fruit plucking is the result of complex force transmission and large deformation. • Force transfer in peduncle and pedicel can be explained by theory of cantilever beam deformation. • Combined loading plays an extremely important role in all plucking patterns. • For reciprocating twisting, fatigue break also affects its performance. • Fruit with shorter & thinner peduncle & pedicel may be plucked easier. • Integrating with innovation of agronomy may greatly improve the plucking performance. Detachment of fruit from the plants is a basic and essential step either in manual or in robotic harvesting. Usually there are two methods involved in fruit detachment one is fruit cutting and the other is fruit plucking. Compared with fruit cutting, fruit plucking is more simple, low cost, and flexible in robotic harvesting. Different plucking patterns have been adopted in various prototypes for tomato, cherry tomato, apple and strawberry fruit harvesting. Although, it was found that there were differences in the success rate, efficiency and damage of different plucking patterns, moreover in-depth analysis still not carried out. In this paper, combining the classical load-deformation and fatigue break theory with the flexibility of the peduncle and pedicel are presented, theoretical modeling and discussion of different plucking based experimental results were performed. It was found from force transmission, that the load applied on the abscission layer was different from the motive power. Large deformation may happen in the whole peduncle and pedicel and lead to failure, inefficiency or even collision with obstacles in the canopy. Complex combined loading lead to the final break of the abscission layer. For reciprocating twisting pattern, long tomato peduncle and pedicel may take excessive swing times to realize the fatigue break of the abscission layer. In addition, the potential damage to fruit or the whole cluster can be well explained by this theory. The performance of different fruit plucking patterns may be related to the different species and cultivation modes, this research provides a good analytical basis for revealing the complex mechanisms of robotic fruit plucking and making a practical decision. [ABSTRACT FROM AUTHOR]

• The stochastic parameters distributions of a hybrid-automata model were estimated via video-recording of a crew's picking activities. • A calibrated Monte-Carlo strawberry harvesting simulator predicted the crew's non-productive times. • Reactively dispatching 5 robots to serve tray-transport requests from 25 pickers improved harvest efficiency up to 10.2%. • The harvest efficiency plateaued by deploying more than 5 robots, due to needed time for a robot to travel to a picker. Harvest-aid robots that transport empty and full trays during manual harvesting of specialty crops such as strawberries or table grapes can increase harvest efficiency, by reducing pickers' non-productive walking times. In Part I of this work, a modeling framework, and a stochastic simulator were presented for all-manual and robot-aided harvesting. This paper reports Part II of our work, which utilized data gathered in two strawberry fields during harvesting, to estimate the stochastic parameters involved in modeling pickers, and evaluate the prediction accuracy of the simulator for all-manual picking. Then, as a case study, non-productive time and harvest efficiency were estimated for robot-aided harvesting, for various picker-robot ratios and three priority-based reactive dispatching strategies for the robots. The simulator predicted the pickers' non-productive time during all-manual harvesting, with 6.4%, 3%, and 1.2% errors for the morning, afternoon, and "all-day" harvesting shifts, respectively. Statistical testing verified that predicted non-productive times followed the same distributions as the measured non-productive times (5% significance level). Simulations robustness was assessed by using morning data to simulate afternoon harvesting and vice-versa: non-productive times distributions were predicted accurately (10% significance level). Robot-aided simulation results – using the calibrated simulator for a 25-picker crew – showed that all-manual harvest efficiencies of 81.8% and 78.2% for morning and afternoon shifts increased to 92% and 86.5%, respectively, when five robots were deployed. Different scheduling policies did not affect efficiency when more than five robots were used, because there were always enough robots to serve pickers' requests immediately. Also, harvest efficiency plateaued when more than five robots were used, as a consequence of the time needed for a robot to travel to a picker. [ABSTRACT FROM AUTHOR]

• We present the design concept of a modular robot, built of basic modules that can be re-ordered. • An optimization algorithm for optimal robot structure design is presented. • The modularity increases the collected fruits by up to 30% and reduces the harvesting time by 9–30%. • The algorithms were checked on real trees measured in an orchard and converted to 3D models. This paper provides a new concept of a modular, task-based agricultural robot whose construction can be changed using different combinations of basic parts. This way, the robot structure can be optimized to different tasks. This kind of robot combines the advantages of economical task-based robots with the ability to multitask, typically provided by costly universal robots. The modular approach can increase the profitability of agriculture robot manipulators and opens the door for robotic harvesters into agriculture. We present the entire flow of designing this type of modular agricultural manipulator. We first measure and model real trees and then calculate the time it takes the robot to reach every fruit according to the robot structure and location. Lastly, we use our optimization approach to find a robot optimal for pre-defined tasks, such as maximizing the number of fruit harvested or minimizing the picking time. Simulation results, based on actual three-dimensional tree models, show that modular robots can collect 30–40 percent more fruit than a robot with a fixed structure of the same complexity. Furthermore, a modular robot can be adapted to every season for different types of fruit; consequently, it provides much better results both in harvesting time and in amount of harvested fruit. Thus, a modular robot has the potential to increase the utilization compared with a fixed structure robot and helps overcome low-utilization barrier of entrance of robots into agriculture. [ABSTRACT FROM AUTHOR]