Search

Based on the appearance of tomatoes, it is possible to determine whether they are diseased. Detecting diseases early can help the yield losses of tomatoes through timely treatment. However, human visual inspection is expensive in terms of the time and labor required. This paper presents an automatic tomato disease monitoring system using modular and extendable mobile robot we developed in a greenhouse. Our system automatically monitors whether tomatoes are diseased and conveys the specific locations of diseased tomatoes to users based on the location information of the image data collected by the robot, such that users can adopt timely treatment. This system consists of two main parts: a modular, extendable mobile robot that we developed and a server that runs a tomato disease detection program. Our robot is designed to be configured and extended according to the actual height of the tomato vines, thus ensuring that the monitoring range covers most tomatoes. It runs autonomously between two rows of tomato plants and collects the image data. In addition to storing the image data of tomatoes, the data server runs a program for detecting diseases. This program contains a two-level disease detection model: a detection network for detecting diseased tomatoes and a validation network for verifying the detection results. The validation network verifies the results of the detection network by classifying the outputs of the detection network, thus reducing the false positive rate of the proposed system. Experimentally, this work focuses on the blossom-end rot of tomatoes. In this paper, YOLOv5, YOLOv7, Faster R-CNN, and RetinaNet are trained and compared on datasets divided by different conditions. YOLOv5l showed the best results on the randomly divided dataset: the mAP@0.5 reached 90.4%, and the recall reached 85.2%. Through the trained YOLOv5l, a dataset was created for training the classification networks: ResNet, MobileNet, and DenseNet. MobileNetv2 achieved the best overall performance with a 96.7% accuracy and a size of 8.8 MB. The final deployment to the system included YOLOv5l and MobileNetv2. When the confidence threshold of YOLOv5l was set to 0.1, the two-level model's false positive and false negative rates were 13.3% and 15.2%, respectively. Compared to using YOLOv5l alone, the false positive rate decreased by 5.7% and the false negative rate increased by only 2.3%. The results of the actual operation of the proposed system reveal that the system can inform the user of the locations of diseased tomatoes with a low rate of false positives and false negatives, and that it is an effective and promotable approach. [ABSTRACT FROM AUTHOR]

In order to meet the needs of intensive mechanized picking in trellised pear orchards, this paper designed a pick-place integrated end-picker based on the analysis of agronomic characteristics of trellised pear gardens and fruit. In order to realize the accurate positioning of pears in picking, based on the kinematic analysis of robot arms and the construction of a private dataset, the YOLOv5s object detection algorithm was used in conjunction with a depth camera to achieve fruit positioning. The hand–eye system calibration was carried out. Aiming at solving the problems of redundancy, inefficiency, and uneven distribution of task volume in the conventional multiple robot arms algorithm, a simulated annealing algorithm was introduced to optimize the picking sequence, and a task allocation method was proposed. On the basis of studying several key parameters affecting the performance of the algorithm, the picking efficiency was greatly optimized. And the effectiveness of the proposed multi-robot collaborative picking method in a trellised pear orchard environment was demonstrated through experiments and simulation verification. The experiments showed that the picking efficiency of the integrated end-picker was increased by about 30%, and the success rate was significantly higher than that of the flexible grippers. The results of this study can be utilized to advance robotic pear-picking research and development. [ABSTRACT FROM AUTHOR]

For high-speed robotic cut-and-catch harvesting, efficient trellis grape recognition and picking point positioning are crucial factors. In this study, a new method for the rapid positioning of picking points based on synchronous inference for multi-grapes was proposed. Firstly, a three-dimensional region of interest for a finite number of grapes was constructed according to the "eye to hand" configuration. Then, a feature-enhanced recognition deep learning model called YOLO v4-SE combined with multi-channel inputs of RGB and depth images was put forward to identify occluded or overlapping grapes and synchronously infer picking points upwards of the prediction boxes of the multi-grapes imaged completely in the three-dimensional region of interest (ROI). Finally, the accuracy of each dimension of the picking points was corrected, and the global continuous picking sequence was planned in the three-dimensional ROI. The recognition experiment in the field showed that YOLO v4-SE has good detection performance in various samples with different interference. The positioning experiment, using a different number of grape bunches from the field, demonstrated that the average recognition success rate is 97% and the average positioning success rate is 93.5%; the average recognition time is 0.0864 s; and the average positioning time is 0.0842 s. The average positioning errors of the x, y, and z directions are 2.598, 2.012, and 1.378 mm, respectively. The average positioning error of the Euclidean distance between the true picking point and the predicted picking point is 7.69 mm. In field synchronous harvesting experiments with different fruiting densities, the average recognition success rate is 97%; the average positioning success rate is 93.606%; and the average picking success rate is 92.78%. The average picking speed is 6.18 s × bunch − 1 , which meets the harvesting requirements for high-speed cut-and-catch harvesting robots. This method is promising for overcoming time-consuming harvesting caused by the problematic positioning of the grape stem. [ABSTRACT FROM AUTHOR]

Based on the appearance of tomatoes, it is possible to determine whether they are diseased. Detecting diseases early can help the yield losses of tomatoes through timely treatment. However, human visual inspection is expensive in terms of the time and labor required. This paper presents an automatic tomato disease monitoring system using modular and extendable mobile robot we developed in a greenhouse. Our system automatically monitors whether tomatoes are diseased and conveys the specific locations of diseased tomatoes to users based on the location information of the image data collected by the robot, such that users can adopt timely treatment. This system consists of two main parts: a modular, extendable mobile robot that we developed and a server that runs a tomato disease detection program. Our robot is designed to be configured and extended according to the actual height of the tomato vines, thus ensuring that the monitoring range covers most tomatoes. It runs autonomously between two rows of tomato plants and collects the image data. In addition to storing the image data of tomatoes, the data server runs a program for detecting diseases. This program contains a two-level disease detection model: a detection network for detecting diseased tomatoes and a validation network for verifying the detection results. The validation network verifies the results of the detection network by classifying the outputs of the detection network, thus reducing the false positive rate of the proposed system. Experimentally, this work focuses on the blossom-end rot of tomatoes. In this paper, YOLOv5, YOLOv7, Faster R-CNN, and RetinaNet are trained and compared on datasets divided by different conditions. YOLOv5l showed the best results on the randomly divided dataset: the mAP@0.5 reached 90.4%, and the recall reached 85.2%. Through the trained YOLOv5l, a dataset was created for training the classification networks: ResNet, MobileNet, and DenseNet. MobileNetv2 achieved the best overall performance with a 96.7% accuracy and a size of 8.8 MB. The final deployment to the system included YOLOv5l and MobileNetv2. When the confidence threshold of YOLOv5l was set to 0.1, the two-level model’s false positive and false negative rates were 13.3% and 15.2%, respectively. Compared to using YOLOv5l alone, the false positive rate decreased by 5.7% and the false negative rate increased by only 2.3%. The results of the actual operation of the proposed system reveal that the system can inform the user of the locations of diseased tomatoes with a low rate of false positives and false negatives, and that it is an effective and promotable approach.

There has been an explosion of innovation in agricultural technologies, but whether the anticipated benefits are fully realised depends partly upon with the institutional structures are supportive. Many types of law will shape what innovations are viable and the scale of the economic returns. Australia does not have a coherent strategy for future rural regulation that will both minimise the public risks and increase the private opportunities from future agricultural innovation. This paper addresses these issues. It considers the diverse legal issues that will affect these opportunities, and it looks particularly at agricultural robotics as an example of the many ways in which legal issues will shape opportunities from innovation. It proposes that an integrated strategy, based on a careful analysis of future issues, would be a significant contributor to Australia's agricultural sector and to its innovating industries. [ABSTRACT FROM AUTHOR]

Autonomous weeding robots need to accurately detect the joint stem of grassland weeds in order to control those weeds in an effective and energy-efficient manner. In this work, keypoints on joint stems and bounding boxes around weeds in grasslands are detected jointly using multi-task learning. We compare a two-stage, heatmap-based architecture to a single-stage, regression-based architecture—both based on the popular YOLOv5 object detector. Our results show that introducing joint-stem detection as a second task boosts the individual weed detection performance in both architectures. Furthermore, the single-stage architecture clearly outperforms its competitors with an OKS of 56.3 in joint-stem detection while also achieving real-time performance of 12.2 FPS on Nvidia Jetson NX, suitable for agricultural robots. Finally, we make the newly created joint-stem ground-truth annotations publicly available for the relevant research community. [ABSTRACT FROM AUTHOR]

An apple-picking robot is now the most widely accepted method in the substitution of low-efficiency and high-cost labor-intensive apple harvesting. Although most current research on apple-picking robots works well in the laboratory, most of them are unworkable in an orchard environment due to unsatisfied apple positioning performance. In general, an accurate, fast, and widely used apple positioning method for an apple-picking robot remains lacking. Some positioning methods with detection-based deep learning reached an acceptable performance in some orchards. However, apples occluded by apples, leaves, and branches are ignored in these methods with detection-based deep learning. Therefore, an apple binocular positioning method based on a Mask Region Convolutional Neural Network (Mask R-CNN, an instance segmentation network) was developed to achieve better apple positioning. A binocular camera (Bumblebee XB3) was adapted to capture binocular images of apples. After that, a Mask R-CNN was applied to implement instance segmentation of apple binocular images. Then, template matching with a parallel polar line constraint was applied for the stereo matching of apples. Finally, four feature point pairs of apples from binocular images were selected to calculate disparity and depth. The trained Mask R-CNN reached a detection and segmentation intersection over union (IoU) of 80.11% and 84.39%, respectively. The coefficient of variation (CoV) and positioning accuracy (PA) of binocular positioning were 5.28 mm and 99.49%, respectively. The research developed a new method to fulfill binocular positioning with a segmentation-based neural network. [ABSTRACT FROM AUTHOR]

Agricultural robotics is a complex, challenging, and exciting research topic nowadays. However, orchard environments present harsh conditions for robotics operability, such as terrain irregularities, illumination, and inaccuracies in GPS signals. To overcome these challenges, reliable landmarks must be extracted from the environment. This study addresses the challenge of accurate, low-cost, and efficient landmark identification in orchards to enable robot row-following. First, deep learning, integrated with depth information, is used for real-time trunk detection and location. The in-house dataset used to train the models includes a total of 2453 manually annotated trunks. The results show that the trunk detection achieves an overall mAP of 81.6%, an inference time of 60 ms, and a location accuracy error of 9 mm at 2.8 m. Secondly, the environmental features obtained in the first step are fed into the DWA. The DWA performs reactive obstacle avoidance while attempting to reach the row-end destination. The final solution considers the limitations of the robot's kinematics and dynamics, enabling it to maintain the row path and avoid obstacles. Simulations and field tests demonstrated that even with a certain initial deviation, the robot could automatically adjust its position and drive through the rows in the real orchard. [ABSTRACT FROM AUTHOR]

The control of large greenhouse installations, especially those with hydroponics crops, is based on the analysis and use of data recorded by many sensors. At the same time, the size of such installations does not allow for their effective terrestrial surveillance, to detect problems promptly. In recent years, there has been an interest in the development of autonomous agbots equipped with agricultural sensors. Several ground-based AGV (automated guided vehicles) and UAV (unmanned aerial vehicles) systems have been developed for use in open-air plots. A key feature of all these innovative systems is spectroscopy, the development of which has been assisted by the surveillance capabilities and speed of modern-day UAVs (drones). However, there is a lag in the use of spectroscopy inside greenhouses since UAVs do not move freely indoors. In this paper, we propose as a solution a three-device (3DS) system. [ABSTRACT FROM AUTHOR]

In the dynamic and unstructured environment where horticultural crops grow, obstacles and interference frequently occur but are rarely addressed, which poses significant challenges for robotic harvesting. This work proposed a tactile-enabled robotic grasping method that combines deep learning, tactile sensing, and soft robots. By integrating fin-ray fingers with embedded tactile sensing arrays and customized perception algorithms, the robot gains the ability to sense and handle branch interference during the harvesting process and thus reduce potential mechanical fruit damage. Through experimental validations, an overall 83.3–87.0% grasping status detection success rate, and a promising interference handling method have been demonstrated. The proposed grasping method can also be extended to broader robotic grasping applications wherever undesirable foreign object intrusion needs to be addressed. [ABSTRACT FROM AUTHOR]

The information that crops offer is turned into profitable decisions only when efficiently managed. Current advances in data management are making Smart Farming grow exponentially as data have become the key element in modern agriculture to help producers with critical decision-making. Valuable advantages appear with objective information acquired through sensors with the aim of maximizing productivity and sustainability. This kind of data-based managed farms rely on data that can increase efficiency by avoiding the misuse of resources and the pollution of the environment. Data-driven agriculture, with the help of robotic solutions incorporating artificial intelligent techniques, sets the grounds for the sustainable agriculture of the future. This paper reviews the current status of advanced farm management systems by revisiting each crucial step, from data acquisition in crop fields to variable rate applications, so that growers can make optimized decisions to save money while protecting the environment and transforming how food will be produced to sustainably match the forthcoming population growth. [ABSTRACT FROM AUTHOR]

Pruning is one of the most important tree fruit production activities, which is highly dependent on human labor. Skilled labor is in short supply, and the increasing cost of labor is becoming a big issue for the tree fruit industry. Meanwhile, worker safety is another issue in the manual pruning. Growers are motivated to seek mechanical or robotic solutions for reducing the amount of hand labor required for pruning. Identifying tree branches/canopies with sensors as well as automated operating pruning activity are the important components in the automated pruning system. This paper reviews the research and development of sensing and automated systems for branch pruning in apple production. Tree training systems, pruning strategies, 3D structure reconstruction of tree branches, and practice mechanisms or robotics are some of the developments that need to be addressed for an effective tree branch pruning system. Our study summarizes the potential opportunities for automatic pruning with machine-friendly modern tree architectures, previous studies on sensor development, and efforts to develop and deploy mechanical/robotic systems for automated branch pruning. We also describe two examples of qualified pruning strategies that could potentially simplify the automated pruning decision and pruning end-effector design. Finally, the limitations of current pruning technologies and other challenges for automated branch pruning are described, and possible solutions are discussed. [ABSTRACT FROM AUTHOR]

The present study focused on the development, optimization, and performance evaluation of a harvesting robot for heavyweight agricultural products. The main objective of developing this system is to improve the harvesting process of the mentioned crops. The pumpkin was selected as a heavyweight target crop for this study. The main components of the robot consist of mobile platforms (the main robot tractor and a parallel robot tractor), a manipulation system and its end-effector, and an integrated control unit. The development procedure was divided into four stages: stage I (designed system using Solidworks), stage II (installation of the developed system on a temporary platform), stage III (developed system on an RT-1 (Yanmar EG453)), and stage IV (developed system on an RT-2 (Yanmar YT5113)). Various indicators related to the performance of the robot were evaluated. The accuracy of 5.8 and 4.78 mm in x and y directions and repeatability of 5.11 mm were observed. The harvesting success rate of 87~92%, and damage rate of 5% resulted in the evaluation of the final version. The average cycle time was 35.1 s, 42.6 s, and 43.2 s for stages II, III, and IV, respectively. The performance evaluations showed that the system's indicators are good enough to harvest big-sized and heavy-weighted crops. Development of the unique and unified system, including a mobile platform, a manipulation system, an end-effector, and an integrated algorithm, completed the targeted harvesting process appropriately. The system can increase the speed and improve the harvesting process because it can work all day long, has a precise robotic manipulation and end-effector, and a programmable controlling system that can work autonomously. [ABSTRACT FROM AUTHOR]

There has been an explosion of innovation in agricultural technologies, but whether the anticipated benefits are fully realised depends partly upon with the institutional structures are supportive. Many types of law will shape what innovations are viable and the scale of the economic returns. Australia does not have a coherent strategy for future rural regulation that will both minimise the public risks and increase the private opportunities from future agricultural innovation. This paper addresses these issues. It considers the diverse legal issues that will affect these opportunities, and it looks particularly at agricultural robotics as an example of the many ways in which legal issues will shape opportunities from innovation. It proposes that an integrated strategy, based on a careful analysis of future issues, would be a significant contributor to Australia’s agricultural sector and to its innovating industries.

The need to intensify agriculture to meet increasing nutritional needs, in combination with the evolution of unmanned autonomous systems has led to the development of a series of "smart" farming technologies that are expected to replace or complement conventional machinery and human labor. This paper proposes a preliminary methodology for the economic analysis of the employment of robotic systems in arable farming. This methodology is based on the basic processes for estimating the use cost for agricultural machinery. However, for the case of robotic systems, no average norms for the majority of the operational parameters are available. Here, we propose a novel estimation process for these parameters in the case of robotic systems. As a case study, the operation of light cultivation has been selected due the technological readiness for this type of operation. [ABSTRACT FROM AUTHOR]

The information that crops offer is turned into profitable decisions only when efficiently managed. Current advances in data management are making Smart Farming grow exponentially as data have become the key element in modern agriculture to help producers with critical decision-making. Valuable advantages appear with objective information acquired through sensors with the aim of maximizing productivity and sustainability. This kind of data-based managed farms rely on data that can increase efficiency by avoiding the misuse of resources and the pollution of the environment. Data-driven agriculture, with the help of robotic solutions incorporating artificial intelligent techniques, sets the grounds for the sustainable agriculture of the future. This paper reviews the current status of advanced farm management systems by revisiting each crucial step, from data acquisition in crop fields to variable rate applications, so that growers can make optimized decisions to save money while protecting the environment and transforming how food will be produced to sustainably match the forthcoming population growth.

Pruning is one of the most important tree fruit production activities, which is highly dependent on human labor. Skilled labor is in short supply, and the increasing cost of labor is becoming a big issue for the tree fruit industry. Meanwhile, worker safety is another issue in the manual pruning. Growers are motivated to seek mechanical or robotic solutions for reducing the amount of hand labor required for pruning. Identifying tree branches/canopies with sensors as well as automated operating pruning activity are the important components in the automated pruning system. This paper reviews the research and development of sensing and automated systems for branch pruning in apple production. Tree training systems, pruning strategies, 3D structure reconstruction of tree branches, and practice mechanisms or robotics are some of the developments that need to be addressed for an effective tree branch pruning system. Our study summarizes the potential opportunities for automatic pruning with machine-friendly modern tree architectures, previous studies on sensor development, and efforts to develop and deploy mechanical/robotic systems for automated branch pruning. We also describe two examples of qualified pruning strategies that could potentially simplify the automated pruning decision and pruning end-effector design. Finally, the limitations of current pruning technologies and other challenges for automated branch pruning are described, and possible solutions are discussed.

This study presents the Segmented Colour Feature Extreme Learning Machine (SCF-ELM). The SCF-ELM is inspired by the Extreme Learning Machine (ELM) which is known for its rapid training and inference times. The ELM is therefore an ideal candidate for an ensemble learning algorithm. The Colour Feature Extreme Learning Machine (CF-ELM) is used in this study due to its additional ability to extract colour image features. The SCF-ELM is an ensemble learner that utilizes feature mapping via k-means clustering, a decision matrix and majority voting. It has been evaluated on a range of challenging agricultural object classification scenarios including weed, livestock and machinery detection. SCF-ELM model performance results were excellent both in terms of detection, 90 to 99% accuracy, and also inference times, around 0.01(s) per image. The SCF-ELM was able to compete or improve upon established algorithms in its class, indicating its potential for remote computing applications in agriculture. [ABSTRACT FROM AUTHOR]

Soft actuator technology and its role in robotic manipulation have been rapidly gaining ground. However, less attention has been given to the potential advantages of its application to the agricultural sector, where soft robotics may be a game changer due to its greater adaptability, lower cost and simplicity of manufacture. This article presents a new design approach for soft grippers based on modules that incorporate the concept of bellows and combine it with the versatility and replicability of a 3D printed structure. In this way, the modules can be freely configured to obtain grippers adaptable to crops of different diameters. Furthermore, the definition of a method to determine the soft grippers features is also presented, with the aim of serving as the basis for a future benchmarking study on soft actuators. The experimental tests carried out demonstrated the feasibility and capability of the end-effectors to manipulate various fruits, ensuring a sufficient contact area for the safe handling of the targets and avoiding damaging the products. [ABSTRACT FROM AUTHOR]

High-throughput phenotyping is playing an increasingly important role in many areas of agriculture. Breeders will use it to obtain values for the traits of interest so that they can estimate genetic value and select promising varieties; growers may be interested in having predictions of yield well in advance of the actual harvest. In most phenotyping applications, image analysis plays an important role, drastically reducing the dependence on manual labor while being non-destructive. An automatic phenotyping system combines a reliable acquisition system, a high-performance segmentation algorithm for detecting fruits in individual images, and a registration algorithm that brings the images (and the corresponding detected plants or plant components) into a coherent spatial reference frame. Recently, significant advances have been made in the fields of robotics, image registration, and especially image segmentation, which each individually have improved the prospect of developing a fully integrated automatic phenotyping system. However, so far no complete phenotyping systems have been reported for routine use in a production environment. This work catalogs the outstanding issues that remain to be resolved by describing a prototype phenotyping system for a production tomato greenhouse, for many reasons a challenging environment. [ABSTRACT FROM AUTHOR]

One of the main challenges for the implementation of artificial intelligence (AI) in agriculture includes the low replicability and the corresponding difficulty in systematic data gathering, as no two fields are exactly alike. Therefore, the comparison of several pilot experiments in different fields, weather conditions and farming techniques enhances the collective knowledge. Thus, this work provides a summary of the most recent research activities in the form of research projects implemented and validated by the authors in several European countries, with the objective of presenting the already achieved results, the current investigations and the still open technical challenges. As an overall conclusion, it can be mentioned that even though in their primary stages in some cases, AI technologies improve decision support at farm level, monitoring conditions and optimizing production to allow farmers to apply the optimal number of inputs for each crop, thereby boosting yields and reducing water use and greenhouse gas emissions. Future extensions of this work will include new concepts based on autonomous and intelligent robots for plant and soil sample retrieval, and effective livestock management. [ABSTRACT FROM AUTHOR]

Investment in technological research is imperative to stimulate the development of sustainable solutions for the agricultural sector. Advances in Internet of Things, sensors and sensor networks, robotics, artificial intelligence, big data, cloud computing, etc. foster the transition towards the Agriculture 4.0 era. This fourth revolution is currently seen as a possible solution for improving agricultural growth, ensuring the future needs of the global population in a fair, resilient and sustainable way. In this context, this article aims at characterising the current Agriculture 4.0 landscape. Emerging trends were compiled using a semi-automated process by analysing relevant scientific publications published in the past ten years. Subsequently, a literature review focusing these trends was conducted, with a particular emphasis on their applications in real environments. From the results of the study, some challenges are discussed, as well as opportunities for future research. Finally, a high-level cloud-based IoT architecture is presented, serving as foundation for designing future smart agricultural systems. It is expected that this work will positively impact the research around Agriculture 4.0 systems, providing a clear characterisation of the concept along with guidelines to assist the actors in a successful transition towards the digitalisation of the sector. [ABSTRACT FROM AUTHOR]

The use of robotic systems in organic farming has taken on a leading role in recent years; the Sureveg CORE Organic Cofund ERA-Net project seeks to evaluate the benefits of strip-cropping to produce organic vegetables. This includes, among other objectives, the development of a robotic tool that facilitates the automation of the fertilisation process, allowing the individual treatment (at the plant level). In organic production, the slower nutrient release of the used fertilisers poses additional difficulties, as a tardy detection of deficiencies can no longer be corrected. To improve the detection, as well as counter the additional labour stemming from the strip-cropping configuration, an integrated robotic tool is proposed to detect individual crop deficiencies and react on a single-crop basis. For the development of this proof-of-concept, one of the main objectives of this work is implementing a robust localisation method within the vegetative environment based on point clouds, through the generation of general point cloud maps (G-PC) and local point cloud maps (L-PC) of a crop row. The plants' geometric characteristics were extracted from the G-PC as a framework in which the robot's positioning is defined. Through the processing of real-time lidar data, the L-PC is then defined and compared to the predefined reference system previously deduced. Both subsystems are integrated with ROS (Robot Operating System), alongside motion planning, and an inverse kinematics CCD (Cyclic Coordinate Descent) solver, among others. Tests were performed using a simulated environment of the crop row developed in Gazebo, followed by actual measurements in a strip-cropping field. During real-time data-acquisition, the localisation error is reduced from 13 mm to 11 mm within the first 120 cm of measurement. The encountered real-time geometric characteristics were found to coincide with those in the G-PC to an extend of 98.6%. [ABSTRACT FROM AUTHOR]