Search

Abstract Pollination is critical for crop development, especially those essential for subsistence. This study addresses the pollination challenges faced by Actinidia, a dioecious plant characterized by female and male flowers on separate plants. Despite the high protein content of pollen, the absence of nectar in kiwifruit flowers poses difficulties in attracting pollinators. Consequently, there is a growing interest in using artificial intelligence and robotic solutions to enable pollination even in unfavourable conditions. These robotic solutions must be able to accurately detect flowers and discern their genders for precise pollination operations. Specifically, upon identifying female Actinidia flowers, the robotic system should approach the stigma to release pollen, while male Actinidia flowers should target the anthers to collect pollen. We identified two primary research gaps: (1) the lack of gender-based flower detection methods and (2) the underutilisation of contemporary deep learning models in this domain. To address these gaps, we evaluated the performance of four pretrained models (YOLOv8, YOLOv5, RT-DETR and DETR) in detecting and determining the gender of Actinidia flowers. We outlined a comprehensive methodology and developed a dataset of manually annotated flowers categorized into two classes based on gender. Our evaluation utilised k-fold cross-validation to rigorously test model performance across diverse subsets of the dataset, addressing the limitations of conventional data splitting methods. DETR provided the most balanced overall performance, achieving precision, recall, F1 score and mAP of 89%, 97%, 93% and 94%, respectively, highlighting its robustness in managing complex detection tasks under varying conditions. These findings underscore the potential of deep learning models for effective gender-specific detection of Actinidia flowers, paving the way for advanced robotic pollination systems.

A novel non-destructive analytical method for early diagnosis of two bacterial diseases, Pseudomonas syringae and Xanthomonas euvesicatoria, in tomato plants, using ultraviolet-visible (UV–Vis) transmittance spectroscopy and chemometric models, is developed. Plant-pathogen interactions caused tissue damage that generated non-linear data patterns compared to the control set (healthy samples), which challenges traditional discrimination models, even when employing non-linear discriminant approaches. Alternatively, an authentication task to conduct one-class classification relying on a data-driven version of soft independent modeling of class analogy (DD-SIMCA) is a wise choice due to its quadratic approach, proper to deal with non-linear data. DD-SIMCA detached the target class (control healthy plant leaflet tissues) from all other samples (target class and non-target class of plant leaflet tissues inoculated with two bacteria, even before the manifestation of macroscopic lesions associated with the diseases) by capturing the main similarities within the samples of the target class through the full distance that acts as a classification analytical signal, reaching 100 % sensitivity in the training and validation sets. Multivariate curve resolution - alternating least-squares (MCR-ALS) constrained analysis allowed the description of the bacterial inoculation process on diseased tissues through pure spectral signatures. DD-SIMCA results indicate that non-target class of samples with higher proximity to the acceptance boundary suggested that they were at earlier stages of infection when compared to more distant ones, presenting lower full distance values. These findings reveal that a handheld UV–Vis transmittance spectrometer is sufficiently sensitive to be used in acquiring biological data with suitable chemometric models for early disease diagnosis and prompt intervention.

Stand-level forest tree species perception and identification are needed for monitoring-related operations, being crucial for better biodiversity and inventory management in forested areas. This paper contributes to this knowledge domain by researching tree trunk types multispectral perception at stand-level. YOLOv5 and YOLOv8 - Convolutional Neural Networks specialized at object detection and segmentation - were trained to detect and segment two tree trunk genus (pine and eucalyptus) using datasets collected in a forest region in Portugal. The dataset comprises only two categories, which correspond to the two tree genus. The datasets were manually annotated for object detection and segmentation with RGB and RGB-NIR images, and are publicly available. The “Small” variant of YOLOv8 was the best model at detection and segmentation tasks, achieving an F1 measure above 87% and 62%, respectively. The findings of this study suggest that the use of extended spectra, including Visible and Near Infrared, produces superior results. The trained models can be integrated into forest tractors and robots to monitor forest genus across different spectra. This can assist forest managers in controlling their forest stands.

Automating pruning tasks entails overcoming several challenges, encompassing not only robotic manipulation but also environment perception and detection. To achieve efficient pruning, robotic systems must accurately identify the correct cutting points. A possible method to define these points is to choose the cutting location based on the number of nodes present on the targeted cane. For this purpose, in grapevine pruning, it is required to correctly identify the nodes present on the primary canes of the grapevines. In this paper, a novel method of node detection in grapevines is proposed with four distinct state-of-the-art versions of the YOLO detection model: YOLOv7, YOLOv8, YOLOv9 and YOLOv10. These models were trained on a public dataset with images containing artificial backgrounds and afterwards validated on different cultivars of grapevines from two distinct Portuguese viticulture regions with cluttered backgrounds. This allowed us to evaluate the robustness of the algorithms on the detection of nodes in diverse environments, compare the performance of the YOLO models used, as well as create a publicly available dataset of grapevines obtained in Portuguese vineyards for node detection. Overall, all used models were capable of achieving correct node detection in images of grapevines from the three distinct datasets. Considering the trade-off between accuracy and inference speed, the YOLOv7 model demonstrated to be the most robust in detecting nodes in 2D images of grapevines, achieving F1-Score values between 70% and 86.5% with inference times of around 89 ms for an input size of 1280 × 1280 px. Considering these results, this work contributes with an efficient approach for real-time node detection for further implementation on an autonomous robotic pruning system.

The efficient application of phytochemical products in agriculture is a complex issue that demands optimised sprayers and variable rate technologies, which rely on advanced sensing systems to address challenges such as overdosage and product losses. This work developed a system capable of processing different tree canopy parameters to support precision fruit farming and environmental protection using intelligent spraying methodologies. This system is based on a 2D light detection and ranging (LiDAR) sensor and a Global Navigation Satellite System (GNSS) receiver integrated into a sprayer driven by a tractor. The algorithm detects the canopy boundaries, allowing spray only in the presence of vegetation. The spray volume spared evaluates the system’s performance compared to a Tree Row Volume (TRV) methodology. The results showed a 28% reduction in the overdosage of spraying product. The second step in this work was calculating and adjusting the amount of liquid to apply based on the tree volume. Considering this parameter, the saving obtained had an average value for the right and left rows of 78%. The volume of the trees was also monitored in a georeferenced manner with the creation of a occupation grid map. This map recorded the trajectory of the sprayer and the detected trees according to their volume.

Early diagnosis of plant diseases is needed to promote sustainable plant protection strategies. Applied predictive modeling over hyperspectral spectroscopy (HS) data can be an effective, fast, cost-effective approach for improving plant disease diagnosis. This study aimed to investigate the potential of HS point-of-measurement (POM) data for in-situ, non-destructive diagnosis of tomato bacterial speck caused by Pseudomonas syringae pv. tomato (Pst), and bacterial spot, caused by Xanthomonas euvesicatoria (Xeu), on leaves (cv. cherry). Bacterial artificial infection was performed on tomato plants at the same phenological stage. A sensing system composed by a hyperspectral spectrometer, a transmission optical fiber bundle with a slitted probe and a white light source were used for spectral data acquisition, allowing the assessment of 3478 spectral points. An applied predictive classification model was developed, consisting of a normalizing pre-processing strategy allied with a Linear Discriminant Analysis (LDA) for reducing data dimensionality and a supervised machine learning algorithm (Support Vector Machine – SVM) for the classification task. The predicted model achieved classification accuracies of 100% and 74% for Pst and Xeu test set assessments, respectively, before symptom appearance. Model predictions were coherent with host-pathogen interactions mentioned in the literature (e.g., changes in photosynthetic pigment levels, production of bacterial-specific molecules, and activation of plants’ defense mechanisms). Furthermore, these results were coherent with visual phenotyping inspection and PCR results. The reported outcomes support the application of spectral point measurements acquired in-vivo for plant disease diagnosis, aiming for more precise and eco-friendly phytosanitary approaches.

Early and accurate disease diagnosis is pivotal for effective phytosanitary management strategies in agriculture. Hyperspectral sensing has emerged as a promising tool for early disease detection, yet challenges remain in effectively harnessing its potential. This study compares parametric spectral Vegetation Indices (VIs) and a nonparametric Gaussian Process Classification based on an Automated Spectral Band Analysis Tool (GPC-BAT) for diagnosing plant bacterial diseases using hyperspectral data. The study conducted experiments on tomato plants in controlled conditions and kiwi plants in field settings to assess the performance of VIs and GPC-BAT. In the tomato experiment, the modeling processes were applied to classify the spectral data measured on the healthy class of plants (sprayed with water only) and discriminate them from the data captured on plants inoculated with the two bacterial suspensions (108 CFU mL−1). In the kiwi experiment, the standard modeling results of the spectral data collected on nonsymptomatic plants were compared to the ones obtained using symptomatic plants’ spectral data. VIs, known for their simplicity in extracting biophysical information, successfully distinguished healthy and diseased tissues in both plant species. The overall accuracy achieved was 63% and 71% for tomato and kiwi, respectively. Limitations were observed, particularly in differentiating specific disease infections accurately. On the other hand, GPC-BAT, after feature reduction, showcased enhanced accuracy in identifying healthy and diseased tissues. The overall accuracy ranged from 70% to 75% in the tomato and kiwi case studies. Despite its effectiveness, the model faced challenges in accurately predicting certain disease infections, especially in the early stages. Comparative analysis revealed commonalities and differences in the spectral bands identified by both approaches, with overlaps in critical regions across plant species. Notably, these spectral regions corresponded to the absorption regions of various photosynthetic pigments and structural components affected by bacterial infections in plant leaves. The study underscores the potential of hyperspectral sensing in disease diagnosis and highlights the strengths and limitations of VIs and GPC-BAT. The identified spectral features hold biological significance, suggesting correlations between bacterial infections and alterations in plant pigments and structural components. Future research avenues could focus on refining these approaches for improved accuracy in diagnosing diverse plant–pathogen interactions, thereby aiding disease diagnosis. Specifically, efforts could be directed towards adapting these methodologies for early detection, even before symptom manifestation, to better manage agricultural diseases.

Plant-soil sensing devices coupled with Artificial Intelligence autonomously collect and process in situ plant phenotypic data. A challenge of this approach is the limited incorporation of phenotype data into decision support systems designed to harness agricultural practices and forecast plant behavior within the intricate context of genotype, environment, and management interactions (G × E × M). To enhance the role of digital phenotyping in supporting Precision Agriculture, this paper proposes a sensing network based on the Internet of Things. The developed system comprises three modules: data collection, communication, and a cloud server. Several processes co-occur in the server, namely data visualization to confirm the correct sensors and data stream functioning. In addition, a crop growth model (CGM) runs on the server, which is powered by the collected data. The simulations generated by the model will support agricultural decisions, obtaining, in advance, insights about plant behavior considering several G × E × M scenarios. To assess the performance of the proposed network to provide reliable data to the model, a greenhouse was equipped with several sensors that collect plant, environment, and soil data (e.g., leaf numbers, air temperature, soil moisture). The proposed network can provide real-time causal support for advanced agricultural practices, evolving from a data-driven approach to an integrative framework where context (G × E × M) drives decision making.

Yield forecasting is of immeasurable value in modern viticulture to optimize harvest scheduling and quality management. Traditionally, this task is carried out through manual and destructive sampling of production components and their accurate assessment is expensive, time-consuming, and error-prone, resulting in erroneous projections. The number of inflorescences and flowers per vine is one of the main components and serves as an early predictor. The adoption of new non-invasive technologies can automate this task and drive viticulture yield forecasting to higher levels of accuracy. In this study, different Single Stage Instance Segmentation models from the state-of-the-art You Only Look Once (YOLO) family, such as YOLOv5 and YOLOv8, were benchmarked on a dataset of RGB images for grapevine inflorescence detection and segmentation, with the aim of validating and subsequently implementing the solution for counting the number of inflorescences and flowers. All models obtained promising results, with the YOLOv8s and the YOLOv5s models standing out with an F1-Score of 95.1% and 97.7% for the detection and segmentation tasks, respectively. Moreover, the low inference times obtained demonstrate the models’ ability to be deployed in real-time applications, allowing for non-destructive predictions in uncontrolled environments.

The potential of hyperspectral UV–VIS–NIR reflectance for the in-field, non-destructive discrimination of bacterial canker on kiwi leaves caused by Pseudomonas syringae pv. actinidiae (Psa) was analyzed. Spectral data (325–1075 nm) of twenty kiwi plants were obtained in vivo and in situ with a handheld spectroradiometer in two commercial kiwi orchards in northern Portugal over 15 weeks, resulting in 504 spectral measurements. The suitability of different vegetation indexes (VIs) and applied predictive models (based on supervised machine learning algorithms) for classifying non-symptomatic and symptomatic kiwi leaves was evaluated. Eight distinct types of VIs were identified as relevant for disease diagnosis, highlighting the relevance of the Green, Red, Red-Edge, and NIR spectral features. The class prediction was achieved with good model metrics, achieving an accuracy of 0.71, kappa of 0.42, sensitivity of 0.67, specificity of 0.75, and F1 of 0.67. Thus, the present findings demonstrated the potential of hyperspectral UV–VIS–NIR reflectance for the non-destructive discrimination of bacterial canker on kiwi leaves.

In recent years, there has been a remarkable surge in the development and research of tethered aerial systems, thus reflecting a growing interest in their diverse applications. Long-term missions involving aerial vehicles present significant challenges due to the limitations of current battery solutions. Tethered vehicles can circumvent such restrictions by receiving their power from an element on the ground such as a ground station or a mobile terrestrial platform. Tethered Unmanned Aerial Vehicles (UAVs) can also be applied to load transportation achieved by a single or multiple UAVs. This paper presents a comprehensive systematic literature review, with a special focus on solutions published in the last five years (2017–2022). It emphasizes the key characteristics that are capable of grouping publications by application scope, propulsion method, energy transfer solution, perception sensors, and control techniques adopted. The search was performed in six different databases, thereby resulting in 1172 unique publications, from which 182 were considered for inclusion in the data extraction phase of this review. Among the various aircraft types, multirotors emerged as the most widely used category. We also identified significant variations in the application scope of tethered UAVs, thus leading to tailored approaches for each use case, such as the fixed-wing model being predominant in the wind generation application and the lighter-than-air aircraft in the meteorology field. Notably, the classical Proportional–Integral–Derivative (PID) control scheme emerged as the predominant control methodology across the surveyed publications. Regarding energy transfer techniques, most publications did not explicitly describe their approach. However, among those that did, high-voltage DC energy transfer emerged as the preferred solution. In summary, this systematic literature review provides valuable insights into the current state of tethered aerial systems, thereby showcasing their potential as a robust and sustainable alternative to address the challenges associated with long-duration aerial missions and load transportation.

Agricultural robotics is nowadays a complex, challenging, and exciting research topic. Some agricultural environments present harsh conditions to robotics operability. In the case of steep slope vineyards, there are several challenges: terrain irregularities, characteristics of illumination, and inaccuracy/unavailability of signals emitted by the Global Navigation Satellite System (GNSS). Under these conditions, robotics navigation becomes a challenging task. To perform these tasks safely and accurately, the extraction of reliable features or landmarks from the surrounding environment is crucial. This work intends to solve this issue, performing accurate, cheap, and fast landmark extraction in steep slope vineyard context. To do so, we used a single camera and an Edge Tensor Processing Unit (TPU) provided by Google's USB Accelerator as a small, high-performance, and low power unit suitable for image classification, object detection, and semantic segmentation. The proposed approach performs object detection using Deep Learning (DL)-based Neural Network (NN) models on this device to detect vine trunks. To train the models, Transfer Learning (TL) is used on several pre-trained versions of MobileNet V1 and MobileNet V2. A benchmark between the two models and the different pre-trained versions is performed. The models are pre-trained in a built in-house dataset, that is publicly available containing 336 different images with approximately 1,600 annotated vine trunks. There are considered two vineyards, one using camera images with the conventional infrared filter and others with an infrablue filter. Results show that this configuration allows a fast vine trunk detection, with MobileNet V2 being the most accurate retrained detector, achieving an overall Average Precision of 52.98%. We briefly compare the proposed approach with the state-of-the-art Tiny YOLO-V3 running on Jetson TX2, showing the outperformance of the adopted system in this work. Additionally, it is also shown that the proposed detectors are suitable for the Localization and Mapping problems.

Developing ground robots for agriculture is a demanding task. Robots should be capable of performing tasks like spraying, harvesting, or monitoring. However, the absence of structure in the agricultural scenes challenges the implementation of localization and mapping algorithms. Thus, the research and development of localization techniques are essential to boost agricultural robotics. To address this issue, we propose an algorithm called VineSLAM suitable for localization and mapping in agriculture. This approach uses both point- and semiplane-features extracted from 3D LiDAR data to map the environment and localize the robot using a novel Particle Filter that considers both feature modalities. The numeric stability of the algorithm was tested using simulated data. The proposed methodology proved to be suitable to localize a robot using only three orthogonal semiplanes. Moreover, the entire VineSLAM pipeline was compared against a state-of-the-art approach considering three real-world experiments in a woody-crop vineyard. Results show that our approach can localize the robot with precision even in long and symmetric vineyard corridors outperforming the state-of-the-art algorithm in this context.

The efficiency of agricultural practices depends on the timing of their execution. Environmental conditions, such as rainfall, and crop-related traits, such as plant phenology, determine the success of practices such as irrigation. Moreover, plant phenology, the seasonal timing of biological events (e.g., cotyledon emergence), is strongly influenced by genetic, environmental, and management conditions. Therefore, assessing the timing the of crops’ phenological events and their spatiotemporal variability can improve decision making, allowing the thorough planning and timely execution of agricultural operations. Conventional techniques for crop phenology monitoring, such as field observations, can be prone to error, labour-intensive, and inefficient, particularly for crops with rapid growth and not very defined phenophases, such as vegetable crops. Thus, developing an accurate phenology monitoring system for vegetable crops is an important step towards sustainable practices. This paper evaluates the ability of computer vision (CV) techniques coupled with deep learning (DL) (CV_DL) as tools for the dynamic phenological classification of multiple vegetable crops at the subfield level, i.e., within the plot. Three DL models from the Single Shot Multibox Detector (SSD) architecture (SSD Inception v2, SSD MobileNet v2, and SSD ResNet 50) and one from You Only Look Once (YOLO) architecture (YOLO v4) were benchmarked through a custom dataset containing images of eight vegetable crops between emergence and harvest. The proposed benchmark includes the individual pairing of each model with the images of each crop. On average, YOLO v4 performed better than the SSD models, reaching an F1-Score of 85.5%, a mean average precision of 79.9%, and a balanced accuracy of 87.0%. In addition, YOLO v4 was tested with all available data approaching a real mixed cropping system. Hence, the same model can classify multiple vegetable crops across the growing season, allowing the accurate mapping of phenological dynamics. This study is the first to evaluate the potential of CV_DL for vegetable crops’ phenological research, a pivotal step towards automating decision support systems for precision horticulture.

Object identification, such as tree trunk detection, is fundamental for forest robotics. Intelligent vision systems are of paramount importance in order to improve robotic perception, thus enhancing the autonomy of forest robots. To that purpose, this paper presents three contributions: an open dataset of 5325 annotated forest images; a tree trunk detection Edge AI benchmark between 13 deep learning models evaluated on four edge-devices (CPU, TPU, GPU and VPU); and a tree trunk mapping experiment using an OAK-D as a sensing device. The results showed that YOLOR was the most reliable trunk detector, achieving a maximum F1 score around 90% while maintaining high scores for different confidence levels; in terms of inference time, YOLOv4 Tiny was the fastest model, attaining 1.93 ms on the GPU. YOLOv7 Tiny presented the best trade-off between detection accuracy and speed, with average inference times under 4 ms on the GPU considering different input resolutions and at the same time achieving an F1 score similar to YOLOR. This work will enable the development of advanced artificial vision systems for robotics in forestry monitoring operations.

Total white blood cells count is an important diagnostic parameter in both human and veterinary medicines. State-of-the-art is performed by flow cytometry combined with light scattering or impedance measurements. Spectroscopy point-of-care has the advantages of miniaturization, low sampling, and real-time hemogram analysis. While white blood cells are in low proportions, while red blood cells and bilirubin dominate spectral information, complicating detection in blood. We performed a feasibility study for the direct detection of white blood cells counts in canine blood by visible-near infrared spectroscopy for veterinary applications, benchmarking current chemometrics techniques (similarity, global and local partial least squares, artificial neural networks and least-squares support vector machines) with self-learning artificial intelligence, introducing data augmentation to overcome the hurdle of knowledge representativity. White blood cells count information is present in the recorded spectra, allowing significant discrimination and equivalence between hemogram and spectra principal component scores. Chemometrics methods correlate white blood cells count to spectral features but with lower accuracy. Self-Learning Artificial Intelligence has the highest correlation (0.8478) and a small standard error of 6.92 × 109 cells/L, corresponding to a mean absolute percentage error of 25.37%. Such allows the accurate diagnosis of white blood cells in the range of values of the reference interval (5.6 to 17.8 × 109 cells/L) and above. This research is an important step toward the existence of a miniaturized spectral point-of-care hemogram analyzer.

Pseudomonas syringae pv. actinidiae (Psa) has been responsible for numerous epidemics of bacterial canker of kiwi (BCK), resulting in high losses in kiwi production worldwide. Current diagnostic approaches for this disease usually depend on visible signs of the infection (disease symptoms) to be present. Since these symptoms frequently manifest themselves in the middle to late stages of the infection process, the effectiveness of phytosanitary measures can be compromised. Hyperspectral spectroscopy has the potential to be an effective, non-invasive, rapid, cost-effective, high-throughput approach for improving BCK diagnostics. This study aimed to investigate the potential of hyperspectral UV–VIS reflectance for in-situ, non-destructive discrimination of bacterial canker on kiwi leaves. Spectral reflectance (325–1075 nm) of twenty plants were obtained with a handheld spectroradiometer in two commercial kiwi orchards located in Portugal, for 15 weeks, totaling 504 spectral measurements. Several modeling approaches based on continuous hyperspectral data or specific wavelengths, chosen by different feature selection algorithms, were tested to discriminate BCK on leaves. Spectral separability of asymptomatic and symptomatic leaves was observed in all multi-variate and machine learning models, including the FDA, GLM, PLS, and SVM methods. The combination of a stepwise forward variable selection approach using a support vector machine algorithm with a radial kernel and class weights was selected as the final model. Its overall accuracy was 85%, with a 0.70 kappa score and 0.84 F-measure. These results were coherent with leaves classified as asymptomatic or symptomatic by visual inspection. Overall, the findings herein reported support the implementation of spectral point measurements acquired in situ for crop disease diagnosis.

Proximity sensing approaches with a wide array of sensors available for use in precision viticulture contexts can nowadays be considered both well-know and mature technologies. Still, several in-field practices performed throughout different crops rely on direct visual observation supported on gained experience to assess aspects of plants’ phenological development, as well as indicators relating to the onset of common plagues and diseases. Aiming to mimic in-field direct observation, this paper presents VineInspector: a low-cost, self-contained and easy-to-install system, which is able to measure microclimatic parameters, and also to acquire images using multiple cameras. It is built upon a stake structure, rendering it suitable for deployment across a vineyard. The approach through which distinguishable attributes are detected, classified and tallied in the periodically acquired images, makes use of artificial intelligence approaches. Furthermore, it is made available through an IoT cloud-based support system. VineInspector was field-tested under real operating conditions to assess not only the robustness and the operating functionality of the hardware solution, but also the AI approaches’ accuracy. Two applications were developed to evaluate VineInspector’s consistency while a viticulturist’ assistant in everyday practices. One was intended to determine the size of the very first grapevines’ shoots, one of the required parameters of the well known 3–10 rule to predict primary downy mildew infection. The other was developed to tally grapevine moth males captured in sex traps. Results show that VineInspector is a logical step in smart proximity monitoring by mimicking direct visual observation from experienced viticulturists. While the latter traditionally are responsible for a set of everyday practices in the field, these are time and resource consuming. VineInspector was proven to be effective in two of these practices, performing them automatically. Therefore, it enables both the continuous monitoring and assessment of a vineyard’s phenological development in a more efficient manner, making way to more assertive and timely practices against pests and diseases.

The harvesting operation is a recurring task in the production of any crop, thus making it an excellent candidate for automation. In protected horticulture, one of the crops with high added value is tomatoes. However, its robotic harvesting is still far from maturity. That said, the development of an accurate fruit detection system is a crucial step towards achieving fully automated robotic harvesting. Deep Learning (DL) and detection frameworks like Single Shot MultiBox Detector (SSD) or You Only Look Once (YOLO) are more robust and accurate alternatives with better response to highly complex scenarios. The use of DL can be easily used to detect tomatoes, but when their classification is intended, the task becomes harsh, demanding a huge amount of data. Therefore, this paper proposes the use of DL models (SSD MobileNet v2 and YOLOv4) to efficiently detect the tomatoes and compare those systems with a proposed histogram-based HSV colour space model to classify each tomato and determine its ripening stage, through two image datasets acquired. Regarding detection, both models obtained promising results, with the YOLOv4 model standing out with an F1-Score of 85.81%. For classification task the YOLOv4 was again the best model with an Macro F1-Score of 74.16%. The HSV colour space model outperformed the SSD MobileNet v2 model, obtaining results similar to the YOLOv4 model, with a Balanced Accuracy of 68.10%.

Robotics navigation and perception for forest management are challenging due to the existence of many obstacles to detect and avoid and the sharp illumination changes. Advanced perception systems are needed because they can enable the development of robotic and machinery solutions to accomplish a smarter, more precise, and sustainable forestry. This article presents a state-of-the-art review about unimodal and multimodal perception in forests, detailing the current developed work about perception using a single type of sensors (unimodal) and by combining data from different kinds of sensors (multimodal). This work also makes a comparison between existing perception datasets in the literature and presents a new multimodal dataset, composed by images and laser scanning data, as a contribution for this research field. Lastly, a critical analysis of the works collected is conducted by identifying strengths and research trends in this domain.

The agricultural sector plays a fundamental role in our society, where it is increasingly important to automate processes, which can generate beneficial impacts in the productivity and quality of products. Perception and computer vision approaches can be fundamental in the implementation of robotics in agriculture. In particular, deep learning can be used for image classification or object detection, endowing machines with the capability to perform operations in the agriculture context. In this work, deep learning was used for the detection of grape bunches in vineyards considering different growth stages: the early stage just after the bloom and the medium stage where the grape bunches present an intermediate development. Two state-of-the-art single-shot multibox models were trained, quantized, and deployed in a low-cost and low-power hardware device, a Tensor Processing Unit. The training input was a novel and publicly available dataset proposed in this work. This dataset contains 1929 images and respective annotations of grape bunches at two different growth stages, captured by different cameras in several illumination conditions. The models were benchmarked and characterized considering the variation of two different parameters: the confidence score and the intersection over union threshold. The results showed that the deployed models could detect grape bunches in images with a medium average precision up to 66.96%. Since this approach uses low resources, a low-cost and low-power hardware device that requires simplified models with 8 bit quantization, the obtained performance was satisfactory. Experiments also demonstrated that the models performed better in identifying grape bunches at the medium growth stage, in comparison with grape bunches present in the vineyard after the bloom, since the second class represents smaller grape bunches, with a color and texture more similar to the surrounding foliage, which complicates their detection.

Mobile robotics in forests is currently a hugely important topic due to the recurring appearance of forest wildfires. Thus, in-site management of forest inventory and biomass is required. To tackle this issue, this work presents a study on detection at the ground level of forest tree trunks in visible and thermal images using deep learning-based object detection methods. For this purpose, a forestry dataset composed of 2895 images was built and made publicly available. Using this dataset, five models were trained and benchmarked to detect the tree trunks. The selected models were SSD MobileNetV2, SSD Inception-v2, SSD ResNet50, SSDLite MobileDet and YOLOv4 Tiny. Promising results were obtained; for instance, YOLOv4 Tiny was the best model that achieved the highest AP (90%) and F1 score (89%). The inference time was also evaluated, for these models, on CPU and GPU. The results showed that YOLOv4 Tiny was the fastest detector running on GPU (8 ms). This work will enhance the development of vision perception systems for smarter forestry robots.

The development of robotic solutions for agriculture requires advanced perception capabilities that can work reliably in any crop stage. For example, to automatise the tomato harvesting process in greenhouses, the visual perception system needs to detect the tomato in any life cycle stage (flower to the ripe tomato). The state-of-the-art for visual tomato detection focuses mainly on ripe tomato, which has a distinctive colour from the background. This paper contributes with an annotated visual dataset of green and reddish tomatoes. This kind of dataset is uncommon and not available for research purposes. This will enable further developments in edge artificial intelligence for in situ and in real-time visual tomato detection required for the development of harvesting robots. Considering this dataset, five deep learning models were selected, trained and benchmarked to detect green and reddish tomatoes grown in greenhouses. Considering our robotic platform specifications, only the Single-Shot MultiBox Detector (SSD) and YOLO architectures were considered. The results proved that the system can detect green and reddish tomatoes, even those occluded by leaves. SSD MobileNet v2 had the best performance when compared against SSD Inception v2, SSD ResNet 50, SSD ResNet 101 and YOLOv4 Tiny, reaching an F1-score of 66.15%, an mAP of 51.46% and an inference time of 16.44ms with the NVIDIA Turing Architecture platform, an NVIDIA Tesla T4, with 12 GB. YOLOv4 Tiny also had impressive results, mainly concerning inferring times of about 5 ms.

Smart and precision agriculture concepts require that the farmer measures all relevant variables in a continuous way and processes this information in order to build better prescription maps and to predict crop yield. These maps feed machinery with variable rate technology to apply the correct amount of products in the right time and place, to improve farm profitability. One of the most relevant information to estimate the farm yield is the Leaf Area Index. Traditionally, this index can be obtained from manual measurements or from aerial imagery: the former is time consuming and the latter requires the use of drones or aerial services. This work presents an optical sensing-based hardware module that can be attached to existing autonomous or guided terrestrial vehicles. During the normal operation, the module collects periodic geo-referenced monocular images and laser data. With that data a suggested processing pipeline, based on open-source software and composed by Structure from Motion, Multi-View Stereo and point cloud registration stages, can extract Leaf Area Index and other crop-related features. Additionally, in this work, a benchmark of software tools is made. The hardware module and pipeline were validated considering real data acquired in two vineyards—Portugal and Italy. A dataset with sensory data collected by the module was made publicly available. Results demonstrated that: the system provides reliable and precise data on the surrounding environment and the pipeline is capable of computing volume and occupancy area from the acquired data.

The development of robotic solutions in unstructured environments brings several challenges, mainly in developing safe and reliable navigation solutions. Agricultural environments are particularly unstructured and, therefore, challenging to the implementation of robotics. An example of this is the mountain vineyards, built-in steep slope hills, which are characterized by satellite signal blockage, terrain irregularities, harsh ground inclinations, and others. All of these factors impose the implementation of precise and reliable navigation algorithms, so that robots can operate safely. This work proposes the detection of semantic natural landmarks that are to be used in Simultaneous Localization and Mapping algorithms. Thus, Deep Learning models were trained and deployed to detect vine trunks. As significant contributions, we made available a novel vine trunk dataset, called VineSet, which was constituted by more than 9000 images and respective annotations for each trunk. VineSet was used to train state-of-the-art Single Shot Multibox Detector models. Additionally, we deployed these models in an Edge-AI fashion and achieve high frame rate execution. Finally, an assisted annotation tool was proposed to make the process of dataset building easier and improve models incrementally. The experiments show that our trained models can detect trunks with an Average Precision up to 84.16% and our assisted annotation tool facilitates the annotation process, even in other areas of agriculture, such as orchards and forests. Additional experiments were performed, where the impact of the amount of training data and the comparison between using Transfer Learning and training from scratch were evaluated. In these cases, some theoretical assumptions were verified.

Sap flow measurements of trees are today the most common method to determine evapotranspiration at the tree and the forest/crop canopy level. They provide independent measurements for flux comparisons and model validation. The most common approach to measure the sap flow is based on intrusive solutions with heaters and thermal sensors. This sap flow sensor technology is not very reliable for more than one season crop; it is intrusive and not adequate for low diameter trunk trees. The non-invasive methods comprise mostly Radio-frequency (RF) technologies, typically using satellite or air-born sources. This system can monitor large fields but cannot measure sap levels of a single plant (precision agriculture). This article studies the hypothesis to use of RF signals attenuation principle to detect variations in the quantity of water present in a single plant. This article presents a well-defined experience to measure water content in leaves, by means of high gains RF antennas, spectrometer, and a robotic arm. Moreover, a similar concept is studied with an off-the-shelf radar solution—for the automotive industry—to detect changes in the water presence in a single plant and leaf. The conclusions indicate a novel potential application of this technology to precision agriculture as the experiments data is directly related to the sap flow variations in plant.

Research and development of autonomous mobile robotic solutions that can perform several active agricultural tasks (pruning, harvesting, mowing) have been growing. Robots are now used for a variety of tasks such as planting, harvesting, environmental monitoring, supply of water and nutrients, and others. To do so, robots need to be able to perform online localization and, if desired, mapping. The most used approach for localization in agricultural applications is based in standalone Global Navigation Satellite System-based systems. However, in many agricultural and forest environments, satellite signals are unavailable or inaccurate, which leads to the need of advanced solutions independent from these signals. Approaches like simultaneous localization and mapping and visual odometry are the most promising solutions to increase localization reliability and availability. This work leads to the main conclusion that, few methods can achieve simultaneously the desired goals of scalability, availability, and accuracy, due to the challenges imposed by these harsh environments. In the near future, novel contributions to this field are expected that will help one to achieve the desired goals, with the development of more advanced techniques, based on 3D localization, and semantic and topological mapping. In this context, this work proposes an analysis of the current state-of-the-art of localization and mapping approaches in agriculture and forest environments. Additionally, an overview about the available datasets to develop and test these approaches is performed. Finally, a critical analysis of this research field is done, with the characterization of the literature using a variety of metrics.

Traditionally farmers have used their perceptual sensorial systems to diagnose and monitor their crops health and needs. However, humans possess five basic perceptual systems with accuracy levels that can change from human to human which are largely dependent on the stress, experience, health and age. To overcome this problem, in the last decade, with the help of the emergence of smartphone technology, new agronomic applications were developed to reach better, cost-effective, more accurate and portable diagnosis systems. Conventional smartphones are equipped with several sensors that could be useful to support near real-time usual and advanced farming activities at a very low cost. Therefore, the development of agricultural applications based on smartphone devices has increased exponentially in the last years. However, the great potential offered by smartphone applications is still yet to be fully realized. Thus, this paper presents a literature review and an analysis of the characteristics of several mobile applications for use in smart/precision agriculture available on the market or developed at research level. This will contribute to provide to farmers an overview of the applications type that exist, what features they provide and a comparison between them. Also, this paper is an important resource to help researchers and applications developers to understand the limitations of existing tools and where new contributions can be performed.