Search

In current plant phenotyping research, the study of plant time-series images based on deep learning has received widespread attention. While such image data is relatively easy to obtain, the cost of annotation is high. One efficient method for achieving cost-effective training is through contrastive learning. Plant growth is slow, and the changes in image sequences over a period of time are small, with simple semantic information. Previous contrastive pre-training models struggled to effectively distinguish positive samples from the same image with different augmented views and similar negative samples from different images. Therefore, this paper proposes a method called self-supervised contrastive learning method for plant time-series images with a Priori Distance Embedding (PDE). The semantic information in images corresponding to different phenological stages of plants varies. This method transforms this crucial domain knowledge into prior distances for image pairs and conducts contrastive learning pre-training. The learned weights can be transferred to downstream tasks. Building upon this method, experiments were conducted on cherry time-series images to assess the quality of pre-training through a plant phenotyping image semantic segmentation task. To provide a comprehensive example of plant time-series image phenotypic analysis, this paper establishes a cherry growth temporal model, specifically including PDE pre-training, anomaly detection, semantic segmentation, and recording the results from the temporal dimension. The experiments indicate that this self-supervised contrastive learning method can be effectively applied to the pre-training of plant time-series images, demonstrating broad applicability in various computer vision studies related to plant phenotyping. • A pre-training method for plant images with a Priori Distance Embedding (PDE). • Transforms the phenological information into prior distances for image pairs. • An image acquisition device was built to obtain cherry time series images. • The PDE method was tested by plant phenotype image segmentation task. • We provided an exhaustive example of cherry time-series image phenotypic analysis. [ABSTRACT FROM AUTHOR]

• An efficient method is proposed for orchard tree trunk detection in natural scenes. • A multi-scenes orchard tree trunk dataset is established for experiments. • The YOLOv5 model structure is improved to enhance detection accuracy and speed. • Proposed method is robust against complex orchard environment. The recognition and localization of fruit tree trunks in orchard are important for orchard operation robots, which are the bases for automatic navigation, fruit tree spraying and fertilization etc. A method was proposed based on machine vision to detect target objects such as fruit tree trunks, person and supporters in orchard by improving the YOLOv5 deep learning algorithm in this paper, which is applicable to the recognition tasks in natural orchard environments. Firstly, 1354 images of the natural orchard collected by camera were image enhanced, weather effects such as rain, snow, bright light, shadow and fog were added to expand the dataset and to increase the robustness of the model. Secondly, the original YOLOv5 model was improved by replacing the Bottleneck network in the C3 module with the lightweight GhostNet V2 to reduce the network parameters, and changing the box loss function CIoU to SIoU in the loss function to make the regression of the detection box more accurate, and coordinate attention mechanism (CA) was added to the network to reduce the interference of useless background information in images. Before training, pre-anchor boxes were generated by using IoU-based K-means clustering, after that the dataset was fed into the improved YOLOv5 for training, and the trained model was used to detect the trunks. Finally, weighted boxes fusion (WBF) was used instead of the non-maximum suppression (NMS) in this paper for the output of the detection boxes. Then the density-based spatial clustering of applications with noise (DBSCAN) algorithm was used for trunk clustering. The improved target detection method was trained and validated on the experimental dataset. The model size is reduced by 43.6 %, model parameters are reduced by 46.9 %, and the mAP reaches 97.1 %, with an average detection speed of 198.2 ms per image. Compared with the original YOLOv5, the model is more lightweight, the detection accuracy and speed are improved. The improved YOLOv5 is also better than YOLOv3, NanoDet and SSD in terms of combined accuracy and speed, and has similar performance to YOLO_MobileNet in orchard dataset. The experimental results show that the improved YOLOv5 target detection model proposed in this paper is lightweight while still having better detection accuracy and detection speed in complex environments, and the model is small enough to be deployed to mobile or low-performance terminals for target detection in natural orchard environments. [ABSTRACT FROM AUTHOR]

• Precision agriculture method applicable to remote on-tree pomegranate pixel-based fruit size monitoring. • Comparison of Deep Learning and zero-shot vision Transformer instance segmentation models. • Image-Based size estimation algorithm for occluded fruits with protuberances. In situ automatic fruit monitoring is of great interest for more accurate and cost-efficient decision making in agriculture. For this purpose, the development of computer vision-based tools is essential. Deep Learning techniques have shown good performance in fruit detection and segmentation. Recently, new models based on Transformer architecture have emerged with promising potential and zero-shot inference capability. In this paper, a Deep Learning model, Mask R-CNN, was trained for on-tree pomegranate fruit segmentation and compared with foundational models based on Vision Transformer, Grounding DINO and Segment Anything Model. Results with Mask R-CNN proved a better performance, according to F1 score and AP metrics, and a lower computational cost, according to prediction time. One of the most interesting derived applications from fruit segmentation is fruit size estimation. However, segmented fruit masks are frequently incomplete due to occlusions. Therefore, image fruit size estimation is not a straightforward process. In this work, we also propose a novel algorithm to estimate and monitor the fruit size in pixel units from the automated masks. A median relative error of 1.39% was obtained, demonstrating the potential and feasibility of future fully-automatic fruit size estimators. [ABSTRACT FROM AUTHOR]

AgroCounters is an open-source repository for counting objects in images in the agricultural domain by utilizing deep-learning algorithms. In this paper, we present the framework of AgroCounters, which integrates state-of-the-art deep learning models, including regression-based counting, detection-based counting, and density-estimation-based counting, to accurately count various agricultural objects, such as fruits, vegetables, and livestock, in single images. The framework utilizes transfer learning techniques to optimize model performance on the limited labeled data available in the agricultural domain. We provide an open-source implementation of AgroCounters, which includes a multitude of algorithms for counting applications and a toolbox that includes metrics, training data tools, visualizations, and a simple installation guide for several open-source implementations of counting methods. We evaluated the performance of AgroCounters on multiple agricultural datasets acquired from RGB sensors, including plant leaves, melons, wheat grains, cherry tomatoes, grapes, apple flowers, bananas (fruit and leaves), pears, and chickens. We compared the results of the various implemented methods over these datasets and showcased the most suitable solution for each. YOLOv5, the most recent of the compared object detectors, provided the best results on all the examined datasets, and there was no clear 'winner' between Faster-RCNN and RetinaNet. Based on the analyzed datasets, when higher accuracy is required, the direct regression network (DRN) should be used; for small datasets, multiple scale regression (MSR) gives superior results. Based on the developments, we proposed guidelines for developing deep-learning-based counting solutions for agricultural applications, focusing on solutions and best practices for the agricultural domain. Overall, AgroCounters presents a promising solution for automated counting in the agricultural domain, offering significant potential for reducing manual labor, improving crop management, and increasing productivity. • AgroCounters is a repository for counting agriculture objects using deep learning. • AgroCounters includes regression, detection, and density estimation algorithms. • AgroCounters provides toolboxes for metric calculation, training, and visualization. • AgroCounters publicizes 10 different datasets with annotations and trained models. • Present guidelines for developing deep learning-based counters in agriculture. [ABSTRACT FROM AUTHOR]

• An adaptive cruise weeding robot for paddy fields was developed for the first time. • A real-time rice seedling recognition method based on MW-YOLOv5s was proposed. • A navigation line fitting method for paddy field adaptive operations was proposed. Weed control in paddy fields is a critical agronomic practice for enhancing rice yield, in which mechanical weed control is widely used due to its high weed control rate and convenient operation. However, traditional mechanical weed control methods require manual operation, leading to increased operational costs. Therefore, adaptive cruise weeding robots hold significant application promise and market potential. In this paper, an adaptive cruise weeding robot for paddy fields based on improved YOLOv5 is designed to adaptively perform weeding operations. Firstly, a real-time rice seedling recognition method based on MW-YOLOv5s is proposed by establishing a rice seedling dataset under different paddy field environments and growth stages. The model combines the Backbone network structure with MobileViTv3, replacing GIoU_loss with WIoU_loss, which significantly improves the precision and speed of rice seedling recognition. Next, the MW-YOLOv5s was integrated into a paddy weeding machine, and seedling navigation lines were extracted using the least square method. Finally, the control system autonomously operates the weeding machine in real time through feedback control based on the navigation path. The test results showed that the MW-YOLOv5s model exhibits strong recognition performance for rice seedlings under different paddy field backgrounds and growth stages, with a precision of 90.05 % and an mAP of 92.32 %, while the real-time performance reaches 19.51 FPS, which meet the requirements for real-time operation of the weeding machine in paddy fields. The results of the paddy field weed control experiment reveal a weed control rate of 82.4 % and a seedling injury rate of 2.8 %, which meet the agronomic requirements for mechanical weed control in paddy fields. The research findings contribute to the design of intelligent agricultural equipment and promote the practice and application of machine vision in the field of paddy field plant protection. [ABSTRACT FROM AUTHOR]

• Detailed review of over 300 research papers. • Object Detection and Tracking are suitable techniques to enhance Precision Farming. • Critical discussion regarding the challenges of Object Detection and Tracking. • Potential research lines to improve Object Detection and Tracking are identified. Object Detection and Tracking have gained importance in recent years because of the great advances in image and video analysis techniques and the accurate results these technologies are producing. Moreover, they have successfully been applied to multiple fields, including the agricultural domain since they offer real-time monitoring of the status of the crops and animals while counting how many are present within a field/barn. This review aims to review the current literature on Object Detection and Tracking within the field of Precision Farming. For that, over 300 research articles were explored, from which 150 articles from the last five years were systematically reviewed and analysed regarding the algorithms they implemented, the domain they belong to, the difficulties they faced, and which limitations should be tackled in the future. Lastly, it examines potential issues that this approach might have, for instance, the lack of open-source datasets with labelled data. The findings of this study indicate that Object Detection and Tracking are critical techniques to enhance Precision Farming and pave the way for robotization for the agricultural sector since they provide accurate results and insights on crop and animal management, and optimize resource allocation. Future work should focus on the optimal acquisition of the datasets prior to Object Detection and Tracking, along with the consideration of the biophysical environment of the farming scenarios. [ABSTRACT FROM AUTHOR]

• We introduce a unique dataset of UAV thermal imagery, meticulously collected under diverse heights, angles, and climatic conditions, designed to challenge and validate thermal imaging analysis. • An approach for tracking mammalian herbivores in thermal imagery using the Multi-Object Tracking and Segmentation model Track R-CNN is proposed. • 3D convolutions outperform LSTM convolutions as a temporal component. • The association head outperforms optical flow as a tracking mechanism. • The proposed approach is robust across a wide range of lighting and thermal conditions and on imagery collected at various heights and angles. Lightweight Unmanned Aerial Vehicles (UAVs) are emerging as a remote sensing survey tool for animal monitoring in several fields, such as precision livestock farming. Together with state-of-the-art computer vision techniques, UAV technology has drastically escalated our ability to acquire and analyse visual data in the field, lowering both costs and complications associated with collection and analysis. This paper addresses monitoring mammalian herbivores using the unexploited field of thermal Multi-Object Tracking and Segmentation (MOTS) in UAV imagery. In our research, a state-of-the-art MOTS algorithm (Track R-CNN) was trained and evaluated in the segmentation, detection and tracking of dairy cattle. Data collection was carried out in two farms with a UAV carrying a thermal camera at various angles and heights, and under different light (overcast/sunny) and thermal (16.5 °C range) conditions. Our findings suggest that dataset diversity and balance, especially regarding the range of conditions under which the data was collected, can significantly enhance tracking efficiency in specific scenarios. For training the algorithm, transfer learning was used as a knowledge migration method. The performance of our best model (68.5 sMOTSA, 79.6 MOTSA, 41 IDS, 100 % counting accuracy, and 87.2 MOTSP), which utilizes 3D convolutions and an association head, demonstrates the applicability and optimal performance of Track R-CNN in detecting, tracking, and counting herbivores in UAV thermal imagery under heterogenous conditions. Our findings demonstrate that 3D convolutions outperform Long-short Term Memory (LSTM) convolutions. However, LSTM convolutions also show optimal performance, offering a viable alternative. Furthermore, our results highlight the inability of Optical Flow to track motionless animals (-15 sMOTSA, −4.1 MOTSA and 2076 IDS) and the proficiency of the association head in differentiating static animals from the background. This research contributes to the growing body of knowledge in automated mammalian herbivore monitoring, with potential applications such as precision livestock farming and wildlife conservation. [ABSTRACT FROM AUTHOR]

• An anti-lodging machine, comprising an anti-lodging device and an AGV (Automated Guided Vehicle) was designed to replace manual methods. • A machine vision system was developed to aim the anti-lodging device at the crown buds. • The YOLOv5s algorithm is enhanced for detecting immature pineapple crown buds. • The mAP_0.5 of the improved YOLOv5s algorithm is 3.71 %, 1.69 %, and 1.19 % higher than the original YOLOv5s, SSD, and Faster R-CNN, respectively. • The operating speed of this intelligent anti-lodging method is 2.7 times faster than manual binding. Sunscorch, a common disease in pineapple cultivation, is caused by direct sunlight on the surface of lodging pineapple fruits. To mitigate this issue, pineapple fruits are often bagged or supported with fixing rods. This paper presents the design of an anti-lodging machine, comprising an anti-lodging device and an AGV (Automated Guided Vehicle), and develops a machine vision system to precisely aim the anti-lodging device at the crown buds. The YOLOv5s algorithm is enhanced for detecting crown buds. The improvement of the YOLOv5s algorithm includes the introduction of the attention mechanism CBAM to enhance feature extraction, the addition of BiFPN and GhostNet to strengthen feature fusion and to reduce detection time and computational cost. By transforming pixel coordinates, the relative coordinates between the crown bud and the device are calculated to achieve accurate targeting. The experiments demonstrate that the precision, recall, and map_0.5 of the improved YOLOv5s algorithm are 91.89 %, 92.25 %, 94.01 %, and the detection speed is 15.71 ms/image. The average detection speed is 1.14 ms/image and 9.45 ms/image faster than that of SSD and Faster R-CNN, benefiting from a parameter number only 3/4 and 3/7 of SSD and Faster R-CNN, respectively. The results of contrast experiments indicate that, although the lodging prevention rate of this intelligent anti-lodging method is 4 % lower than the manual binding, its operating speed is 2.7 times faster. [ABSTRACT FROM AUTHOR]

• Cows and their abdomen areas were located using YOLOv5. • Individual cows were measured remotely, non-invasively, and simultaneously. • Heat stress was detected when the respiration rate reached 60 breaths per minute. In cattle, respiration rate (RR) provides researchers and practitioners with valuable physiological information. However, traditional visual observation requires massive labour and is impractical for large-scale commercial farms. Recently developed vision-based RR measurement methods are highly limited to measuring a small number of animals in a controlled environment. Therefore, this paper aimed to propose a vision-based multi-object RR measurement method for dairy cows lying in free stalls. An RGB camera was aimed at the lying zone and was able to cover about 16 stalls. The proposed framework first utilised two YOLOv5-based networks to segment cow instances and detect cow flank objects. Next, an object tracker was used to link the predictions of each cow throughout the video clip. The Lucas-Kanade optical flow was then calculated specifically on the overlapped area of the cow mask and the flank bounding box. Finally, RR was extracted using Fast Fourier Transform. The results show that the proposed method had a precise RR measurement with a correlation coefficient of 0.944, a root mean square error of 5.35 breaths per minute, and an intraclass correlation coefficient of 0.974 when compared to visual observation. The piecewise regression models identified a change in RR when the ambient temperature reached 23.6 °C or the temperature-humidity index reached 72. The corresponding RR thresholds were 60.9 and 60.2 breaths per minute, respectively. Collectively, these results can be used to inform an automated local cooling system, e.g., fans in the lying area. However, more experiments and calibration with data collected using more cost-effective video recording systems are required before this technology can be applied on farms. [ABSTRACT FROM AUTHOR]

• Amaranthus palmeri growth stage recognition was possible with YOLO architectures. • YOLO v8-X achieved the highest mAP@[0.5:0.95] for growth stage recognition of 47.34% • A. palmeri single-class outperformed 8-class recognition by over 20% (v7-original) • The first high-quality, benchmarking, growth stage dataset is provided. Many advanced image-based precision agricultural technologies for plant breeding, field crop research, and site-specific crop management hinge on the reliable detection and phenotyping of plants across highly variable morphological growth stages. Convolutional neural networks (CNNs) have shown promise for image-based plant phenotyping and weed recognition, but the ability to recognize growth stages, often with stark differences in appearance, is uncertain. Palmer amaranth (Amaranthus palmeri) is a particularly challenging weed plant in cotton (Gossypium hirsutum) production; due to high genetic diversity, it exhibits highly variable plant morphology across growth stages over a growing season, as well as between plants at a given growth stage. This paper investigates eight-class growth stage recognition of A. palmeri in cotton as a challenging detection case study for You Only Look Once (YOLO) architectures. In total, 26 architecture variants from YOLO v3, v5, v6, v6 3.0, v7, and v8 are compared on an eight-class growth stage dataset of A. palmeri. The highest mean average precision (mAP@[0.5:0.95]) for recognition of all growth stage classes was 47.34% achieved by v8-X, with inter-class confusion across visually similar classes. With all growth stages grouped as a single class, performance increased, with a maximum mAP@[0.5:0.95] of 67.05% achieved by v7-Original. Single-class recall of up to 81.42% was achieved by v5-X, and precision of up to 89.72% was achieved by v8-X. Class activation maps (CAM) were used to understand model attention on the complex dataset. Fewer classes, grouped by visual or size features improved performance over the ground-truth eight-class dataset. Successful growth stage detection highlights the substantial opportunity for improving plant phenotyping and weed recognition technologies with open-source object detection architectures. Additionally, the first open-access, benchmarking growth stage dataset ' Palmer amaranth growth stages – 8 (PAGS8) ' is presented, which is available here: https://weed-ai.sydney.edu.au/datasets/5c78d067-8750-4803-9cbe-57df8fae55e4. [ABSTRACT FROM AUTHOR]

• A transformer-based multiple fish tracking network: TFMFT. • Multiple Association (MA) method for maximum matching. • Empirical studies on diverse transformer tracking architectures. • A multiple fish tracking dataset with various fish species and scenarios. Recent advancements in fish tracking methodologies provide valuable solutions for assessing fish growth, marine fisheries, and biological research. In particular, there has been a burgeoning interest in vision-based methods for fish tracking, owing to the enhanced computational capabilities facilitated by deep learning models. However, these methods face several challenges, including poor fish detection performance under complex backgrounds, the potential for identification switches caused by the non-rigid features and occlusions of fish, and the limited fault tolerance of extant approaches. In this paper, a t rans f ormer-based m ultiple f ish t racking model (TFMFT) is proposed, specifically designed to address the issue of instance loss of fish targets in aquaculture ponds with complex background disturbance. In particular, we introduce a Multiple Association (MA) method that bolsters fault tolerance in tracking by synthesizing simple Intersection-over-Union matching in the identification (ID) matching module. Through empirical studies across diverse Transformer-based models, we comprehensively assessed the influence of architecture design on data requirements. Furthermore, to evaluate the performance and generalizability of fish tracking models, we present the Multiple_Fish_Tracking_2022 (MFT22) dataset. The results demonstrate that TFMFT achieves 30.6% IDF1 (Identification F-Score) on the MFT22 dataset, outperforming the state-of-the-art by 10.9% and showcasing superior performance over other models. The resources and pre-trained model will be available at: https://github.com/vranlee/TFMFT. [ABSTRACT FROM AUTHOR]

• A dataset on small target pests on greenhouse sticky traps. • A novel solution was developed for counting multiple small targets based on density maps, simultaneously. • An adaptive sample equalization algorithm was developed to tackle the issue of class imbalance. Whiteflies and fruit flies are common pests that adversely affect greenhouse crops, so it is vital to control their numbers promptly. Current research involves setting up artificial monitoring devices with fixed yellow sticky traps (YSTs), for pest surveillance. However, these devices can only be installed in specific environments and additional interventions may impact the actual distribution of greenhouse pests. On the other hand, detection methods, which are the most commonly used, struggle to accurately detect overlapping or closely interacting small insects, which is a common occurrence in the presence of pests. In this paper, we propose a novel solution for the multi-species pest recognition and counting method based on a density map to tackle pest counting in the greenhouse. To reduce interference between different types of objects, we used two annotation maps to represent the distribution of each object separately. Then, two counting branches are designed to count the pests, respectively. Finally, the detected pests are mapped onto the original image to display their spatial positions. In addition, an adaptive sample equalization method is proposed to address the issue of class imbalance in images, and the atrous spatial pyramid pooling (ASPP) structure and the CBAM attention mechanism are added to the fruit fly branches to focus on learning fruit fly features from crowded edges of YSTs. When applied to split images, our method achieved reliable counting performance in counting both whiteflies and fruit flies, with R2 scores of 0.973 and 0.972, respectively. This method has been embedded into mobile devices to assist greenhouse workers in monitoring pest populations in the field. The experiments show that the proposed method provides a promising direction for counting multiple small target insects and monitoring pest populations in crowded images. At the same time, it demonstrates the potential of density map methods in various small target detection scenarios. [ABSTRACT FROM AUTHOR]