1. Video-based fruit detection and tracking for apple counting and mapping
- Author
-
Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions, Universitat Politècnica de Catalunya. IDEAI-UPC - Intelligent Data sciEnce and Artificial Intelligence Research Group, Gené Mola, Jordi, Felip Pomés, Marc, Net Barnés, Francesc, Morros Rubió, Josep Ramon, Miranda, Juan C., Arnó Satorra, Jaume, Asín Jones, Luis, Lordan Sanahuja, Jaume, Ruiz Hidalgo, Javier, Gregorio López, Eduard, Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions, Universitat Politècnica de Catalunya. IDEAI-UPC - Intelligent Data sciEnce and Artificial Intelligence Research Group, Gené Mola, Jordi, Felip Pomés, Marc, Net Barnés, Francesc, Morros Rubió, Josep Ramon, Miranda, Juan C., Arnó Satorra, Jaume, Asín Jones, Luis, Lordan Sanahuja, Jaume, Ruiz Hidalgo, Javier, and Gregorio López, Eduard
- Abstract
Automatic fruit counting systems have garnered interest from farmers and agronomists to monitor fruit production, predict yields in advance, and identify production variability across orchards. However, accurately counting fruits poses challenges, particularly due to occlusions. This study proposes a multi-view sensing approach using continuous motion videos captured by a camera moved along the row of trees, followed by fruit detection in all video frames and application of Multi-Object Tracking (MOT) algorithms to prevent double-counting. Three tracking methods, namely SORT, DeepSORT, and ByteTrack, are compared for fruit counting using the YOLOv5x object detector. The methodology is applied to map fruit production in an experimental apple orchard at two different dates: four weeks and one week before harvest. The results demonstrate that ByteTrack (MOTA=0.682; IDF1=0.837; HOTA=0.689) outperforms SORT and DeepSORT, indicating its superior tracking performance. Computational efficiency analysis reveals similar processing times between SORT and ByteTrack (about 15 ms), while DeepSORT requires significantly more processing time per image (128 ms). Fruit counting evaluation shows reasonably accurate yield predictions on both dates, with reduced errors and improved performance closer to the harvest date (MAPE=7.47 %; R2=0.70). The system proves effective in estimating orchard fruit production using computer vision technology, offering valuable insights for yield forecasting. These findings contribute to optimizing fruit production and supporting precision agriculture practices. The code and the dataset have been made publicly available and a video visualization of results is accessible at http://www.grap.udl.cat/en/publications/video_fruit_counting., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s predoctoral fellowship (2020 FI_B 00586)., Peer Reviewed, Postprint (author's final draft)
- Published
- 2023