Search

Green fruit detection is of great significance for estimating orchard yield and the allocation of water and fertilizer. However, due to the similar colors of green fruit and the background of images, the complexity of backgrounds and the difficulty in collecting green fruit datasets, there is currently no accurate and convenient green fruit detection method available for small datasets. The YOLO object detection model, a representative of the single-stage detection framework, has the advantages of a flexible structure, fast inference speed and excellent versatility. In this study, we proposed a model based on the improved YOLOv5 model that combined data augmentation methods to detect green fruit in a small dataset with a background of similar color. In the improved YOLOv5 model (YOLOv5-AT), a Conv-AT block and SA and CA blocks were designed to construct feature information from different perspectives and improve the accuracy by conveying local key information to the deeper layer. The proposed method was applied to green oranges, green tomatoes and green persimmons, and the mAPs were higher than those of other YOLO object detection models, reaching 84.6%, 98.0% and 85.1%, respectively. Furthermore, taking green oranges as an example, a mAP of 82.2% was obtained on the basis of retaining 50% of the original dataset (163 images), which was only 2.4% lower than that obtained when using 100% of the dataset (326 images) for training. Thus, the YOLOv5-AT model combined with data augmentation methods can effectively achieve accurate detection in small green fruit datasets under a similar color background. These research results could provide supportive data for improving the efficiency of agricultural production. [ABSTRACT FROM AUTHOR]

In order to enable intelligent orchard management and the application of harvesting robots, it is necessary to improve the accuracy of computer vision technology for green fruit segmentation in complex orchard environments. However, existing segmentation algorithms are unable to generate precise fruit masks in such environments. This paper proposes a novel and efficient segmentation algorithm called Mask Positioner for accurate fruit segmentation. The Mask Positioner applies a layer-by-layer filtering approach to refine feature maps generated by the detail refinement network, resulting in a refined mask. The selected pixels are then input to the order decoder to determine their relevance to the fruit region. Finally, the determined pixels are used to generate the final mask, resulting in accurate and efficient fruit segmentation. Mask Positioner is verified by a green persimmon dataset made for the complex background. The experimental results show that the segmentation accuracy of Mask Positioner reaches 67.4%, and the detection accuracy reaches 69.1%. For small fruits, its detection and segmentation accuracy are at least 1.0 and 3.2 percentage points higher than other algorithms. Additionally, the generalization ability of the algorithm is verified using a green apple dataset. Experiments show that it does well in the green fruit segmentation.

Green fruit detection is of great significance for estimating orchard yield and the allocation of water and fertilizer. However, due to the similar colors of green fruit and the background of images, the complexity of backgrounds and the difficulty in collecting green fruit datasets, there is currently no accurate and convenient green fruit detection method available for small datasets. The YOLO object detection model, a representative of the single-stage detection framework, has the advantages of a flexible structure, fast inference speed and excellent versatility. In this study, we proposed a model based on the improved YOLOv5 model that combined data augmentation methods to detect green fruit in a small dataset with a background of similar color. In the improved YOLOv5 model (YOLOv5-AT), a Conv-AT block and SA and CA blocks were designed to construct feature information from different perspectives and improve the accuracy by conveying local key information to the deeper layer. The proposed method was applied to green oranges, green tomatoes and green persimmons, and the mAPs were higher than those of other YOLO object detection models, reaching 84.6%, 98.0% and 85.1%, respectively. Furthermore, taking green oranges as an example, a mAP of 82.2% was obtained on the basis of retaining 50% of the original dataset (163 images), which was only 2.4% lower than that obtained when using 100% of the dataset (326 images) for training. Thus, the YOLOv5-AT model combined with data augmentation methods can effectively achieve accurate detection in small green fruit datasets under a similar color background. These research results could provide supportive data for improving the efficiency of agricultural production.

• A clearly structured segmentation framework that applies an edge attention module to focus on features effectively. • New method cooperates the segmentation process of fruit images by two stages of detection followed by segmentation. • BAM module merges the box features to generate the predicted boundary classification scores and boundary locations. • EAM is applied to extract the edge features of the feature map to generate the upsampled feature maps. Accurate fruit detection and segmentation based on deep learning is the key to successful harvesting robot operations, but the complex background of orchards, light and branch shading, and fruit overlap lead to low detection and segmentation accuracy and high complexity of existing methods. To address these problems, an improved green fruit segmentation method based on FCOS is proposed in this study. Firstly, its segmentation function for green fruits is realized by adding segmentation module. Then, the FCOS head network is improved by adding the Border-attention module (BAM) to detect the boundary of green fruits with higher accuracy. In addition, the features of mask branch and edge segmentation branch are fused in the segmentation module, and the appearance commonality is learned by modeling the pairwise affinity between all pixels of the feature map using non-local affinity-parsing, and finally the segmentation prediction results are output by combining the feature map of fruit shape and appearance commonality. The experimental results show that this model achieves 81.2% segmentation accuracy on apple dataset and 77.9% segmentation accuracy on persimmon dataset with relatively low guarantee complexity, which exceeds most current segmentation models. Meanwhile, this model has high robustness and can be used for the detection and segmentation work of other green fruits and vegetables in orchards, while effectively extending the application of agricultural equipment. [ABSTRACT FROM AUTHOR]

High-quality orchard picking has become a new trend, and achieving the picking of homogeneous fruit is a huge challenge for picking robots. Based on the premise of improving picking efficiency of homo-chromatic fruit in complex environments, this paper proposes a novel homo-chromatic fruit segmentation model under Polar-Net. The model uses Densely Connected Convolutional Networks (DenseNet) as the backbone network, Feature Pyramid Network (FPN) and Cross Feature Network (CFN) to achieve feature extraction and feature discrimination for images of different scales, regions of interest are drawn with the help of Region Proposal Network (RPN), and regression is performed between the features of different layers. In the result prediction part, polar coordinate modeling is performed based on the extracted image features, and the instance segmentation problem is reduced to predict the instance contour for instance center classification and dense distance regression. Experimental results demonstrate that the method effectively improves the segmentation accuracy of homo-chromatic objects and has the characteristics of simplicity and efficiency. The new method has improved the accuracy of segmentation of homo-chromatic objects for picking robots and also provides a reference for segmentation of other fruit and vegetables. [ABSTRACT FROM AUTHOR]

Fruit and vegetable picking robots are affected by the complex orchard environment, resulting in poor recognition and segmentation of target fruits by the vision system. The orchard environment is complex and changeable. For example, the change of light intensity will lead to the unclear surface characteristics of the target fruit; the target fruits are easy to overlap with each other and blocked by branches and leaves, whic hmakes the shape of the fruits incomplete and difficult to accurately identify and segment one by one. Aiming at various difficulties in complex orchard environment, a two-stage instance segmentation method based on the optimized mask region convolutional neural network (mask RCNN) was proposed. The newmodel proposed to apply the lightweight backbone network MobileNetv3, which not only speeds up the model but also greatly improves the accuracy of the model and meets the storage resource requirements of the mobile robot. To further improve the segmentation quality of the model, the boundary patch refinement (BPR) post-processing module is added to the new model to optimize the rough mask boundaries of the model output to reduce the error pixels. The new model has a high-precision recognition rate and an efficient segmentation strategy, which improves the robustness and stability of the model. This study validates the effect of the new model using the persimmon dataset. The optimized mask RCNN achieved mean average precision (mAP) and mean average recall (mAR) of 76.3 and 81.1%, respectively, which are 3.1 and 3.7% improvement over the baseline mask RCNN, respectively. The new model is experimentally proven to bring higher accuracy and segmentation quality and can be widely deployed in smart agriculture. [ABSTRACT FROM AUTHOR]

The detection of banana fruits is an important part of intelligent management in the banana plantation. To detect the banana fruit quickly and accurately in the complex orchard environment, this article proposes a method based on the latest deep learning algorithm to detect the banana fruit. Using a monocular camera, we applied the YOLOv4 neural network algorithm to extract the deep features of banana fruits, realizing accurate detection of different banana sizes. The detection algorithm achieved a 99.29% detection rate, the average execution time was 0.171s, the shortest execution time was 0.135s, and the AP was 0.9995. Moreover, the detection results were discussed with the YOLOv3 algorithm and the machine learning algorithm. Compared with the machine learning algorithm, deep learning algorithm was superior to both detection accuracy and detection time. YOLOv4 had higher detection confidence and higher detection rate than YOLOv3. The results show that the proposed method could realize the fast detection of different varieties and different maturity in banana plantations, under different illumination and occlusion conditions, and provide information for banana picking, maturity and yield estimation.

The real-time detection of banana bunches and stalks in banana orchards is a key technology in the application of agricultural robots. The complex conditions of the orchard make accurate detection a difficult task, and the light weight of the deep learning network is an application trend. This study proposes and compares two improved YOLOv4 neural network detection models in a banana orchard. One is the YOLO-Banana detection model, which analyzes banana characteristics and network structure to prune the less important network layers; the other is the YOLO-Banana-l4 detection model, which, by adding a YOLO head layer to the pruned network structure, explores the impact of a four-scale prediction structure on the pruning network. The results show that YOLO-Banana and YOLO-Banana-l4 could reduce the network weight and shorten the detection time compared with YOLOv4. Furthermore, YOLO-Banana detection model has the best performance, with good detection accuracy for banana bunches and stalks in the natural environment. The average precision (AP) values of the YOLO-Banana detection model on banana bunches and stalks are 98.4% and 85.98%, and the mean average precision (mAP) of the detection model is 92.19%. The model weight is reduced from 244 to 137 MB, and the detection time is shortened from 44.96 to 35.33 ms. In short, the network is lightweight and has good real-time performance and application prospects in intelligent management and automatic harvesting in the banana orchard.

Early detection of green fruits from colour images is important for precise growth status monitoring and production estimation, allowing effective improvement of fruit quality and optimisation of orchard management. Due to the similar colours of the fruit skin and background, variable illumination and occlusions, algorithmic recognition of green apple targets in natural scenes is difficult. To effectively address this problem, a GrabCut model based on the visual attention mechanism was designed for preliminary fruit region extraction of the salient fruits, with a strong anti-noise advantage. For the overlapping fruit targets, the Ncut algorithm was applied to accurately segment the extracted fruits. After fruit segmentation, the three-point circle fitting method was developed to reconstruct each segmented apple target. A total of 200 green apple images were tested in this study. The improved GrabCut algorithm was compared against the standard GrabCut, GBVS (graph-based visual saliency), AIM (attention based on information maximisation), SDSR (saliency detection by self-resemblance), MR (manifold ranking), mean-shift and K-means clustering algorithms. The F 1 score of the improved GrabCut model for fruit region extraction was 94.08%, which was 1.92%, 40.62%, 46.98%, 33.58%, 13.89%, 12.89% and 33.74% higher than the scores of the other seven algorithms, respectively. Among them, the improved GrabCut model achieved 100.00% recognition of the 200 apple images. Moreover, a F 1 score of 94.12% was achieved by the proposed segmentation and recognition method, and the detection error was 7.37%. Experimental results showed that the proposed method could accurately recognise green apples under natural light conditions. • An improved GrabCut algorithm was designed to extract green apples from images. • The Ncut algorithm was employed to segment overlapping fruits. • Three-point circle fitting method was developed to recognise each segmented fruit. • The proposed algorithm could detect green apples with F 1 score of 94.12%. • The detection error of green apples in the orchard was 7.37%. [ABSTRACT FROM AUTHOR]

Fruit color and pigment content are two important traits that largely determine the quality of tomato fruits. In this study, six genetic populations, namely, P 1 , P 2 , F 1 , BC 1 , BC 2 and F 2 , were generated using tomato plants with green and purple fruits as female and male parents, respectively. The genetic developments of fruit color and pigment content were investigated through multigenerational joint analysis of quantitative traits. Results demonstrated that the optimal genetic model of fruit color between green and purple fruits was MX1-AD-ADI. The major gene heritabilities of shade values in BC 1 , BC 2 and F 2 generations were 88%, 63% and 82%, and their polygene heritabilities were 9%, 11% and 14%, respectively. The heredity of lycopene content conformed to MX2-ADI-AD model. The heritabilities of major genes in BC 1 , BC 2 and F 2 were 67%, 78% and 97%, while those of polygenes were 21%, 0% and 0%, respectively. The genetic model of chlorophyll was MX1-AD-ADI. The heritabilities of major genes in BC 1 , BC 2 and F 2 were 62%, 77% and 62%, respectively, and those of polygenes were all 0%. The inheritance of carotene was in accordance with MX2-ADI-ADI model. The heritabilities of major genes in BC 1 , BC 2 and F 2 were 80%, 47% and 57%, while those of polygenes were 0.01%, 0.02% and 14%, respectively. This study provides valuable information for fruit quality improvement in tomato breeding. [ABSTRACT FROM AUTHOR]

In order to enable intelligent orchard management and the application of harvesting robots, it is necessary to improve the accuracy of computer vision technology for green fruit segmentation in complex orchard environments. However, existing segmentation algorithms are unable to generate precise fruit masks in such environments. This paper proposes a novel and efficient segmentation algorithm called Mask Positioner for accurate fruit segmentation. The Mask Positioner applies a layer-by-layer filtering approach to refine feature maps generated by the detail refinement network, resulting in a refined mask. The selected pixels are then input to the order decoder to determine their relevance to the fruit region. Finally, the determined pixels are used to generate the final mask, resulting in accurate and efficient fruit segmentation. Mask Positioner is verified by a green persimmon dataset made for the complex background. The experimental results show that the segmentation accuracy of Mask Positioner reaches 67.4%, and the detection accuracy reaches 69.1%. For small fruits, its detection and segmentation accuracy are at least 1.0 and 3.2 percentage points higher than other algorithms. Additionally, the generalization ability of the algorithm is verified using a green apple dataset. Experiments show that it does well in the green fruit segmentation. [ABSTRACT FROM AUTHOR]

The variation in physicochemical characteristics, bioactive components and antioxidant capacity of three climacteric fruits, namely banana, mango and papaya were investigated. Correlations between ripeness and bioactive compounds as well as antioxidant capacity were also studied. From physicochemical analysis, hardness and firmness of fruit declined during ripening, which were consistent with loss in crude fiber and increase in total soluble solid. Total phenolic, vitamin C and antioxidant capacity were superior for green fruit, while opposite was observed for β-carotene content. Ripening clearly decreased total phenolic acids and total flavonoids contents determined using RP-HPLC. We found that ripening was correlated at a highly significant level with individual phenolic acids (gallic, vanillic, chlorogenic and sinapic acids) and individual flavonoid (quercetin) (p < 0.001), whereas a very significant correlation (p < 0.01) was observed with vitamin C, β-carotene and caffeic acid. However, no correlation existed between ripening and antioxidant capacity. Our findings have provided novel information about how best to select the appropriate maturity of fruits that contains the highest amount of specific bioactive markers linking to the best health benefits or functional properties. [ABSTRACT FROM AUTHOR]

High-quality orchard picking has become a new trend, and achieving the picking of homogeneous fruit is a huge challenge for picking robots. Based on the premise of improving picking efficiency of homo-chromatic fruit in complex environments, this paper proposes a novel homo-chromatic fruit segmentation model under Polar-Net. The model uses Densely Connected Convolutional Networks (DenseNet) as the backbone network, Feature Pyramid Network (FPN) and Cross Feature Network (CFN) to achieve feature extraction and feature discrimination for images of different scales, regions of interest are drawn with the help of Region Proposal Network (RPN), and regression is performed between the features of different layers. In the result prediction part, polar coordinate modeling is performed based on the extracted image features, and the instance segmentation problem is reduced to predict the instance contour for instance center classification and dense distance regression. Experimental results demonstrate that the method effectively improves the segmentation accuracy of homo-chromatic objects and has the characteristics of simplicity and efficiency. The new method has improved the accuracy of segmentation of homo-chromatic objects for picking robots and also provides a reference for segmentation of other fruit and vegetables., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Jia, Liu, Lu, Liu, Zhang and Dong.)

• The proposed model achieves 85.3% segmentation accuracy for green fruits. • FPN is replaced by RFPN, which improves the segmentation accuracy. • A double-layer CBA network is added to segment overlapping fruits more accurately. Accurate detection and segmentation of obscured green fruits is a key factor in the efficiency of the harvesting robot. In the real and complex orchard environment, there are many influencing factors such as illumination changes, occlusion of branches and leaves, weather, etc., which bring difficulties to the work of picking robots. Therefore, higher requirements are put forward for accurate detection and segmentation algorithms of machine vision. An accurate detection and segmentation method for obscured green fruit based on a fully convolutional one-stage (FCOS) object detection model is proposed. First, the feature pyramid network (FPN) in FCOS is replaced by residual feature pyramid network (RFPN), which improves the detection accuracy of green fruits with different sizes. Next, as the boundaries of fuzzy fruits are often not accurately identified, a two-layer convolutional block attention network is connected to a modified FCOS to recover the edges of incomplete green fruits. The experimental results show that the detection and segmentation accuracy of green fruit reaches 81.2% and 85.3% on Apple dataset. On the Apple-ape dataset, the detection accuracy and segmentation accuracy reached 77.2% and 79.7%, respectively. The Params of the model is 39.7 M, and the floating point operations (FLOPs) are 169.8G, which has great advantages over other ten advanced models in terms of accuracy and complexity. The model also improves the working efficiency of orchard harvesting robots and lays the foundation for the further development of intelligent agriculture. [ABSTRACT FROM AUTHOR]

Fruit and vegetable picking robots are affected by the complex orchard environment, resulting in poor recognition and segmentation of target fruits by the vision system. The orchard environment is complex and changeable. For example, the change of light intensity will lead to the unclear surface characteristics of the target fruit; the target fruits are easy to overlap with each other and blocked by branches and leaves, which makes the shape of the fruits incomplete and difficult to accurately identify and segment one by one. Aiming at various difficulties in complex orchard environment, a two-stage instance segmentation method based on the optimized mask region convolutional neural network (mask RCNN) was proposed. The new model proposed to apply the lightweight backbone network MobileNetv3, which not only speeds up the model but also greatly improves the accuracy of the model and meets the storage resource requirements of the mobile robot. To further improve the segmentation quality of the model, the boundary patch refinement (BPR) post-processing module is added to the new model to optimize the rough mask boundaries of the model output to reduce the error pixels. The new model has a high-precision recognition rate and an efficient segmentation strategy, which improves the robustness and stability of the model. This study validates the effect of the new model using the persimmon dataset. The optimized mask RCNN achieved mean average precision (mAP) and mean average recall (mAR) of 76.3 and 81.1%, respectively, which are 3.1 and 3.7% improvement over the baseline mask RCNN, respectively. The new model is experimentally proven to bring higher accuracy and segmentation quality and can be widely deployed in smart agriculture., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Jia, Wei, Zhang, Pan, Niu, Yin, Ding and Ge.)

• A detection algorithm for banana bunches and stalks in the natural environment. • High resolution input is helpful for the detection of banana bunches and stalks. • Enhanced detection based on ROI can improve the detection accuracy of stalks. • The AP of banana bunches and the stalks is 99.55% and 87.82%, respectively. With the widespread application of machine vision technology in agriculture, the intelligent management of banana orchards is urgent. Accurate detection of banana bunches and stalks is a precondition for orchard yield estimation and automatic harvesting. In a complex banana orchard environment, banana bunches and stalks are similar to the leaves in color, and banana stalks are similar to the petiole in texture, making the detection of banana bunches and stalks in banana orchards challenging. This study proposes an accurate and fast multiclass detection method for banana bunches and stalks. A regular RGB camera was used to collect images. The well-known YOLOv4 network was used to detect the banana bunches and stalks, and the input image resolution was discussed by training and comparison. The banana bunch and stalk detection model showed excellent reliability and generalization ability in different illumination and occlusion scenarios. The AP of the banana bunch and stalk detection was 99.55% and 87.82%, respectively, and the mAP of the detection model was 93.69%. The average execution time was 44.96 ms. The detection of small-sized banana bunches and stalks was discussed, and its significance in banana orchard applications was analyzed. The experimental results show that the fast real-time detection of banana bunches and stalks in the natural environment is helpful for the intelligent management of banana orchards. [ABSTRACT FROM AUTHOR]

The real-time detection of banana bunches and stalks in banana orchards is a key technology in the application of agricultural robots. The complex conditions of the orchard make accurate detection a difficult task, and the light weight of the deep learning network is an application trend. This study proposes and compares two improved YOLOv4 neural network detection models in a banana orchard. One is the YOLO-Banana detection model, which analyzes banana characteristics and network structure to prune the less important network layers; the other is the YOLO-Banana-l4 detection model, which, by adding a YOLO head layer to the pruned network structure, explores the impact of a four-scale prediction structure on the pruning network. The results show that YOLO-Banana and YOLO-Banana-l4 could reduce the network weight and shorten the detection time compared with YOLOv4. Furthermore, YOLO-Banana detection model has the best performance, with good detection accuracy for banana bunches and stalks in the natural environment. The average precision (AP) values of the YOLO-Banana detection model on banana bunches and stalks are 98.4% and 85.98%, and the mean average precision (mAP) of the detection model is 92.19%. The model weight is reduced from 244 to 137 MB, and the detection time is shortened from 44.96 to 35.33 ms. In short, the network is lightweight and has good real-time performance and application prospects in intelligent management and automatic harvesting in the banana orchard. [ABSTRACT FROM AUTHOR]

• A concise and accurate FoveaMask model is proposed for segmentation of green fruits. • A position attention module(PAM) is added into mask branch for improving robustness. • FoveaMask eliminates dependence on anchors. • It can be referred and even migrated directly to other fruit segmentation. In the process of agricultural automation production, efficient and accurate segmentation of target fruit is the basis and guarantee for numerous applications including crop growth monitoring, yield prognosis, and machine picking. Green fruit is apt to affect by complicated scenes such as occlusions and overlaps, as well as the homo-chromatic background, which brings great challenges to the recognition and segmentation. In this paper, we propose a novel methodology named FoveaMask for improving the robustness and generalization of green fruit segmentation framework. Features of input images are firstly extracted by ResNet and fused by FPN. The classification and bounding-box regression of each spatial position on the feature maps are then carried out directly by the way of full convolution. RoI Align layer is applied to fix feature region of proposals to the same size yet preserve exact spatial locations. Finally, instance-level fruit segmentation is realized by pixel-level classification on each proposal using embedded mask branches. The whole network architecture of the new model does not involve the related design and operation of anchor, which greatly improves the generalization ability for different shape fruits, alleviates the computing and storage resources, and balances the contradiction between accuracy and efficiency simultaneously. Additionally, a position attention module(PAM) is introduced into the embedding mask branch to aggregate the effective information pixels, which can improve the robustness of the segmentation model in the actual complex environment of the orchard. We test the model on green apple and immature persimmon data sets and the experimental results show that FoveaMask performs best in terms of both recognition accuracy and model complexity compared with other 11 different types of detection and segmentation models. [ABSTRACT FROM AUTHOR]

• Detection algorithm based on machine vision for banana in the natural environment. • Coarse detection stage could improve detection efficiency and save detection time. • Two texture features with SVM classifier realized the fine detection of the banana. • The average single-scale detection rate based on the proposed algorithm was 89.63%. • The multi-scale detection method was discussed to improve the detection accuracy. Banana detection by picking robots in outdoor conditions is difficult due to the color similarity with leaves and stems. A method of banana detection in the natural environment based on color and texture features was performed in this study by using a regular red–green–blue color camera. First, part of the background was removed in HSV color space by analyzing the relationship between the S color component and V color component; this saved detection time and improved the detection efficiency. Then, the banana area was found by adopting support vector machine with local binary pattern features and histogram of oriented gradient features of the banana. Single-feature and multi-feature fusion with different classifiers were compared to find the most suitable classification algorithm for banana detection. A validation set containing 4400 samples was used to evaluate the proposed classification algorithm. The precision and recall of banana detection were 100%. A total of 120 photos under different illumination conditions were selected as the test set. The average single-scale detection rate based on the proposed algorithm was 89.63%, the average execution time was 1.325 s, and the shortest execution time was 0.343 s. At last, the multi-scale detection method based on the proposed algorithm was discussed to improve the detection accuracy. The results showed that the developed method can be applied to the detection of banana in plantations under different illumination and occlusion conditions. [ABSTRACT FROM AUTHOR]