Search

Cutting-edge detection is a critical step in mechanized rice harvesting. Through visual cutting-edge detection, an algorithm can sense in real-time whether the rice harvesting process is along the cutting-edge, reducing loss and improving the efficiency of mechanized harvest. Although convolutional neural network-based models, which have strong local feature acquisition ability, have been widely used in rice production, these models involve large receptive fields only in the deep network. Besides, a self-attention-based Transformer can effectively provide global features to complement the disadvantages of CNNs. Hence, to quickly and accurately complete the task of cutting-edge detection in a complex rice harvesting environment, this article develops a Transformer Convolution Network (TCNet). This cutting-edge detection algorithm combines the Transformer with a CNN. Specifically, the Transformer realizes a patch embedding through a 3 × 3 convolution, and the output is employed as the input of the Transformer module. Additionally, the multi-head attention in the Transformer module undergoes dimensionality reduction to reduce overall network computation. In the Feed-forward network, a 7 × 7 convolution operation is used to realize the position-coding of different patches. Moreover, CNN uses depth-separable convolutions to extract local features from the images. The global features extracted by the Transformer and the local features extracted by the CNN are integrated into the fusion module. The test results demonstrated that TCNet could segment 97.88% of the Intersection over Union and 98.95% of the Accuracy in the unharvested region, and the number of parameters is only 10.796M. Cutting-edge detection is better than common lightweight backbone networks, achieving the detection effect of deep convolutional networks (ResNet-50) with fewer parameters. The proposed TCNet shows the advantages of a Transformer combined with a CNN and provides real-time and reliable reference information for the subsequent operation of rice harvesting. [ABSTRACT FROM AUTHOR]

Rice lodging is a crucial problem in rice production. Lodging during growing and harvesting periods can decrease rice yields. Practical lodging judgment for rice can provide effective reference information for yield prediction and harvesting. This article proposes a binocular camera-based lodging judgment method for rice in real-time. As a first step, the binocular camera and Inertial Measurement Unit (IMU) were calibrated. Secondly, Census and Grayscale Level cost features are constructed for stereo matching of left and right images. The Cross-Matching Cost Aggregation method is improved to compute the aggregation space in the LAB color space. Then, the Winner-Takes-All algorithm is applied to determine the optimal disparity for each pixel. A disparity map is constructed, and Multi-Step Disparity Refinement is applied to the disparity map to generate the final one. Finally, coordinate transformation obtains 3D world coordinates corresponding to pixels. IMU calculates the real-time pose of the binocular camera. A pose transformation is applied to the 3D world coordinates of the rice to obtain its 3D world coordinates in the horizontal state of the camera (pitch and roll angles are equal to 0). Based on the distance between the rice and the camera level, thresholding was used to determine whether the region to be detected belonged to lodging rice. The disparity map effect of the proposed matching algorithm was tested on the Middlebury Benchmark v3 dataset. The results show that the proposed algorithm is superior to the widely used Semi-Global Block Matching (SGBM) stereo-matching algorithm. Field images of rice were analyzed for lodging judgments. After the threshold judgment, the lodging region results were accurate and could be used to judge rice lodging. By combining the algorithms with binocular cameras, the research results can provide practical technical support for yield estimation and intelligent control of rice harvesters. [ABSTRACT FROM AUTHOR]

Agricultural robotics is a complex, challenging, and exciting research topic nowadays. However, orchard environments present harsh conditions for robotics operability, such as terrain irregularities, illumination, and inaccuracies in GPS signals. To overcome these challenges, reliable landmarks must be extracted from the environment. This study addresses the challenge of accurate, low-cost, and efficient landmark identification in orchards to enable robot row-following. First, deep learning, integrated with depth information, is used for real-time trunk detection and location. The in-house dataset used to train the models includes a total of 2453 manually annotated trunks. The results show that the trunk detection achieves an overall mAP of 81.6%, an inference time of 60 ms, and a location accuracy error of 9 mm at 2.8 m. Secondly, the environmental features obtained in the first step are fed into the DWA. The DWA performs reactive obstacle avoidance while attempting to reach the row-end destination. The final solution considers the limitations of the robot's kinematics and dynamics, enabling it to maintain the row path and avoid obstacles. Simulations and field tests demonstrated that even with a certain initial deviation, the robot could automatically adjust its position and drive through the rows in the real orchard. [ABSTRACT FROM AUTHOR]