Search

In recent years, with the rapid development and expansion of the global aquaculture industry, and the continuous enlargement of aquaculture farms, the industrialization, intelligence, and informatization of aquaculture have become a trend in the industry. China has become the largest producer of fisheries and aquaculture. Fish farming is an important component of aquaculture, and fish farming monitoring has become an important technology to enhance the efficiency, production, and management of fish farming. Fish farming monitoring can provide real-time and accurate data for farms, assisting farm managers in making decisions to improve efficiency and production. With the emergence of artificial intelligence technology in recent years, deep learning has rapidly developed and been widely applied in various fields such as image and audio recognition, natural language processing, robotics, bioinformatics, chemistry, and finance. The monitoring of fish farming focuses on the quantity, growth, behavior, and health status of fish. Using deep learning technology, we can quickly and accurately obtain information related to fish farming and enhance its efficiency and management. This paper presents a deep learning-based method for fish farming monitoring and reviews the literature progress in fish length measurement, fish counting, fish feeding, fish swimming behavior, and fish disease diagnosis. Although deep learning-based fish length measurement has achieved high accuracy in underwater environments, some errors still exist. The counting methods based on deep learning can be categorized into segmentation counting, detection counting, tracking counting, and density regression counting. Deep learning models based on video data have higher accuracy in recognizing fish feeding behavior than imagebased models. There have been many studies on fish tracking, but practical applications still face challenges such as fish feature extraction, the influence of fish size and obstructions, and occlusion issues. In fish disease diagnosis, it is necessary to establish standardized and shared fish disease datasets and utilize data fusion, data level information fusion, feature level information fusion, and decision level information fusion. This article also summarizes the main problems of deep learning-based visual technologies in fish farming monitoring from the aspects of monitoring data acquisition and transmission, dataset standardization and processing, deep learning model design, and the lack of business application in fish farming intelligent monitoring. The problems in data acquisition include a limited variety of experimental subjects, a small number of samples, and poor performance of experimental equipment. In the data transmission process, there are challenges in data security and realtime transmission. In terms of datasets, there is a low level of standardization and a lack of large-scale unified datasets. There is also a lack of research on large models and embedded models in deep learning model design. Furthermore, there is a realistic problem of insufficient business application in practical settings. The paper also proposes future research directions, including establishing fish farming monitoring datasets, super-scale parameter models for fish farming, edge computing for terminal monitoring devices, and digital twinning in fish farming monitoring, aiming to provide scientific references for the widespread application of deep learning in fish farming monitoring. [ABSTRACT FROM AUTHOR]

Rice shrimp co-cultivation, as a circular agriculture model that integrates ecological, economic, and social benefits, has been widely applied in China. Accurately obtaining information on the planting area, spatial distribution, and changes of rice shrimp co-cultivation is of great significance for ensuring safe production, social supply of rice and shrimp, and formulating corresponding management and planning policies by government departments. Most of the existing methods for extracting rice shrimp farming fields used medium-resolution temporal images as data sources to analyze the characteristics of water or vegetation changes within a year to construct models. The accuracy of the extraction depended on the results of water or vegetation analysis, the boundaries were unclear and there was a lot of noise. To solve the problems of complex temporal analysis, low accuracy of extraction results, and incomplete boundaries in existing extraction methods, this paper used high-resolution images of a single temporal phase as the data source, without analyzing the temporal features of rice shrimp farming fields. Only the spatial features of rice shrimp farming fields on high-resolution images were used for high-precision extraction. We proposed a deep learning semantic segmentation model named EASNet (edge assisted segmentation network), which was mainly composed of three modules: feature extraction module (FE Module), edge assist module (EA Module), and information fusion Module (IF Module). The FE Module was used to extract multi-level and fused contextual semantic information features of the target object. The EA module used the unique edge "shrimp groove" of rice and shrimp farming fields as an auxiliary information for separate segmentation in the designed EA Module. The IF Module integrated the outputs of the FE Module and the EA Module, enabling the main task to not only enhanced the boundary structure information of rice shrimp farming fields but also learned the unique spatial and semantic information of rice shrimp farming fields. The experimental results showed that with the enhancement of the EA Module, the segmentation results of rice shrimp farming fields were more complete and the boundaries were clearer. The IoU (Intersection over Union) of semantic accuracy and the F1-Score of boundary accuracy were improved by 1.5% and 5.8%, respectively. The recall, IoU, and F1-Score of overall semantic accuracy reached 0.970, 0.964, and 0.930, respectively. The Recall and F1-Score of boundary accuracy reached 0.864 and 0.859, while the Recall and F1-Score of relaxed boundary accuracy reached 0.876 and 0.913. The trained EASNet model was applied to the whole area of Xuyi County, and the spatial distribution map of rice shrimp farming fields in Xuyi County in 2020 was obtained. In comparison with the results of rice shrimp breeding fields extracted by the traditional Water Seasonal Difference method and the Random Forest method, our method obtained the optimal results with an OA(overall accuracy) of 96.71% and a Kappa coefficient of 0.934. The method used in this paper has higher accuracy, lower missing rate, more regular boundaries, and fewer instances of similar integration in extracting results. It can provide a basis for natural resource surveys, and government departments to formulate corresponding rice shrimp breeding management and planning policies. [ABSTRACT FROM AUTHOR]

Hyperspectral technology has been widely used in many fields due to its excellent performance characteristics such as high sensitivity, fine resolution, multi-channel data acquisition and processing capabilities, and non-destructive testing. However, despite its outstanding performance in data acquisition and information extraction, it also faces the challenges of high requirements and costs of hardware equipment. Hardware equipment such as high-precision optics, high-quality detectors, and complex data acquisition devices required for hyperspectral imaging have become one of the key factors limiting its popularity and application. To overcome this dilemma, researchers have begun to explore the reconstruction of RGB images into hyperspectral images by extracting information from traditional RGB images through spectral reconstruction techniques at relatively low cost. This innovative approach significantly reduces the complexity and cost of spectral data acquisition and provides a brand new way for wider application of hyperspectral technology. In the study of this paper, we use surimi adulteration detection as an example to explore the performance of different spectral reconstruction algorithms in depth. By mixing the dorsal muscle of silver carp with commercially available starch in different ratios and simulating the real state of surimi when it is sold, we successfully acquired RGB images and corresponding spectral data of a series of samples. In order to ensure the scientific validity and reliability of the experiment, we adopted the spectral-physical-chemical value covariance distance method (SPXY) for the sample set and divided the data set into calibration and validation sets according to the ratio of 5:3. In the process of spectral reconstruction, we explored two mainstream methods, the multivariate polynomial least squares regression algorithm (PMLR) and the deep learning hierarchical regression network (HRNet). Comparative experimental results show that the HRNet-based reconstruction technique has relative and root-mean-square errors of 0.010 4 between the spectra and the actual spectra, respectively, while the PMLR-based algorithm corresponds to relative and root-mean-square errors of 0.012 6. These results indicate that both methods are within acceptable error ranges, which provides a subsequent experimental results provide a solid foundation. In order to verify the reliability of the reconstructed spectra in practical applications, we established models of extreme learning machine regression (ELMR), support vector machine regression (SVR), and partial least squares regression (PLSR) based on the reconstructed spectra, and combined S-G convolutional smoothing (SG), mean centering (MC), standard normal variable transformation (SNV), derivative smoothing (DE), and normalization (NOR) methods were used to preprocess the spectra with a view to improving the model performance. By using corrected correlation coefficient (RC) corrected root mean square error (RMSEC) predicted correlation coefficient (RP) and root mean square error of prediction (RMSEP ) as evaluation metrics, we successfully detected the adulteration ratio of adulterated surimi. The experimental results show that the support vector machine regression model with standard normal transform (SNV) preprocessing achieves the best results with a prediction correlation coefficient (RP) of 0.983 0 and a root mean square error of prediction (RMSEP ) of 3.954 4% in the model based on the reconstruction of spectra by the PMLR algorithm. In the model based on the reconstructed spectrum of deep learning HRNet network, the support vector machine regression model, also using SNV preprocessing, achieved the best performance, with a prediction correlation coefficient of 0.998 7 and a prediction root-mean-square error of 4.080 8%. Therefore, the PMLR algorithm and HRNet network reconstruction technique based on RGB images not only realized efficient spectral reconstruction, but also provided a reliable solution for surimi adulteration detection. This research result provides new ideas and methods for quality and safety detection of food and agricultural products using low-cost hardware devices. [ABSTRACT FROM AUTHOR]

It is widely recognized that the value of rice in the market is highly depends on its variety, with some varieties commanding significantly higher prices than others. Unfortunately, this has led to a situation where unscrupulous vendors may attempt to pass off inferior or lower-value rice varieties as more valuable ones. This urgently requires a reliable and efficient methods for accurately detecting and identifying different rice varieties. The use of near-infrared spectroscopy and deep learning for rice variety detection is an important research direction in this field. The establishment of an accurate detection model depends on a large-scale sample data, however, the process of collecting and preprocessing samples is very timeconsuming, which limits the improvement of accuracy. To address the aforementioned needs and limitations, this paper proposes a rice variety detection method based on near-infrared spectroscopy combined with an improved deep convolutional generative adversarial network (DCGAN) for data augmentation. Based on the classical deep convolution generative adversarial neural network, the network increases the number of convolution layers and adjusts the network structure. Subsequent experiments showed that the network has better generation ability. In this study, we selected four rice varieties, namely Wuchang, Xiangshui, Yinshui, and Yueguang, and collected 80 samples for each variety, amounting to a total of 320 rice powder samples as experimental samples. Firstly, we collected near-infrared spectra of the samples and preprocessed the raw spectral data by applying detrend correction (DC) and uninformative variable elimination (UVE) algorithms to eliminate irrelevant spectral features. Then, we established an improved DCGAN model to augment the preprocessed spectral data. Three traditional data enhancement methods, random left and right translation, superimposed Gaussian noise and translation noise, were used for comparison. Comparative experiments demonstrated that the spectral data generated by the improved DCGAN showed significant superiority in terms of structural similarity compared to traditional data augmentation methods. Finally, we investigated different data augmentation methods combined with different classification methods to establish a rice variety classification model. We employed five classifiers, namely partial least squares discriminant analysis (PLS-DA), k-nearest neighbors (KNN), random forest (RF), decision tree (DT), and one-dimensional convolutional neural network (1D-CNN), and combined them with four data augmentation methods to build the rice variety classification model. The results showed that the model built using DCGAN-augmented data achieved the highest accuracy on the test set, the accuracy rate was 98.21%. Through quantitative analysis of the models built using DCGAN combined with the five classifiers, it was found that the variety detection model established by DCGAN-1D-CNN method performed the best on the test set, with a coefficient of determination R 2 of 0.992 2, mean absolute error (MAE) of 0.008 9, and root mean square error (RMSE) of 0.050 5, indicating its optimal performance. Based on the comprehensive experimental results, the model established by combining the improved DCGAN data augmentation with the 1D-CNN classification algorithm exhibited the best performance in terms of accuracy on the test set, providing a new approach for convenient and accurate rice variety detection applications. By utilizing near-infrared spectroscopy and deep learning methods, combined with the improved DCGAN for data augmentation, we can enhance the accuracy and reliability of rice variety detection, thus meeting the market demand and consumer expectations. [ABSTRACT FROM AUTHOR]

Abstract: Accurate identification of cotton top bud is important for cotton topping operation to detect cotton terminal bud accurately in field, a recognition method using Faster R-CNN (Faster Region Convolutional Neural Networks, Faster R-CNN) by integrating dynamic mechanism was proposed to solve the recognition difficulties caused by the small size of cotton terminal bud when it is topped in densely planted fields. The RegNetX-6.4GF model was used as the backbone network to improve its image feature extractor capabilities. Due to number of proposals under a higher IOU(Intersection Over Union, IOU) as well as the matching degree between anchor and the target shape affect the performance of the detector, the method proposed in this paper changed the original anchor generation mechanism by combining FPN (Feature Pyramid Network, FPN) and Guided Anchoring in RPN (Region Proposal Network, RPN), which will cause the distribution of the proposals generated by the RPN of the algorithm under different IOUs dynamically change during the training process. To adapt the dynamic change of proposals distribution, we integrated Dynamic Region Convolutional Neural Networks ( Dynamic R-CNN) in Faster R-CNN, which can dynamically adjust the value of IOU to obtain high quality proposals. And the GROIE (Generic ROI Extractor, GROIE) mechanism was inducted to extract ROI (Region of Interest, ROI) to improve the feature fusion capability. In this paper, 4 819 images of gossypium hirsutum population which contain seven leaf types were taken from the top of cotton plant at distance of 30-50 cm (medium distance) and 50-100 cm (long distance) under uniform light, oblique strong light, direct strong light, and shadows. Those images were processed as MS COCO 2017 format dataset and assigned to the training set, validation set, and test set, which contained 2 815, 704, and 1 300 pictures respectively. The experimental results demonstrated that FPS (Frames Per Second, FPS) of proposed model was up to 10.3 frames/s and the Mean Average Precision (MAP) of bud identification reached to 98.1% which was 3.2 percentage points higher than original Faster R-CNN model. The validation set were used to compare performance of mainstream recognition algorithm and proposed method. Results showed that the improved Faster R-CNN's MAP was 7.3% higher than original Faster R-CNN, which was also higher than RetinaNet, Cascade R-CNN (Cascade Region Convolutional Neural Networks, Cascade R-CNN) and RepPoints by 78.9%, 10.1% and 8.3% when IOU was set to 0.5. The improved Faster R-CNN proposed in this paper meets the accuracy and real-time requirements of cotton topping operation. [ABSTRACT FROM AUTHOR]

China is the major producer and consumer of pork all around the world. In China, pork production has accounted for more than 60% of meat production for a long time. Nowadays, the pig industry has gradually shifted from decentralized and extensive breeding to intensive large-scale breeding, promoting production efficiency. At present, as people's demand for the quantity and quality of pork is increasing, China's pig industry faces the problem that the output and quality of pork can not meet people's daily needs. With the rise of artificial intelligence technology in recent years, deep learning technology has been developed rapidly and widely used in image and audio recognition, natural language processing, robotics, bioinformatics, chemistry, finance, and other fields. It is also an essential tool in developing precision livestock farming. The body condition, behavior and health status of pigs could directly affect the income level of pig farms, so through using deep learning technology, we can quickly and accurately acquire the relevant information about pigs, and carry out precise management to improve the feeding efficiency and welfare levels of pigs. This paper expounds on the research progress and application status of deep learning technology used in target pig detection, pig image segmentation, pig body condition and abnormal monitoring, as well as pig behavior recognition. Then we put forward the improvement strategy of deep learning technology used in the process of pig feeding, which make it easier for researchers to understand. At the same time, we summarize and analyze the data sources and datasets, application scope, and model optimization of previous works using deep learning technology in pig breeding. In the field of data sources and datasets, mobile computer vision systems are currently more suitable than systems with many fixed cameras when they are applied in pig houses, therefore, further research could focus on how to use deep learning technology to mobile computer vision systems. Deep learning technology requires a large amount of data to learn data features, which is a massive disadvantage for pig applications. To get pre-training weights suitable for agricultural data sets, a large number of public datasets and a unified dataset standard suited to the pig field should be established. In terms of application scope, due to the short application time of deep learning in the process of raising pigs, many critical occasions are not involved or seldom involved in existing research, so the application scope of deep learning should further expand. In model optimization, optimizing the model to meet the needs of practical application scenarios is the direction of future research. Optimizing the model to achieve a better balance between size and model performance requires further study. Optimizing the model to locate the start and end time of the behavior in unclipped video and recognize the behavior of target pigs is the challenge in identifying the behavior of pigs in future. China's research on deep learning is at the top level, so the application prospect of deep learning in pig farming is highly expected. Combined with the actual production scene of pigs, deep learning technology will significantly contribute to the development of precision livestock farming. [ABSTRACT FROM AUTHOR]

The existing pig counting system based on image processing are seriously affected by light conditions, and the counting accuracy is poor when pigs are crowded and obscured.To realize the intelligent pig counting, the paper proposed a counting scheme based on the improved instance segmentation algorithm was proposed in this paper. Aiming at the problems of image illumination and target edge blur, Laplace operator was used to preprocess the images. The feature extraction network of MASK R-CNN network was improved by using Resnet-152 as the Mask R-CNN feature extraction network, and the original Feature Pyramid Network(FPN) was followed by a bottom-up enhancement path, which directly fused the low-level edge features with the high-level features to improve the recognition ability of the target edge bluring. The non maximum suppression process and loss function were optimized and improved to improve the segmentation accuracy. The experiments were carried out at three different real pig farms to verify the counting accuracy, respectively in Hebei province, Jilin province and Inner Mongolia. The size of the cage in Hebei pig farm was 5.5 m × 1.8 m, with an average of 12 pigs in a single pen, with a feeding density of 1.21 pigs/m²; the size of the cage in Jilin pig farm was 5.5 m × 3.9 m, with an average of 22 pigs in a single pen, and the average rearing density was 1.03 pigs/m²; the size of the cage in Inner Mongolia pig farm was 11.4 m × 5.28 m, with an average of 80 pigs in a single pen, the average feeding density was 1.32 pigs/m². The RGB camera is was installed on the top of the pen and acquired the image in daytime. 2 400 images were collected in total, and 2000 images were selected after image preprocessing, and 1 250 images of three pig farms were selected as the original data set according to the ratio of 2:2:1. The training set and verification set were enhanced to 1 500 and 150 images and 250 images for the test set, The experimental results showed that 98 images could realize exact counting and 2 images missed 1 pig in the Jilin pig farm, the accuracy of pig counting was 98%. In Hebei pig farm, 99 images could realize the exact counting and the accuracy of pig counting was 99%. For Inner Mongolia pig farms with high feeding density, the accuracy of pig counting was 86%, among the 50 test images, 7 images missed detection, 4 images missed 1 target, 3 images missed 2 targets, The results can provide the application of the artificial intelligent in agriculture field. [ABSTRACT FROM AUTHOR]

Manually classifying citrus based on its surface defects is tedious and time-consuming and a new real-time method is proposed in this paper based on the improved SSD deep learning model. In the testing bench of the waxing machine, 2 500 images of a variety of citrus species were taken, of which 2 000 were randomly selected as training set and 500 as testing set. Among them, the method classified 19 507 as normal, 9 097 skin defects and 4 327 mechanically damaged. Considering that traditional methods using near-infrared spectra, support vector machines, HSV and RGB color space model are inefficient to detect surface defects of citrus and can only identify one, we proposed an improved method to calculate the image using the one-stage detection model - SSD-ResNet18. The method gets the feature maps through backbone first, and then predicts the number of boundary boxes from the feature maps before determining the location and category of citrus using confidence and non-maximum suppression. This can detect a batch of citrus. In the proposed method, we used the mAP (mean average precision) as the precision index and the mean detection time as the speed index. Optimization in the proposed method was solved using the SGD (stochastic gradient descent) algorithm. The learning scheduler was based on cosine decay, enabling the learning rate to drop to 0 at the end of the training period. This ensures the lost value during the training period to continuously decline. As the model was stable at the end of the training period, it can be saved at the end of the training for further use. While the VGG16 was used as the original SSD backbone, it needs a multitude of parameters and is hence computationally inefficient. We replaced it with the ResNet18, which is approximately 100 times more efficient than the VGG16. An improved feature map was obtained from the analysis of the effective sensory field of different feature maps and the size of citrus in the map, the anchor in which was obtained using the K-means clustering algorithm from the manual label box. The suitable image resolution for the proposed model was obtained by comparing images taken at five resolutions: 512×512 pixels, 640×640 pixels, 768×768 pixels, 896×896 pixels and 1024×1024 pixels. The results showed that the accuracy of the mAP of SSD-ResNet18 was 87.89%, improving 0.34 percentage points higher than the original SSD. The average detecting time of the SSD-ResNet18 was 20.72 ms, reduced by 436.90% compared to the original SSD's 108.83 ms. The accuracy of the AP of SSD-ResNet18 was 94.72%, 85.79% and 83.17%, respectively, for detecting normal, skin lesion and mechanical damage. We compared MobileNetV3, ESPNetV2, VoVNet39 and ResNet18 as backbones and did not find significant difference between their accuracy, but ResNet18 was 10.52 ms, 16.78 ms and 36.76 ms less than MobileNetV3, ESPNetV2 and VoVNet39 in detection time, respectively. The method proposed in the paper meets the requirement on detecting speed in real-time citrus production line and can effectively classify and detect a multitude of citrus simultaneously. [ABSTRACT FROM AUTHOR]

Depth estimation is critical to 3D reconstruction and object location in intelligent agricultural machinery vision system, and a common method in it is stereo matching. Traditional stereo matching method used low-quality image extracted manually. Because the color and texture in the image of field plant is nonuniform, the artificial features in the image are poorly distinguishable and mismatching could occur as a result. This would compromise the accuracy of the depth of the map. While the supervised learning-based convolution neural network (CNN) is able to estimate the depth of each pixel in plant image directly, it is expensive to annotate the depth data. In this paper, we present a depth estimation model based on the self-supervised learning to phenotype tomato canopy. The tasks of the depth estimation method were to reconstruct the image. The dense disparity maps were estimated indirectly using the rectified stereo pair of images as the network input, from which a bilinear interpolation was used to sample the input images to reconstruct the warping images. We developed three channel wise group convolutional (CWGC) modules, including the dimension invariable convolution module, the down-sampling convolution module and the up-sampling convolution module, and used them to construct the convolutional auto-encoder - a key infrastructure in the depth estimation method. Considering the shortage of manual features for comparing image similarity, we used the loss in image convolutional feature similarity as one objective of the network training. A CWGC-based CNN classification network (CWGCNet) was developed to extract the low-level features automatically. In addition to the loss in image convolutional feature similarity, we also considered the whole training loss, which include the image appearance matching loss, disparity smoothness loss and left-right disparity consistency loss. A stereo pair of images of tomato was sampled using a binocular camera in a greenhouse. After epipolar rectification, the pair of images was constructed for training and testing of the depth estimation model. Using the Microsoft Cognitive Toolkit (CNTK), the CWGCNet and the depth estimation network of the tomato images were calculated using Python. Both training and testing experiments were conducted in a computer with a Tesla K40c GPU (graphics processing unit). The results showed that the shallow convolutional layer of the CWGCNet successfully extracted the low-level multiformity image features to calculate the loss in image convolutional feature similarity. The convolutional auto-encoder developed in this paper was able to significantly improve the disparity map estimated by the depth estimation model. The loss function in image convolutional feature similarity had a remarkable effect on accuracy of the image depth. The accuracy of the disparity map estimated by the model increased with the number of convolution modules for calculating the loss in convolutional feature similarity. When sampled within 9.0 m, the root means square error (RMSE) and the mean absolute error (MAE) of the corner distance estimated by the model were less than 2.5 cm and 1.8 cm, respectively, while when sampled within 3.0m, the associated errors were less than 0.7cm and 0.5cm, respectively. The coefficient of determination (R²) of the proposed model was 0.8081, and the test speed was 28 fps (frames per second). Compared with the existing models, the proposed model reduced the RMSE and MAE by 33.1% and 35.6% respectively, while increased calculation speed by 52.2%. [ABSTRACT FROM AUTHOR]

Pigs like to fight with each other to form a hierarchy relationship in groups. Aggressive behaviors, mostly fighting, are frequently found in intensive pig raising facilities. Strong aggressive behaviors can cause other pigs lack of food and water, growing slowly, wounds, sick and even dead in serious situation. This considerably reduces health and welfare of pigs and further decreases economic benefits of pig industries. Monitoring and recognizing aggressive behaviors among pig group is the first step to manage the aggressive behaviors in group pigs effectively. Traditional human recording method is time-consuming and labor-intensive. This method can’t be used 24 hours a day, 7 days a week. Machine vision technique brings an automatic monitoring method to solve this problem. In this paper, we introduced a new method for aggressive behaviors monitoring based on deep learning. The experiments were held under controlled environments, which were achieved in an environment-controlled chamber designed previously. The details of the chamber were depicted in a published paper written by our research group. Nursery pigs were fed under three different concentration levels of NH3 gas, which were <3.80 mg/m³, 15.18 mg/m³, 37.95 mg/m³, with a suitable temperature of around 27 ℃ and the comfortable humidity between 50%-70%. Each nursery group had six pigs and were weight around 9.6 kg. During each 28 days’ experiment of three concentration levels of NH3, videos were taken from the top of the chamber. An end-to-end network, named 3D CONVNet, was proposed for aggressive behavior recognition of group pigs in this paper, which based on a C3D network and built with 3D convolution kernels. The network structure of the 3D CONVNet was improved in both width and depth dimensions. The number of main convolutional layers was increased to 19, extra batch normalization and dropout layers were added to deepen the network. Furthermore, the multi-scale feature fusion method was introduced to widen the network. This improvement had bettered the performance of the algorithm considerably. To train the 3D CONVNet, 380 aggressive (14 074 frames) and 360 none-aggressive videos (13 040 frames) were chosen from experimental videos recording in experiments of two concertation levels. These videos were randomly divided into training set and validation set, and the ratio of each set is 3:1. Another 556 aggressive videos and 510 none-aggressive videos from the three experimental batches were chosen to build the testing set. There was no overlap among training set, validation set, and testing set. Results showed a total of 981 videos, including aggressive and non-aggressive behaviors, was correctly recognized from the whole 1066 testing videos. The precision of the 3D CONVNet was proved to be 92.03% on testing set. Among them, the precision, recall rate and F1-Score for aggressive behaviors were 94.86%, 89.57%, and 92.14%, respectively. The precision for different NH3 concentration experimental levels were 94.29%, 89.44%, and 85.91%, respectively, which showed the generalization performance of the 3D CONVNet. With the similar heat environments, the 3D CONVNet also showed the good performances under different illumination condition. The comparison with C3D，C3D_1 (19 layers) and C3D_2 (BN) networks resulted in 95.7% on validation set, 43.27 percent higher than the C3D network. The recognition on single image using the 3D CONVNet was only 0.5 s, which was much faster than the other three networks. Therefore, the 3D CONVNet was effective and robust in aggressive behavior recognition among group pigs. The algorithm provides a new method and technique for aggressive behavior auto-monitoring of group pigs and helps improve establishment of auto-monitoring system in pig farms and manage level of pig industry. [ABSTRACT FROM AUTHOR]

Mechanical weeding can reduce the use of pesticides and is of great significance to ensure high yield of crops. Real-time and accurate identification of crops is a key technical problem needs to be solved in mechanical weeding equipment. Because of the subjectivity of feature extraction process in weed recognition, the accuracy of traditional methods in actual field environment is low. In recent years, the method of weed identification based on convolution neural network has been widely studied. Although the accuracy is obviously improved, there are still problems such as large parameters and poor real-time performance. In order to solve the above problems, a four-channel input image is constructed by collecting near infrared and visible images of sugar beet in the field, and a lightweight convolution neural network based on codec structure is proposed. In this paper, Sugarbeet and weed images collected from a farm in Bonn, Germany, in 2016 were used as data sets, which covered images of different growth stages of sugar beet, and 226 pictures of which were randomly selected as training sets, and the remaining 57 pictures were used as test sets. The experimental data set was composed of three channels of visible light image and one channel of near infrared image, which are merged into a four-channel image by pixel level superposition, and the depth-wise separable convolution was used in the deep model. Firstly, the input feature image was convoluted in 2 dimensions convolution kernel and the number of channels was expanded. Then, the 1×1 convolution kernel was used to make the 3 dimensions convolution which combined channel feature and compressed the channels to enhance the nonlinear mapping ability of the model. In order to avoid the problem of the gradient disappearing, the residual block was used to connect the input and output of the depth-wise separate convolution. Finally, the coding and decoder structure was designed and the shallow features were combined with deep features to refine the segmentation effect. Due to the imbalance of pixel proportions of soil, crops and weeds, the weighted loss function was used to optimize the model. The segmentation accuracy, parameters and operating efficiency of models at different input resolutions and different width factor were introduced to evalute the model. When the width factor was 1, the segmentation accuracy of the model increased with the increase of the input image resolution, the model accuracy of four channel input was higher than that of the model based on original visible image input, which showed that the near-infrared image features can compensate the defects of ordinary RGB images to some extent, and make the model more suitable for the dark environment. Under the same input image resolution, the model with a width factor of 2 or 4 performs better than the model with a width factor of 1. With the increases of width factor, the parameters of the model increase greatly. The amount of calculation is related to the size of the input image, so the frame rate gradually decreases with the increase of the size of input image. The experimental results show that the optimal model in this paper is a four channel input model with a width coefficient of 2, and the average intersection union ration is 87.58%, the average pixel accuracy is 99.19%, the parameters are 525 763 and the frame rate is 42.064 frames/s. The model has high segmentation and recognition accuracy and good real-time performance, and can provide theoretical basis for the development of intelligent mechanization weeding equipment. [ABSTRACT FROM AUTHOR]

Object detection for seafood is a key to intelligence marine ranching, but its real-time and accuracy need to be improved. In this paper we present an YOLOv3-based real-time object detection method for seafood. Firstly, for better real-time performance, we replace the 3×3 standard convolution in YOLOv3 with the depthwise separable convolution technology. This technology resolves standard convolution into depthwise convolution and pointwise convolution. It greatly reduces the amount of parameters and calculations of the model. Thus, we obtain a lightweight network model, named Depthwise Separable Convolution-YOLO (DSC-YOLO); Then, in the data preprocessing stage, we introduce UGAN-based image enhancement methods to relief the color distortion of underwater seafood images. This method uses an unsupervised method to automatically learn the distribution of training data and generate new data. The network consists of generator network and discriminators network. Its main idea is confronting and gaming between networks, on the basis of original data information to generate a new data information. It realizes the mapping process from underwater blurred image to underwater clear image. To be specific, we use CycleGAN to convert images in air to underwater-like images. Thus, lots of image pairs can be obtained and used to train UGAN model. Once UGAN finish training, learned generator of UGAN can transform real underwater images to clear images. Finally, we introduce Mosaic-based data augment methods to enrich image diversity. To be specific, four images are randomly selected from underwater dataset images, and operated with traditional data augmentation (rotation, translation, scaling, roll). Then, the four images are randomly cropped and spliced, obtaining a Mosaic image. The new image has the same dimension as the original images. Combining the three strategies, we obtain a real-time object detection method for seafood. In order to verify the effectiveness of our proposed method, we conduct several experiments. In these experiments, DSC-YOLO, UGAN underwater image enhancement and Mosaic data enhancement were combined to obtain four models: DSC-YOLO, DSC-YOLO+UGAN, DSC-YOLO+Mosaic, DSC-YOLO+UGAN+Mosaic. The four models compare with the YOLOv3 model over the same dataset. The results of experiments show that compared with YOLOv3 model, the model size of the proposed method reduces by 70%, the inference time reduces by 16%, the Recall (R) improves by 2.7%, the mean Average Precision 0.5 (mAP0.5) promotes by 2.4%, and the F1-Score increases by 0.4%. The results indicate that the proposed method has the potential to meet real-time requirements and deploy on mobile devices. [ABSTRACT FROM AUTHOR]

Mechanical weeding can reduce the use of pesticides and is of great significance to ensure high yield of crops. Real-time and accurate identification of crops is a key technical problem needs to be solved in mechanical weeding equipment. Because of the subjectivity of feature extraction process in weed recognition, the accuracy of traditional methods in actual field environment is low. In recent years, the method of weed identification based on convolution neural network has been widely studied. Although the accuracy is obviously improved, there are still problems such as large parameters and poor real-time performance. In order to solve the above problems, a four-channel input image is constructed by collecting near infrared and visible images of sugar beet in the field, and a lightweight convolution neural network based on codec structure is proposed. In this paper, Sugarbeet and weed images collected from a farm in Bonn, Germany, in 2016 were used as data sets, which covered images of different growth stages of sugar beet, and 226 pictures of which were randomly selected as training sets, and the remaining 57 pictures were used as test sets. The experimental data set was composed of three channels of visible light image and one channel of near infrared image, which are merged into a four-channel image by pixel level superposition, and the depth-wise separable convolution was used in the deep model. Firstly, the input feature image was convoluted in 2 dimensions convolution kernel and the number of channels was expanded. Then, the 1×1 convolution kernel was used to make the 3 dimensions convolution which combined channel feature and compressed the channels to enhance the nonlinear mapping ability of the model. In order to avoid the problem of the gradient disappearing, the residual block was used to connect the input and output of the depth-wise separate convolution. Finally, the coding and decoder structure was designed and the shallow features were combined with deep features to refine the segmentation effect. Due to the imbalance of pixel proportions of soil, crops and weeds, the weighted loss function was used to optimize the model. The segmentation accuracy, parameters and operating efficiency of models at different input resolutions and different width factor were introduced to evalute the model. When the width factor was 1, the segmentation accuracy of the model increased with the increase of the input image resolution, the model accuracy of four channel input was higher than that of the model based on original visible image input, which showed that the near-infrared image features can compensate the defects of ordinary RGB images to some extent, and make the model more suitable for the dark environment. Under the same input image resolution, the model with a width factor of 2 or 4 performs better than the model with a width factor of 1. With the increases of width factor, the parameters of the model increase greatly. The amount of calculation is related to the size of the input image, so the frame rate gradually decreases with the increase of the size of input image. The experimental results show that the optimal model in this paper is a four channel input model with a width coefficient of 2, and the average intersection union ration is 87.58%, the average pixel accuracy is 99.19%, the parameters are 525 763 and the frame rate is 42.064 frames/s. The model has high segmentation and recognition accuracy and good real-time performance, and can provide theoretical basis for the development of intelligent mechanization weeding equipment. [ABSTRACT FROM AUTHOR]

With the advancement of remote sensing technology, the high spatial resolution remote sensing image contains rich special information with a great detail. At the same time, the complexity of high spatial resolution remote sensing images also requires higher the classification technology of remote sensing images. However, in the face of high spatial resolution remote sensing image more obvious geometrical structure and the more rich texture characteristics, how to design rational system of characteristics, select the appropriate sorting algorithms to accurately and quickly grasp the number of rural land of building and its distribution status, are of great significance to balance urban and rural areas, save land, and realize sustainable development. This will help in exploring the application of deep learning model in high spatial resolution remote sensing image building extraction, and have research significance for improving the classification accuracy of high resolution remote sensing image. In this paper, the semantic segmentation model (SegNet) was used for extracting buildings. SegNet is mainly composed of encoder network, decoder network and pixel-wise classification layer. The encoder network transforms high-dimensional vectors into low-dimensional vectors, enabling low-dimensional extraction of high-dimensional features. The decoder network maps low-resolution feature maps to high spatial resolution feature maps, realizing the reconstruction of low-dimensional vectors to high-dimensional vectors. The softmax classifier separately classifies each pixel, which outputs the probability that each pixel belongs to each class. In this paper, a 3000 pixel × 3000 pixel and two 2000 pixel × 2000 pixel slices were taken from the global remote sensing image of Bazhou City, Hebei Province as training samples, and a 3000 pixel × 3000 pixel slice was taken as the verification sample. In this paper, five comparative experiments were used to extract the buildings, including PSPNet, support vector machine, random forest, ISO clustering and maximum likelihood method. The confusion matrix of each classification method was obtained by calculating the difference between the classification results of the comparison experiment and the real value. From the traditional classification algorithm to the shallow learning algorithm to the deep learning algorithm, the Kappa coefficient and overall accuracy of classification kept constantly increasing, among which SegNet semantic segmentation algorithm based on the deep convolutional network performed better than the other five algorithms in extracting buildings from high spatial resolution remote sensing image. The Kappa coefficient and the overall accuracy of SegNet semantic segmentation algorithm were 0.90 and 96.61%, respectively, and the ground truth value was basically the same as the classification result. The F1Score of building extraction of SegNet semantic segmentation algorithm based on deep convolution network was 0.91, but the other five algorithms were below 0.87. SegNet had the lowest error rate of 9.71% for buildings, indicating that the ability to identify buildings of semantic segmentation algorithm from high spatial resolution remote sensing was superior to traditional classification algorithms, shallow layer learning algorithms based on machine learning, and PSPNet semantic segmentation algorithm based on deep convolution network. The Kappa coefficient and overall accuracy of the remaining five classification algorithms were respectively below 0.83 and 94.68%, and the difference between the ground truth value and the classification result was relatively large. SegNet can not only make use of spectral information but also make full use of abundant spatial information. During SegNet training, more essential features can be learned, and more ideal features suitable for pattern classification were finally formed, which can enhance the ability of convergence and generalization of the model and improve the classification accuracy. Traditional classification algorithms, such as ISO clustering and maximum likelihood method, failed to make use of the rich spatial information of the high-resolution remote sensing image, so the accuracy was relatively low. Due to limited computing units and large amount of high spatial resolution remote sensing image data, shallow layer learning algorithms based on machine learning such as support vector machines and random forest cannot effectively express complex features of ground objects, so their advantages are not obvious in building extraction from the high spatial resolution remote sensing images. The experimental results showed that the SegNet based on deep learning has the best performance, and it has important theoretical significance to explore the application of deep learning model to remote sensing image classification methods. At the same time, the research results also provide a reference for improving the classification accuracy of high resolution remote sensing images. [ABSTRACT FROM AUTHOR]

In this paper, a soil pore segmentation method based on fully convolutional network (FCN) is proposed to improve the accuracy of pore segmentation in soil image and provide technical support for the research of soil science. Taking the soil of typical black soil as the research object, the soil computed tomography image were obtained by scanning and cutting. Based on the FCN network, the soil image and the calibrated image of pore structure were input for convoluting, pooling and deconvoluting operations, and the error between the prediction image and the calibration image was used as feedback to complete the forward inference operation. Then, the weight value was updated by the back propagation algorithm to establish the soil pore segmentation model. Fully considering the pore geometry and spatial distribution characteristics, the pore model can accurately output the soil pore binary image. Meanwhile, the commonly used segmentation methods in the literature, such as Otsu method, watershed method, regional growth method and Fuzzy C-means method (FCM) were adopted for the comparative experiments on soil computed tomography images with low pore density (0-0.03), medium pore density (0.03-0.1) and high pore density (0.1-1) which were defined by porosity of soil. The experimental results showed that the watershed method and the regional growth method overestimate the pore structure of different geometries, including cracks between the pores, whereas the Otsu method and FCM method tended to overestimate the macropores and underestimate the micropores. Compared the five methods, the FCN method can accurately extract the pore structures with vary topologies from the complex soil computed tomography images with low, medium and high pore density. Moreover, the segmentation accuracy rate, over-segmentation rate, and under-segmentation rate were used to evaluate the soil pore segmentation performance of five methods. Based on 1487 soil computed tomography images, the average segmentation accuracy of FCN pore segmentation method was 98.1%, which was 25.6%, 48.3%, 55.7% and 9.5% higher than that of Otsu method, watershed method, regional growth method and FCM method. The average over-segmentation rate of the FCN pore segmentation method was 2.2%, which was only 33.8% of the suboptimal method (FCM method), respectively. And the average under-segmentation rate of the FCN pore segmentation method was 1.3%, which was only 23.6% of the suboptimal method (FCM method). In total, the FCN method can accurately extract the pore topology, restore the spatial distribution of pores and its application can make up for the shortcoming that the traditional segmentation method only uses the low-level features (gray and edge) when extracting the pore structure. Owing to the multiple convolution layers in the network, the FCN method can obtain the vary features of pore structure, so it has strong generalization ability and robustness of pore segmentation for different types of soil images. This paper will has a good reference for the microscopic process simulation , 3D reconstruction and soil structure analysis on the pore scale, and can provide a more intelligent technical method for soil science. [ABSTRACT FROM AUTHOR]

The existing pig counting system based on image processing are seriously affected by light conditions, and the counting accuracy is poor when pigs are crowded and obscured.To realize the intelligent pig counting, the paper proposed a counting scheme based on the improved instance segmentation algorithm was proposed in this study. Aiming at the problems of image illumination and target edge blur, Laplace operator was used to preprocess the images. The feature extraction network of MASK R-CNN network was improved by using Resnet-152 as the Mask R-CNN feature extraction network, and the original Feature Pyramid Network(FPN) was followed by a bottom-up enhancement path, which directly fused the low-level edge features with the high-level features to improve the recognition ability of the target edge bluring. The non maximum suppression process and loss function were optimized and improved to improve the segmentation accuracy. The experiments were carried out at three different real pig farms to verify the counting accuracy, respectively in Hebei province, Jilin province and Inner Mongolia. The size of the cage in Hebei pig farm was 5.5 m × 1.8 m, with an average of 12 pigs in a single pen, with a feeding density of 1.21 pigs/m²; the size of the cage in Jilin pig farm was 5.5 m × 3.9 m, with an average of 22 pigs in a single pen, and the average rearing density was 1.03 pigs/m²; the size of the cage in Inner Mongolia pig farm was 11.4 m × 5.28 m, with an average of 80 pigs in a single pen, the average feeding density was 1.32 pigs/m². The RGB camera is was installed on the top of the pen and acquired the image in daytime. 2 400 images were collected in total, and 2000 images were selected after image preprocessing, and 1 250 images of three pig farms were selected as the original data set according to the ratio of 2:2:1. The training set and verification set were enhanced to 1 500 and 150 images and 250 images for the test set, The experimental results showed that 98 images could realize exact counting and 2 images missed 1 pig in the Jilin pig farm, the accuracy of pig counting was 98%. In Hebei pig farm, 99 images could realize the exact counting and the accuracy of pig counting was 99%. For Inner Mongolia pig farms with high feeding density, the accuracy of pig counting was 86%, among the 50 test images, 7 images missed detection, 4 images missed 1 target, 3 images missed 2 targets, The results can provide the application of the artificial intelligent in agriculture field. [ABSTRACT FROM AUTHOR]

Effective recognition method of crop and weed is the basis for promoting the development of intelligent mechanization weeding pattern. Summarizing the previous research, we found that the accuracy and stability of the recognition model could be improved by natural and diversified feature presentation, but there are still 2 main problems. On the one hand, feature presentation of the natural property of target was difficult to be obtained by the hand-engineered feature extractor. The spatial consistency of the obtained features was bad, and the real-time performance of recognition system was reduced for the complex feature extraction algorithm. On the other hand, the effect of image preprocessing has important influence on recognition results, especially the overlapping segmentation of crop and weed. For overlapped objects, it is usually difficult to segment them without affecting their respective feature presentations, resulting in low recognition accuracy and stability. In order to solve the main problems in the current research, we explored the way to improve the recognition accuracy, stability and real-time performance, and a recognition method of crop and weed based on multiscale hierarchical feature learning combined with superpixels segmentation was proposed. The main research contents of this paper were as follows: 1) Excellent internal features of image are hierarchical. In this research, the multiscale hierarchical feature is a scene level feature with invariance and consistency in scale space. Multiscale convolutional neural network was built to extract multiscale hierarchical feature. Multiscale convolutional neural network contains multiple copies of a single CNN (convolutional neural network) that are applied to multi-scale Gaussian pyramid of the input image. The CNN model as feature extractor in this paper includes 3 stages. In the first 2 stages, it contains a bank of filters (convolution kernels) to produce dense feature maps, a point to point nonlinear mapping activation function, and a spatial pooling module for sub-sampling of each feature map, While the last stage only contains a bank of filters. Each filter is applied to the input feature maps through 2-dimensional convolution operation, in which local feature presentations are detected at all pixel locations on the input image. For each pixel, the CNN model is used to collectively encode the internal information in a large sense region around the location of given pixel. The CNN is fed with raw pixels and trained with back propagation method. With complete training, this CNN model can automatically extract hierarchical feature representations from the input image, thereby decreasing the need for hand-engineered features extracting. A series of feature maps for multiscale regions centered on each pixel of the image are produced by the multiscale convolutional neural network; these representations contain shape, texture and sense information. Therefore, the multiscale hierarchical feature is learned to allow the recognition of the class of all pixels in the image. The average pixel recognition rate is 93.41%. 2) In this research, multiscale hierarchical feature is used to produce the class distribution for every pixel through a 2-layer MLP (multi-layer perceptron). But recognizing the class of pixel independently from its surrounding regions could produce some interference at the boundary of targets. Accurate boundary segmentation for each target in the image is not provided. In this paper, the superpixel method is used for generating an over segmentation of the image, where each segmentation component (superpixel) is an irregular pixel block composed of similar pixels in texture, color and brightness characteristics. Object in the image is composed of these superpixels, and has the exact original boundary. The superpixels segmentation is computed following the simple linear iterative clustering method described in this paper. 3) We proposed the recognition strategy of multiscale hierarchical feature learning combined with the superpixels segmentation. Firstly, an over segmentation of the original image was produced through the superpixels segmentation. At the same time, each pixel location of the image was classified densely based on the multiscale hierarchical features. These predictions of pixels in each superpixel were aggregated to produce the class prediction of superpixel, through computing the average class distribution within the superpixel. Adjacent superpixels with the same class were merged to obtain the final target class prediction and image segmentation. The accurate image segmentation was achieved while recognizing the target in the image by this recognition method, which effectively avoided the problems caused by targets overlapping, and the recognition results were more stable and accurate. Maize seedlings at 2-5 leaves stage and weed during the same stage were used as research object, and the recognition method of multiscale hierarchical feature learning combined with superpixels segmentation was tested. The results showed that the average target recognition rate with this method was 98.92%, and the standard deviation was 0.55%. The average target recognition rate with the method in previous research was 98.36%, and the standard deviation was 1.05%. So, the accuracy and stability of recognition results by this method in this paper were improved with different degrees. In aspect of real-time performance, the average time to recognize a single image was 1.68 s with this method, and compared with the method proposed in previous research, it was decreased by 1.58 s. The real-time performance of this method can be further improved by GPU (graphics processing unit) hardware acceleration, and the average time to recognize a single image was only 0.72 s. Therefore, the recognition method based on multiscale hierarchical feature learning combined with superpixels segmentation can effectively achieve accurate, stable and efficient recognition of maize and weed. The research results provide reference for the development of precision weeding. [ABSTRACT FROM AUTHOR]

Chrysanthemum is one of the top 10 traditional famous flowers in China, which has significant importance and great ornamental value and medicinal value. The chrysanthemum flowers are characterized by a large number of varieties and a wide range of petal shapes, which pose big challenges to their intelligent identification and efficient management. Currently, the identification and management of chrysanthemum mainly relies on the traditional manual way, and as a result, the efficiency is quite low. At the contemporary era, deep learning as a powerful technique in artificial intelligence field is becoming a prevalent way of identification and classification on text, image, video, and so on. Based on the end-to-end convolutional neural network deep neural network directly acting on the original chrysanthemum image dataset, this paper aims at obtaining the characteristic information of chrysanthemums through the multi-layer neural network. By this means, the problem of extracting the features manually is avoided, and then optimization target function is applied to achieve a better image recognition accuracy. Based on this, the system of chrysanthemum flower pattern intelligent recognition and breed classification is researched and implemented. In view of the subtle differences among the flower patterns of chrysanthemums, on the one hand, in order to preserve as much information as possible for the data, tensor is employed to represent the image data; on the other hand, the pairwise confusion loss function based on pair similarity is used to distinguish pattern differences and similarities. By this means, the objective function of distinguishing the different flower patterns is realized on the fine grain size. The system not only can identify the chrysanthemum pattern, but also can give the probability value of the top 3 results. In addition to this, the variety information covered by the flower pattern is also provided. The operation of the system can be divided into 2 stages: the off-line training and the online classification. Off-line models can be hosted in the cloud environment such as Amazon AWS for the easy usage on the mobile platform. Moreover, the model can be replanted and updated with little hindrance. In order to train the network model, we collected a large amount of data of real chrysanthemum image, and manually marked the relevant pattern and category information of chrysanthemum. Based on the datasets, we conducted extensive experiments with our system and made comparisons with 3 existing systems, and experimental results show that: The identification accuracy of the system has been significantly improved compared with the existing systems for chrysanthemum flower pattern. Beyond that, the system can provide more detailed chrysanthemum species information at the same time. The average recognition rate can reach about 0.95, and even surpass the rate of 0.98 for some chrysanthemum patterns. The system provides a powerful means for the automatic management of chrysanthemum and fills the gaps in chrysanthemum pattern recognition and classification. In this paper, the research on the intelligent identification and effective management of chrysanthemum flower pattern has great significance in theory and practice. [ABSTRACT FROM AUTHOR]

Recognition of the main organs of tomato from plant images has important significance for the detection of plant diseases and insect pests, the application of targeted pesticides, and the development of intelligent agricultural machinery. The traditional object recognition methods which are limited by hand-engineered features extraction process and the selection of filter operator types, have poor generality and low recognition accuracy. The convolutional layers of convolutional neural network (CNN) can achieve automatic extraction of image features. In view of this, we propose a tomato main organs recognition method which relies on deep convolutional neural network (DCNN) in this paper. Under different illumination conditions, and considering the morphological diversity of tomato organs, 5 kinds of images including flower, fruit, stem, leaf, and environment background and the plant images are collected, which are used as training and testing samples. Under the premise of maintaining the basic structure of VGGNet, by adjusting the number of convolutional kernels and the depth of convolutional layers, 10 kinds of tomato organs classification networks are constructed. To reduce overfitting on tomato organ image dataset, before using the dataset to train the individual network, we use many data augmentation techniques to enlarge the dataset such as label-preserving transformations, specifically image rotation at random angles, extracting random patches of one image, image flip horizontal, and altering the intensities of the RGB (red, green, blue) channels of training images. At the same time, the median filter is used to denoise image samples. We train our models end to end using stochastic gradient descent with input image size of 64×64 or 128×128 pixels, mini-batch size of 128 examples, and momentum of 0.9. The classification test results show that the 10 kinds of networks all have very good classification performance by top-1 error rate. Among the 10 kinds of networks, the 8-layer structure network's top-1 error rate is 0.297%, which has the best classification performance. The results reveal that 8-layer structure network can satisfy the features extraction and classification requirements of tomato image dataset with fewer categories. Considering the classification performance and computing speed, referencing Fast R-CNN, with the 8-layer structure network as the basic architecture, and using the region of interest pooling (RoI pooling) layer to replace the last maximum pooling (max-pool) layer, a tomato main organ detector is designed, and named as TD-Net. The detector can output the category probabilities for each object proposal through the soft-max layer. The selective search algorithm is adopted for generating a large number of proposal regions for each tomato plant image, and the non-maximum suppression algorithm is used to suppress the outputs of the TD-Net. Using plant images and proposal regions as input, the detector is trained iteratively. The detection experiment results show that the detection average precisions (AP) of the DCNN-based tomato organ detector for the fruit, flower and stem are 81.64%, 84.48% and 53.94% correspondingly, and the detector can effectively identify the fruits of different maturity degree and the flowers of different flower age. On the same tomato plant image dataset, compared with R-CNN and Fast R-CNN, TD-Net has a better detection performance and detection speed, which illustrates that the method of this paper is effective. [ABSTRACT FROM AUTHOR]

Cotton (Gossypium hirsutum) is one of the most important cash crops in China, The timely and effective removal of weeds in cotton seedling stage is an important measure to ensure high and stable yield of cotton. Nowadays, weed recognition based on machine vision is widely used. The fast and effective recognition of crop and weed in the field under natural illumination is one of the key technologies for the development of intelligent mechanization weeding pattern. In the one hand, cotton and weeds have similar color feature in the field. Feature presentation of the natural property of target is difficult to be obtained by the hand-engineered feature extractor. The spatial consistency of the obtained features is not good, and the real-time performance of recognition system is reduced for the complex feature extraction algorithm. On the other hand, the effect of image preprocessing has important influence on recognition results. In order to solve the main problems in the current research, we explored the way to improve the recognition accuracy, stability and real-time performance, and a recognition method of crop and weed based on Faster R-CNN．In this paper, cotton seedling at 2-5 leaves stages and weeding during the same stage were used as research objects under natural illumination. Weed identification from digital images taken under natural illumination at field level is still challenging in agricultural image processing applications, though a lot of research has been conducted related to this topic. To address this problem, images including cottons and weeds were taken vertically from top to bottom. A method based on Faster R-CNN convolutional neural network was proposed to identify weeds from cotton plants more accurately and quickly. The residual network was used to extract image features, with ReLU as the activation function and Max-pooling as the down-sampling method. In the region of proposal network, feature pyramid network was introduced to generate target candidate frame, and Softmax regression classifier was utilized to optimize the CNN network. The proposed methodology was implemented on 200 digital images taken under natural illumination. The experimental results demonstrated that, the average accuracy of weed identification reached 95.5%, and the average time for individual weed plant identification was 1.51 s, which was reduced to 0.09 s by using GPU. To test the efficiency of the proposed methodology, YOLOv3 method was also carried out on the same training and test datasets. The weed identification results were assessed by mean average precision and average precision. The experimental results showed that better performance was achieved by using our proposed methodology, and better identification accuracy was reached as well. This indicated that the proposed method had a good effect on weed detection under natural illumination, and it will greatly promote the development of precise weed control. [ABSTRACT FROM AUTHOR]

Accurate and timely winter wheat yield estimation has significant effect to grain markets and policy. Most crop estimation methods can be divided into two categories, one is based on the crop model and the other is the statistical learning method. For statistical learning methods with recent advances in deep learning, convolutional neural network (CNN) have become state-of-the-art algorithms. can extract the depth-dependent features of crop growth. However, the pivotal challenge is to combine remote sensing images with CNN. In this paper, we employ the method of histogram dimensionality reduction and time series fusion to generate the input layer. The experiment firstly performed projection transformation, splicing, mask, fusion, and clipping for 6 different MODIS images in the research area from 2006 to 2016, and then generated 21 600 fusion images of 12 bands (surface reflectance data of 7 different wavelengths in MOD09A1, surface temperature of day and night in MYD11A2, NDVI and EVI in MOD13A1, and FPAR in MOD15A2H). Then, the sensitivity range of winter wheat growth in each band is divided into 36 sections, and the histogram statistics are used to reduce the dimension to generate a vector of length 36, so the remote sensing image generates a matrix of 36×36×12 in the 228-day growing season. The corresponding time and regional statistics are applied as the output layer to construct a complete sample. The yield estimation sample database of 12 indices in the winter wheat region of north China (60 prefecture-level cities) from 2006 to 2016 was constructed, and the training set and verification set were divided into 10:1 for the training and evaluation of yield estimation model. Finally, the neural network structure is designed according to the sample, which consists of the input layer, 7 convolution layers (c1-c7), 7 activation layers, 7 batch normalization layers, 3 dropout layers, 2 full connection layers, and output layer. The number of c1-c7 convolution kernels is 64, 64, 128, 128, 256, 256, 256, the convolution kernel size is 3×3 dpi, and the sliding step length is 2, 1, 2, 2, 2, 1 and 2 respectively, 1 zero paddings per convolutional layer. At the same time, batch normalization and Relu function activation are performed on each convolutional layer, and the Dropout layer is used in the fully connected layer. The results show that: 1) The root-mean-square error (RMSE) and coefficient of determination (R²) of the convolutional neural network model on the training set are 183.82 kg/hm² and 0.98 respectively. In the validation set, RMSE and R² are 689.72 kg/hm² and 0.71. 2) With the same neural network structure, the average RMSE of the estimated samples from 2006 to 2016 trained as validation sets for 11 models was 772.03 kg/hm². The error of the yield estimation model was the largest in 2007 and the smallest in 2012, and the RMSE was 920.45 kg/hm² and 632.08 kg/hm² respectively. Crop estimation algorithm based on CNN has high robustness and precision; 3) The accuracy analysis of prediction yield at the municipal level of different provinces in three temporal points of 2007, 2012 and 2016 indicates that the model has higher accuracy in most areas of the northern winter wheat region, especially, RMSE of Hebei and Shandong provinces is approximately 500 kg/hm². The result shows that CNN is well applied to the estimation of winter wheat production. This is a great thought of remote sensing combined with the deep learning algorithm. This method can be used to estimate yield by remote sensing in different scales and regions. Compared with the traditional method, this “start-to-end” learning method has the advantage of synergy and can obtain the optimal estimation model relative to the whole area. Meanwhile, As data accumulates, the estimation accuracy will be continuously improved, and it has a good application prospect in the national agricultural production forecast. [ABSTRACT FROM AUTHOR]

Accurate and timely winter wheat yield estimation has significant effect to grain markets and policy. Most crop estimation methods can be divided into two categories, one is based on the crop model and the other is the statistical learning method. For statistical learning methods with recent advances in deep learning, convolutional neural network (CNN) have become state-of-the-art algorithms. CNN can extract the depth-dependent features of crop growth. However, the pivotal challenge is to combine remote sensing images with CNN. In this paper, we employ the method of histogram dimensionality reduction and time series fusion to generate the input layer. The experiment firstly performed projection transformation, splicing, mask, fusion, and clipping for 6 different MODIS images in the research area from 2006 to 2016, and then generated 21 600 fusion images of 12 bands (surface reflectance data of 7 different wavelengths in MOD09A1, surface temperature of day and night in MYD11A2, NDVI and EVI in MOD13A1, and FPAR in MOD15A2H). Then, the sensitivity range of winter wheat growth in each band is divided into 36 sections, and the histogram statistics are used to reduce the dimension to generate a vector of length 36, so the remote sensing image generates a Matrix of 36×36×12 in the 228-day growing season. The corresponding time and regional statistics are applied as the output layer to construct a complete sample. The yield estimation sample database of 12 indices in the winter wheat region of north China (60 prefecture-level cities) from 2006 to 2016 was constructed, and the training set and verification set were divided into 10:1 for the training and evaluation of yield estimation model. Finally, the neural network structure is designed according to the sample, which consists of the input layer, 7 convolution layers (c1-c7), 7 activation layers, 7 batch normalization layers, 3 dropout layers, 2 full connection layers, and output layer. The number of c1-c7 convolution kernels is 64, 64, 128, 128, 256, 256, 256, the convolution kernel size is 3×3 dpi, and the sliding step length is 2, 1, 2, 2, 2, 1 and 2 respectively, 1 zero paddings per convolutional layer. At the same time, batch normalization and Relu function activation are performed on each convolutional layer, and the Dropout layer is used in the fully connected layer. The results show that: 1) The root-mean-square error (RMSE) and coefficient of determination (R2) of the convolutional neural network model on the training set are 183.82 kg/hm2 and 0.98 respectively. In the validation set, RMSE and R2 are 689.72 kg/hm2 and 0.71. 2) With the same neural network structure, the average RMSE of the estimated samples from 2006 to 2016 trained as validation sets for 11 models was 772.03 kg/hm2. The error of the yield estimation model was the largest in 2007 and the smallest in 2012, and the RMSE was 920.45 kg/hm2 and 632.08 kg/hm2 respectively. Crop estimation algorithm based on CNN has high robustness and precision; 3) The accuracy analysis of prediction yield at the municipal level of different provinces in three temporal points of 2007, 2012 and 2016 indicates that the model has higher accuracy in most areas of the northern winter wheat region, especially, RMSE of Hebei and Shandong provinces is approximately 500 kg/hm2. The result shows that CNN is well applied to the estimation of winter wheat production. This is a great thought of remote sensing combined with the deep learning algorithm. This method can be used to estimate yield by remote sensing in different scales and regions. Compared with the traditional method, this “start-to-end” learning method has the advantage of synergy and can obtain the optimal estimation model relative to the whole area. Meanwhile, As data accumulates, the estimation accuracy will be continuously improved, and it has a good application prospect in the national agricultural production forecast. [ABSTRACT FROM AUTHOR]

China is the largest country for cultivating kiwifruit, and Shaanxi Province provides the largest production, which accounts for approximately 70% of the production in China and 33% of the global production. Harvesting kiwifruit in this region relies mainly on manual picking which is labor-intensive. Therefore, the introduction of robotic harvesting is highly desirable and suitable. The fast and effective recognition of kiwifruit in the field under natural scenes is one of the key technologies for robotic harvesting. Recently, the study on kiwifruit recognition has been limited to a single cluster and multi clusters in the field have seldom been considered. In this paper, according to growth characteristics of kiwifruit grown on sturdy support structures, an RGB (red, green, blue) camera was placed around 100 cm underneath the canopy so that kiwifruit clusters could be included in the images. We proposed a kiwifruit image recognition system based on the convolutional neural network (CNN), which has a good robustness avoiding the subjectivity and limitation of the features selection by artificial means. The CNN could be trained end to end, from raw pixels to ultimate categories, and we optimized the critical structure parameters and the training strategy. Ultimately, the network was made up of 1 input layer, 3 convolutional layers, 2 sub-sampling layers, 1 full convolutional layer, and 1 output layer. The CNN architecture was optimized by using batch normalization (BN) method, which normalized the data distribution of the middle layer and the output data, accelerating the training convergence and reducing the training time. Therefore, the BN layers were added after the 1, 3 and 5th convolutional layer (Conv1, Conv3, and Conv5 layer) of the original LeNet network. The size of all convolutional kernels was 5×5, and that of all the sub-sampling layers was 2×2. The feature map numbers of Conv1, Conv3, and Conv5 were 6, 16 and 120, respectively. After manual selection and normalizing, the RGB image of kiwifruit was transferred into a matrix with the size of 32×32 as the input of the network, stochastic gradient descent was used to train our models with mini-batch size of 100 examples, and momentum was set as 0.9. In addition, the CNN took advantages of the part connections, the weight sharing and Max pooling techniques to lower complexity and improve the training performance of the model simultaneously. The network used rectified linear units (ReLU) as activation function, which could greatly accelerate network convergence. The proposed model for training kiwifruit was represented as 32×32-6C-2S-16C-2S-120C-2. Finally, 100 images of kiwifruit in the field (including 5918 fruits) were used to test the model, and the results showed that the recognition ratios of occluded fruit, overlapped fruit, adjacent fruit and separated fruit were 78.97%, 83.11%, 91.01% and 94.78%, respectively. The overall recognition rate of the model reached 89.29%, and it only took 0.27 s in average to recognize a fruit. There was no overlap between the testing samples and the training samples, which indicated that the network had a high generalization performance, and the testing images were captured from 9 a.m. to 5 p.m., which indicated the network had a good robustness to lightness variations. However, some fruits were wrongly detected and undetected, which included the fruits occluded by branches or leaves, overlapped to each other and the ones under extremely strong sunlight. Particularly, 2 or more fruits overlapped were recognized as one fruit, which was the main reason to the success rate not very high. This phenomenon demands a further research. By comparing with the conventional methods, it suggested that the method proposed obtained a higher recognition rate and better speed, and especially it could simultaneously identify multi-cluster kiwifruit in the field, which provided significant support for multi-arm operation of harvesting robotic. It proves that the CNN has a great potential for recognition of fruits in the field. [ABSTRACT FROM AUTHOR]

Plant leaf diseases are a serious problem in agricultural production. To solve this problem and prevent diseases deterioration, accurate identification of diseases types is of great significance. In this paper, we proposed a recognition model of plant leaf diseases based on convolutional neural network (CNN), which combines the batch normalization and global pooling methods. The parameters of the traditional CNN model are large and have difficulty to converge. The proposed model was modified in the traditional structure of the CNN, which could optimize the training time and achieve the higher accuracy, and also reduce the size of model. In order to speed up the training convergence, we used the batch normalization layers. We put the input of every convolutional layer in batch, calculated the mean and variance of the batch, and then normalized this batch. We reduced some feature maps of some layers and removed the last full connect layer, with the global pooling layer instead. The proposed model has 5 convolutional layers and 4 pooling layers. In the last pooling layer pool5, the same kernel size of convolutional layer Conv5 was used to take advantage of the information of Conv5's feature map comprehensively. For the image preprocessing, we had zoomed, flipped and rotated the original pictures of dataset randomly to get the augmented dataset, and used the 80% of pictures as the train dataset and the rest as the test dataset. These pictures were quantized to 256×256 dpi for CNN training, and the original dataset and augmented dataset were used to train models. To look for the best size of the first layer kernel, in the first convolutional layer, different kernel sizes i.e. 11×11, 9×9 and 7×7 dpi were used respectively. Furthermore, we chose the type of global pooling layer, like max pooling and average pooling. Then we designed 8 models with different Conv1 kernel sizes or global pooling types. To further improve the efficiency of this model, besides using the Gaussian initialization, we used the other common type of convolutional initialization such as Xavier initialization, and also used the PRelu activation function for each convolution layer. So the optimal model could be selected to recognize the 26 kinds of leaf diseases which involved 14 kinds of plants, and then we analyzed the model's convergence rate, memory usage and robustness. After the experiment, we compared the test accuracy between the traditional model and the proposed model based on original dataset and augmented dataset. The proposed model could accelerate the training convergence, and the test accuracy could achieve about 90% while the traditional model was only about 77% after 3 training epochs. Different kernel sizes of Conv1 had little impact on the accuracy but small kernel was proved to be more beneficial to the recognition of plant diseases, which could get more texture features than the big kernel size filter, and average pooling also made better results than max pooling. We got the best performance model which used the 9×9 dpi kernel size and global average pooling layer. To show the proposed model's performance, we tested the accuracy on each class, and the mean accuracy of augmented test dataset was 99.56%, and the weighted average score of recall and precision rate achieved 99.41%. The proposed model had the size of only 2.6 MB. In addition, compared with the traditional methods, the change of the spatial position of the pictures had little effect on the performance of the improved model, and the proposed model could identify different diseases of various plant leaves. The results show that the model has higher recognition accuracy and stronger robustness, and can be used for the identification of plant leaf diseases. [ABSTRACT FROM AUTHOR]

Tea is one of important cash crops in China. Computer vision plays an important role in pest detection. Automatic classification of insect species in field is more difficult than the generic object classification because of complex background in filed and high appearance similarity among insect species. In this paper, we proposed an insect recognition system on the basis of image saliency analysis and a deep learning model, i.e. convolutional neural network (CNN), which has a good robustness with avoiding the features selected by artificial means. In image saliency analysis, we segmented the original images into super-pixel regions firstly. Then we quantized each RGB (red, green, blue) color channel and made them have 10 different values, which reduced the number of colors to 1000, and sped up the process of the color contrast of the pest objects and the background at region level. Finally, we obtained the saliency value of each region by combining their color contrast and spatial distances. The saliency values of all regions in each image were used to construct a saliency map, which was offered as the initial area for GrabCut algorithm to define the segmentation result and localize the pest object. The images after localization were quantized to 256×256 dpi for CNN training and classifying. CNN was trained end to end, from raw pixels to ultimate categories, thereby alleviating the requirement to manually design a suitable feature extractor. Based on theoretical analysis and experimental evaluation, we optimized the critical structure parameters and training strategy of CNN to seek the best configuration. The overall architecture included a number of sensitive parameters and optimization strategies that could be changed. We determined the local receptive field size, number, and convolutional stride as 7×7 dpi, 64 and 4, respectively. Dropout ratio for the fully-connected layers was 0.7. The loss function Softmax was fit for the pest classification system. To further improve the practical utility of CNN, we focused on structural changes of the overall architecture that enabled a faster running with small effects on the performance. We analyzed the performance and the corresponding runtime of our model by reducing its depth (number of layers) and width (number of convolution kernel in each layer). Removing the fully-connected layers (FC6, FC7) made only a slight difference to the overall architecture. These layers contained almost 90% of the parameters and when they were removed, the memory consumption decreased to 29.8 MB. But, removing the intermediate convolutional layers (Conv2, Conv3, Conv4, Conv5) resulted in a dramatic decrease in both accuracy and runtime. This suggested that the intermediate convolutional layers (Conv2, Conv3, Conv4, Conv5) constituted the main part of the computational resource, and their depth was important for achieving good results. We then investigated the effects of adjusting the sizes of all convolutional layers, and the filters in each convolutional layer were reduced to 64 each time. Surprisingly, all architectures showed significant decreases in running time with relatively small effects on performance. Finally, we determined the convolution kernel numbers of Conv2-5: 64-192-192-64. On the test set of tea field images, the architecture before and after shrinking respectively achieved the average accuracy (AA) of 0.915 and 0.881, respectively, superior to previous methods for pest image recognition. Further, after optimization the running time reduced to 0.7 ms and the memory required was 6 MB. [ABSTRACT FROM AUTHOR]