Exploring Google Street View with deep learning for crop type mapping

Ground reference data are an essential prerequisite for supervised crop mapping. The lack of a low-cost and efficient ground referencing method results in pervasively limited reference data and hinders crop classification. In this study, we apply a convolutional neural network (CNN) model to explore the efficacy of automatic ground truthing via Google Street View (GSV) images in two distinct farming regions: Illinois and the Central Valley in California. We demonstrate the feasibility and reliability of our new ground referencing technique by performing pixel-based crop mapping at the state level using the cloud-based Google Earth Engine platform. The mapping results are evaluated using the United States Department of Agriculture (USDA) crop data layer (CDL) products. From ~ 130,000 GSV images, the CNN model identified ~ 9,400 target crop images. These images are well classified into crop types, including alfalfa, almond, corn, cotton, grape, rice, soybean, and pistachio. The overall GSV image classification accuracy is 92% for the Central Valley and 97% for Illinois. Subsequently, we shifted the image geographical coordinates 2–3 times in a certain direction to produce 31,829 crop reference points: 17,358 in Illinois, and 14,471 in the Central Valley. Evaluation of the mapping results with CDL products revealed satisfactory coherence. GSV-derived mapping results capture the general pattern of crop type distributions for 2011–2019. The overall agreement between CDL products and our mapping results is indicated by R2 values of 0.44–0.99 for the Central Valley and 0.81–0.98 for Illinois. To show the applicational value of the proposed method in other countries, we further mapped rice paddy (2014–2018) in South Korea which yielded fairly well outcomes (R2 = 0.91). These results indicate that GSV images used with a deep learning model offer an efficient and cost-effective alternative method for ground referencing, in many regions of the world.