Multi-year crop type mapping using pre-trained deep long-short term memory and Sentinel 2 image time series

This work presents a system for multi-year crop type mapping based on the multi-temporal Long Short-Term Memory (LSTM) Deep Learning (DL) model and Sentinel 2 image Time Series (TS). The method assumes the availability of a pre-trained LSTM model for a given year and aims to update the corresponding crop type map fora different year considering a small amount of recent reference data. To this end, the proposed approach combines Self-Paced Learning (SPL) and fine-tuning (FT) techniques. While the SPL technique gradually incorporates samples from crop types that can be classified with high-confidence by the pre-trained model, the FT strategy adapts the network to those classes having low-confidence accuracy. This condition allows us to reduce the labeled samples required to achieve accurate classification results. The experimental results obtained on three tiles of the Austrian country on TSs of Sentinel 2 data acquired in 2019 and 2020 (considering a model pre-trained on images of 2018) demonstrate the capability of the LSTM to adapt to TS of images with different temporal and radiometric characteristic with respect to the one used to pre-train the model, with a relatively small number of training samples. As expected, by directly applying the model without performing any adaptation, we obtain a mean F-score (F1%) of 64% and 62% compared to 76% and 70% achieved by the proposed technique with only 1500 samples for 2019 and 2020, respectively.