Hidden Markov guided Deep Learning models for forecasting highly volatile agricultural commodity prices.

Predicting agricultural commodity prices accurately is of utmost importance due to various factors such as perishability, seasonality, production uncertainty etc. Moreover, the substantial volatility that may be exhibited in time series further adds to the complexity and constitutes a significant challenge. In this paper, a Hidden Markov (HM) guided Deep Learning (DL) models has been developed on nonlinear and nonstationary price data of agricultural commodities for forecasting by considering technical indicators viz., Moving Average (MA), Bollinger Bands (BB), Moving Average Convergence Divergence (MACD), Exponential MA (EMA) and Fast Fourier Transformation (FFT). HM Models (HMMs) can effectively handle the sequential dependencies and hidden states, while DL approach can learn complex patterns and relationships within the price series and thus the drawback of lack of generalization capability in the DL model has been overcome by HMM. In this study, the Potato price data of the Champadanga district of West Bengal, India has been utilized to assess the performance of the proposed technique. HMM has been combined with six baseline DL models viz., Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), Bidirectional LSTM (BiLSTM) and Bidirectional GRU (BiGRU) for forecast modeling. Performance evaluation metrics viz., Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE) and the insightful Diebold–Mariano (DM) test revealed that Hidden Markov hybridized with DL models surpassed baseline DL models in forecasting accuracy for 1-week, 4-week, 8-week and 12-week ahead DL predictions. The proposed approach holds significant promise for enhancing the precision of agricultural commodity price forecasting with far-reaching implications for various stakeholders such as farmers and planners. • A novel Hidden Markov based Deep Learning (DL) models for accurate agricultural commodity price forecasting. • Addresses DL model generalization challenges by integrating sequential dependencies and hidden states obtained from HMMs. • Proposed models outperform baseline models - RNN, CNN, LSTM, GRU, BiLSTM, and BiGRU for price predictions of agriculture commodities. • Evaluated using RMSE, MAPE, MAE and Diebold Mariano test, for accuracy and reliability of the proposed approach. • Offers precision in commodity price forecasts, benefiting stakeholders such as farmers, planners and policymakers. [ABSTRACT FROM AUTHOR]