Analyzing Efficiency-based Hyperparameter Tuning Optimization Methods on LSTMs for Generative ARIMA Models.

In the deep learning field, Long Short-Term Memory networks (LSTMs) are a type of recurrent neural network (RNN) that use gates to control and retain important information, enabling them to handle long-range dependencies and effectively forecast time series models. Despite their effectiveness, LSTMs can be further optimized for increased accuracy and efficiency. Optimizing these networks involves fine-tuning hyperparameters, which are external parameters determined manually. While this can be done through trial and error, it is more efficiently accomplished using hyperparameter optimization algorithms such as random search, Bayesian optimization, and Hyperband. This paper compares these three algorithms, finding that random search achieved low mean-squared error but raised concerns about reproducibility and consistency, Hyperband excelled in resource efficiency but struggled with optimal configuration, and the Bayesian algorithm offered consistency and a smooth learning process at a higher computational cost and required parameter adjustments. [ABSTRACT FROM AUTHOR]