A context-based meta-reinforcement learning approach to efficient hyperparameter optimization.

[Display omitted] • A context helps the agent to learn knowledge from previous tasks. • A simple multi-task objective meta-training procedure is proposed to run faster. • A quadratic penalty technique is used to preserve the historical experience. • A predictive model is applied to further accelerate the adaptation phase. In this paper, we present a context-based meta-reinforcement learning approach to tackle the challenging data-inefficiency problem of Hyperparameter Optimization (HPO). Specifically, we design an agent which sequentially selects hyperparameters to maximize the expected accuracy of the machine learning algorithm on the validation set. First, we design a context variable that learns the latent embedding of prior experience, and the agent can solve the new tasks efficiently conditioned on it. Second, we employ a multi-task objective method that aims to maximize the average reward across all the meta-training tasks to meta-train the agent. Third, in the adaptation phase, we introduce a quadratic penalty technique to achieve better performance of the agent. Finally, to further improve the efficiency in the adaptation phase, we use a predictive model to evaluate the accuracy of machine learning algorithm instead of training it. We evaluate our approach on 18 real-world datasets and the results demonstrate that our approach outperforms other state-of-the-art optimization methods in terms of test set accuracy and runtime performance. [ABSTRACT FROM AUTHOR]