1. Visual Analysis of the Impact of Neural Network Hyper-Parameters
- Author
-
Jönsson, Daniel, Eilertsen, Gabriel, Shi, Hezi, Jianmin, Zheng, Ynnerman, Anders, Unger, Jonas, Jönsson, Daniel, Eilertsen, Gabriel, Shi, Hezi, Jianmin, Zheng, Ynnerman, Anders, and Unger, Jonas
- Abstract
We present an analysis of the impact of hyper-parameters for an ensemble of neural networks using tailored visualization techniques to understand the complicated relationship between hyper-parameters and model performance. The high-dimensional error surface spanned by the wide range of hyper-parameters used to specify and optimize neural networks is difficult to characterize - it is non-convex and discontinuous, and there could be complex local dependencies between hyper-parameters. To explore these dependencies, we make use of a large number of sampled relations between hyper-parameters and end performance, retrieved from thousands of individually trained convolutional neural network classifiers. We use a structured selection of visualization techniques to analyze the impact of different combinations of hyper-parameters. The results reveal how complicated dependencies between hyper-parameters influence the end performance, demonstrating how the complete picture painted by considering a large number of trainings simultaneously can aid in understanding the impact of hyper-parameter combinations.
- Published
- 2020
- Full Text
- View/download PDF