Clustering and Ranking Based Methods for Selecting Tuned Search Heuristic Parameters

This paper presents two alternative methods for selecting tuned hyper-parameters of search heuristics for computationally expensive simulation-based optimization problems. The set of available hyper-parameters are obtained by conducting a meta-optimization to a number of mathematical test functions at different problem dimensions. Since a large number of tuned hyper-parameters is obtained from the meta-optimization, we develop methods to select tuned hyper-parameters for our actual simulation-based problems. The first selection method is based on clustering of the tuned hyper-parameters, where the medoids of the largest clusters are chosen. The second method constructs a ranking matrix that cross-checks the expected cost value for each tuned hyper-parameter configuration when solving the other training functions in the set, and that configuration is selected which has the minimal sum of ranks in the matrix. We apply our methods to custom particle swarm optimization and evolutionary algorithms. The feasibility of our approach is demonstrated by benchmarking the tuned search heuristics to a number of other search heuristics from open-source libraries on continuous building energy simulation problems. Our results show that our tuning and selection methods are successful in identifying hyper-parameters that result in competitive performance to other popular or recent algorihtms such as CMA-ES or RBFOpt. Future work should apply the methods developed to more recent search heuristics and make use of synthetic training functions in the meta-optimization that better resemble building simulation problems, instead of the mathematical test functions currently employed.