Search

Radial basis function interpolation (RBF) and Kriging based on the Gaussian processes models (GPs) are great tools for multidimensional surrogate modeling. They can often deliver accurate models for complicated nonlinear functions. Although RBFs and GPs have very different origins, they share many fundamentals in practice. Gaussian processes not only provide a model but also an error estimate associated with the model which indicates the uncertainty of the model at different points. The uncertainty property determined by GPs is dependent on the distribution of the sample points in the input space and as expected the model error estimate is low where the distribution is dense and large where the distribution is sparse. Therefore, the uncertainty property can be determined regardless of the modeling technique. In this work, we compare RBFs and GPs from the theoretical point of view. We show how to calculate the model uncertainty for any arbitrary kernel, e. g. cubic RBF, augmented cubic RBF and other types. Furthermore, we replace the Kriging model from the DiceKriging R package used in the SOCU framework [1] with a vectorized RBF model and report some preliminary results. We show that the new implementation of SOCU with RBF is faster than the older one. It delivers in many cases equal or even better optimization accuracy. [ABSTRACT FROM AUTHOR]

Modern real-world optimization problems are often high dimensional and subject to many constraints. These problems are typically expensive in terms of cost and computational time. Conventional constraint-based solvers often require a high number of function evaluations which are not affordable for such problems in practice. Employment of fast surrogate models to approximate objective and constraint functions is a known approach for efficient optimization. In this paper we present a new algorithm called Self-Adjusting COBRA (SACOBRA) based on Regis' COBRA [11]. We evaluate our approach by using 11 G-problems. We get very good results on 10 of the 11 G-problems with a severely limited budget of only 300 evaluations. [ABSTRACT FROM AUTHOR]

In real-world optimization often constraints must be respected, restricting the number of feasible solutions. Therefore algorithms and strategies have been proposed to repair constraint-violating solutions or to avoid extensive search in infeasible regions. Such constraint handling methods are well-known from the literature, but most algorithms have the drawback that they require a large number of function evaluations. This can be especially problematic for real-world optimization tasks, which often incorporate expensive simulations. Up to now, only little work has been devoted to efficient constraint-based optimization (severely reduced number of function evaluations). A possible solution in that regard is to use surrogate models for the objective and constraint functions respectively. While the real function might be expensive to evaluate the surrogate functions are often much faster. Recently, as an example for this approach, the solver COBRA was proposed and outperforms most other algorithms in terms of required function evaluations on a large number of benchmark functions. In this paper we propose a new implementation of COBRA and compare it with other constraint-based optimization algorithms. We discuss the internal components of the algorithm and find that by adding new strategies, the algorithm can be significantly improved. We also report on negative results where COBRA still shows a bad behaviour and gives indications for possible improvements. [ABSTRACT FROM AUTHOR]

Support Vector Machines (SVMs) have shown to be strong methods for classification problems. Especially for difficult tasks the performance of SVMs is often superior to other learning algorithms. A main issue arising with this kernel-based learning is the high computation time and also the large memory demand required for training with large data. As a solution to this, ensemble-based SVM approaches have recently been proposed. Meyer et al. [1] investigated SVM ensembles based on bagging [2] and Cascade SVMs [3]. Stork et al. [4, 5] proposed ensembles based on boosting [6] and bagging with subsampling of the training data. In their experimental study they observed that subsampling is a necessary ingredient to impede overfitting. Unfortunately no rule-of-thumb could be given for the sample size parameter. The goal of this study is to get a deeper understanding which elements in a fruitful combination of individuals in SVM ensembles lead to considerable time savings while maintaining a good classification accuracy. First, we expect to obtain an asymptotic behaviour when we increase the ensemble size for a fixed training set size setting. Secondly, we want to measure the influence of the training set size on the classification accuracy. With these findings we try to give recommendations for sample size and ensemble size in order to balance computation time and accuracy. As a nice side effect, the observations made in this study can be used to create ensembles of other learning algorithms as well. [ABSTRACT FROM AUTHOR]