Search

This paper integrates an estimation of distribution (EoD)-based update operator into decomposition-based multiobjective evolutionary algorithms for binary optimization. The probabilistic model in the update operator is a probability vector, which is adaptively learned from historical information of each subproblem. We show that this update operator can significantly enhance decomposition-based algorithms on a number of benchmark problems. Moreover, we apply the enhanced algorithms to the constrained optimal software product selection (OSPS) problem in the field of search-based software engineering. For this real-world problem, we give its formal definition and then develop a new repair operator based on satisfiability solvers. It is demonstrated by the experimental results that the algorithms equipped with the EoD operator are effective in dealing with this practical problem, particularly for large-scale instances. The interdisciplinary studies in this paper provide a new real-world application scenario for constrained multiobjective binary optimizers and also offer valuable techniques for software engineers in handling the OSPS problem. [ABSTRACT FROM AUTHOR]

Benchmarking various metaheuristics and their new enhancements, strategies, and adaptation mechanisms has become standard in computational intelligence research. Recently, many challenges and issues regarding fair comparisons and recommendations toward good practices for benchmarking of metaheuristic algorithms, have been identified. This article is aimed at an important issues in metaheuristics design and benchmarking, which are boundary strategies or boundary control methods (BCMs). This work aims to investigate whether the choice of a BCM could significantly influence the performance of competitive algorithms. The experiments encompass the top three performing algorithms from IEEE CEC competitions 2017 and 2020 with six different BCMs. We provide extensive statistical analysis and rankings resulting in conclusions and recommendations for metaheuristics researchers and possibly also for the future direction of benchmark definitions. We conclude that the BCM should be considered another vital metaheuristics input variable for unambiguous reproducibility of results in benchmarking and for a better understanding of population dynamics, since the BCM setting could impact the optimization method performance. [ABSTRACT FROM AUTHOR]

Convergence and diversity are interdependently handled during the evolutionary process by most existing many-objective evolutionary algorithms (MaOEAs). In such a design, the degraded performance of one would deteriorate the other, and only solutions with both are able to improve the performance of MaOEAs. Unfortunately, it is not easy to constantly maintain a population of solutions with both convergence and diversity. In this paper, an MaOEA based on two independent stages is proposed for effectively solving many-objective optimization problems (MaOPs), where the convergence and diversity are addressed in two independent and sequential stages. To achieve this, we first propose a nondominated dynamic weight aggregation method by using a genetic algorithm, which is capable of finding the Pareto-optimal solutions for MaOPs with concave, convex, linear and even mixed Pareto front shapes, and then these solutions are employed to learn the Pareto-optimal subspace for the convergence. Afterward, the diversity is addressed by solving a set of single-objective optimization problems with reference lines within the learned Pareto-optimal subspace. To evaluate the performance of the proposed algorithm, a series of experiments are conducted against six state-of-the-art MaOEAs on benchmark test problems. The results show the significantly improved performance of the proposed algorithm over the peer competitors. In addition, the proposed algorithm can focus directly on a chosen part of the objective space if the preference area is known beforehand. Furthermore, the proposed algorithm can also be used to effectively find the nadir points. [ABSTRACT FROM AUTHOR]

Robust optimization over time is a new way to tackle dynamic optimization problems where the goal is to find solutions that remain acceptable over an extended period of time. The state-of-the-art methods in this domain try to identify robust solutions based on their future predicted fitness values. However, predicting future fitness values is difficult and error prone. In this paper, we propose a new framework based on a multipopulation method in which subpopulations are responsible for tracking peaks and also gathering characteristic information about them. When the quality of the current robust solution falls below the acceptance threshold, the algorithm chooses the next robust solution based on the collected information. We propose four different strategies to select the next solution. The experimental results on benchmark problems show that our newly proposed methods perform significantly better than existing algorithms. [ABSTRACT FROM AUTHOR]

Subset selection, aiming to select the best subset from a ground set with respect to some objective function, is a fundamental problem with applications in many areas, such as combinatorial optimization, machine learning, data mining, computer vision, information retrieval, etc. Along with the development of data collection and storage, the size of the ground set grows larger. Furthermore, in many subset selection applications, the objective function evaluation is subject to noise. We thus study the large-scale noisy subset selection problem in this paper. The recently proposed DPOSS algorithm based on multiobjective evolutionary optimization is a powerful distributed solver for large-scale subset selection. Its performance, however, has been only validated in the noise-free environment. In this paper, we first prove its approximation guarantee under two common noise models, i.e., multiplicative noise and additive noise, disclosing that the presence of noise degrades the performance of DPOSS largely. Next, we propose a new distributed multiobjective evolutionary algorithm called DPONSS for large-scale noisy subset selection. We prove that the approximation guarantee of DPONSS under noise is significantly better than that of DPOSS. We also conduct experiments on the application of sparse regression, where the objective evaluation is often estimated using a sample data, bringing noise. The results on various real-world data sets, whose size can reach millions, clearly show the excellent performance of DPONSS. [ABSTRACT FROM AUTHOR]

Phylogenetic inference allows building a hypothesis about the evolutionary relationships between a group of species, which is usually represented as a tree. The phylogenetic inference problem can be seen as an optimization problem, searching for the most qualified tree among all the possible topologies according to a selected criterion. These criteria can be based on different principles. Due to the combinatorial number of possible topologies, diverse heuristics and meta-heuristics have been proposed to find approximated solutions according to one criterion. However, these methods may result in several phylogeny trees which could be in conflict with one another. In order to deal with this problem, models based on multiobjective optimization with different configurations have been used. In this paper, we propose an ad-hoc multiobjective memetic algorithm (MO-MA) to infer phylogeny using two objectives: 1) maximum parsimony and 2) likelihood. Several population operators and local search strategies are proposed and evaluated in order to measure their contribution to the algorithm. Additionally, we perform a comparison among different configurations and tree rearrangement strategies. The results show that the proposed MO-MA is able to identify a Pareto set of solutions that include new trees which were nondominated by solutions from the current state of the art single-objective optimization tools. Furthermore, the MO-MA improves the results presented in the literature for multiobjective approaches in all of the studied data sets. These results make our proposal a good alternative for phylogenetic inference. [ABSTRACT FROM AUTHOR]

In multiobjective optimization, defining a good solution is a multifactored process. Most existing evolutionary multi- or many-objective optimization (EMO) algorithms have utilized two factors: 1) domination and 2) crowding levels of each solution. Although these two coarse-grained factors are found to be adequate in many EMO algorithms, their relative importance in an algorithm has been a matter of great concern to many current studies. We argue that beside these issues, other more fine-grained factors are of importance. For example, since extreme objective-wise solutions are important in establishing a noise-free and stable normalization process, reaching extreme solutions is more crucial than finding other solutions. In this paper, we propose an integrated algorithm, B-NSGA-III, that produces much better convergence and diversity preservation. For this purpose, in addition to emphasizing extreme objective-wise solutions, B-NSGA-III tries to find solutions near intermediate undiscovered regions of the front. B-NSGA-III addresses critical algorithmic issues of convergence and diversity-preservation directly through recent progresses in literature and integrates all these critical fine-grained factors seamlessly in an alternating phases scheme. The proposed algorithm is shown to perform better than a number of commonly used existing methods. [ABSTRACT FROM AUTHOR]

The first cooperative co-evolutionary algorithm (CCEA) was proposed by Potter and De Jong in 1994 and since then many CCEAs have been proposed and successfully applied to solving various complex optimization problems. In applying CCEAs, the complex optimization problem is decomposed into multiple subproblems, and each subproblem is solved with a separate subpopulation, evolved by an individual evolutionary algorithm (EA). Through cooperative co-evolution of multiple EA subpopulations, a complete problem solution is acquired by assembling the representative members from each subpopulation. The underlying divide-and-conquer and collaboration mechanisms enable CCEAs to tackle complex optimization problems efficiently, and hence CCEAs have been attracting wide attention in the EA community. This paper presents a comprehensive survey of these CCEAs, covering problem decomposition, collaborator selection, individual fitness evaluation, subproblem resource allocation, implementations, benchmark test problems, control parameters, theoretical analyses, and applications. The unsolved challenges and potential directions for their solutions are discussed. [ABSTRACT FROM AUTHOR]

In most real-world settings, designs are often gradually adapted and improved over time. Consequently, there exists knowledge from distinct (but possibly related) design exercises, which have either been previously completed or are currently in-progress, that may be leveraged to enhance the optimization performance of a particular target optimization task of interest. Further, it is observed that modern day design cycles are typically distributed in nature, and consist of multiple teams working on associated ideas in tandem. In such environments, vast amounts of related information can become available at various stages of the search process corresponding to some ongoing target optimization exercise. Successfully exploiting this knowledge is expected to be of significant value in many practical settings, where solving an optimization problem from scratch may be exorbitantly costly or time consuming. Accordingly, in this paper, we propose an adaptive knowledge reuse framework for surrogate-assisted multiobjective optimization of computationally expensive problems, based on the novel idea of multiproblem surrogates. This idea provides the capability to acquire and spontaneously transfer learned models across problems, facilitating efficient global optimization. The efficacy of our proposition is demonstrated on a series of synthetic benchmark functions, as well as two practical case studies. [ABSTRACT FROM AUTHOR]

Insurance has been increasingly realized as an important way of investment and risk aversion. Fruitful insurance products are launched by insurers, but there is little research on how to make a proper insurance investment plan for a specific policyholder given different kinds of policies. In this paper, we aim to propose a practical approach to multipolicy insurance investment planning with a data-driven model and an estimation of distribution algorithm (EDA). First, by making use of the insurance data accumulated in the modern financial market, an optimization model about how to choose endowment and hospitalization policies is built to maximize the yearly profit of insurance investment. With the model parameters set according to the real data from insurance market, the resulting plan is practical and individualized. Second, as the optimal solution cannot be achieved by mathematical deduction under this data-driven model, an EDA is introduced. To adapt the EDA for the considered problem, the proposed EDA is mixed with both the continuous and discrete probability distribution models to handle different kinds of variables. In addition, an adaptive scheme for choosing suitable distribution models and an efficient constraint handling strategy are proposed. Experiments under different conditions confirm the effectiveness and efficiency of the proposed model and method. [ABSTRACT FROM AUTHOR]

Dynamic interval multiobjective optimization problems (DI-MOPs) are very common in real-world applications. However, there are few evolutionary algorithms (EAs) that are suitable for tackling DI-MOPs up to date. A framework of dynamic interval multiobjective cooperative co-evolutionary optimization based on the interval similarity is presented in this paper to handle DI-MOPs. In the framework, a strategy for decomposing decision variables is first proposed, through which all the decision variables are divided into two groups according to the interval similarity between each decision variable and interval parameters. Following that, two subpopulations are utilized to cooperatively optimize decision variables in the two groups. Furthermore, two response strategies, i.e., a strategy based on the change intensity and a random mutation strategy, are employed to rapidly track the changing Pareto front of the optimization problem. The proposed algorithm is applied to eight benchmark optimization instances as well as a multiperiod portfolio selection problem and compared with five state-of-the-art EAs. The experimental results reveal that the proposed algorithm is very competitive on most optimization instances. [ABSTRACT FROM AUTHOR]

Recently, large-scale multiobjective optimization has received increasing attention from the evolutionary multiobjective optimization (EMO) community. This has led to the emergence of a specialized research area called evolutionary large-scale multiobjective optimization (ELMO). In general, it is believed that multiobjective optimization problems become more difficult as the number of decision variables increases. However, the following two counterintuitive observations are obtained from careful examinations of recent ELMO studies. One is that experimental results on some large-scale multiobjective test problems were improved by increasing the number of decision variables. The other is that better results were obtained for some other large-scale multiobjective test problems by conventional EMO algorithms (EMOAs) than state-of-the-art ELMO algorithms (ELMOAs). These observations suggest that ELMOAs have not always been evaluated on appropriate test problems. Moreover, their performance is not always better than the performance of conventional EMOAs. In this letter, we first re-examine the performance of ELMOAs and conventional EMOAs on a wide variety of scalable multiobjective test problems. Then, counterintuitive experimental results are analyzed using the anytime performance evaluation scheme and distributions of randomly generated initial solutions. Based on the analysis, suggestions on how to handle large-scale multiobjective test problems with counterintuitive results are proposed. [ABSTRACT FROM AUTHOR]

Traditionally, visualizing benchmark problems is an integral task in the domain of evolutionary algorithms development. Researchers get inspired for new search heuristics by challenges observed in functional landscapes. Moreover, landscape characteristics, features, and even terminology to describe them are derived from visualizations. And most importantly, benchmark designers need visualizations for identifying diverse problems that potentially challenge different aspects of optimization algorithms. As easy as it is to visualize single-objective problems, until recently there were hardly any approaches for gaining similar insights for multi-objective problems. Also, there have been no seamlessly accessible tools to support such visualizations. This article presents a comprehensive overview of the available visualization techniques from literature, including two interactive techniques to visualize 3-D problems, as well as two novel techniques which are suitable to scale some visualization properties to even higher-dimensional spaces. All presented techniques are integrated into a single tool, the moPLOT-dashboard, which enables users to perform landscape analyses in an interactive manner. Finally, the value of the tool and the visualizations is demonstrated in a series of usage scenarios on well-known benchmark problems. [ABSTRACT FROM AUTHOR]

Quality-diversity (QD) algorithms refer to a class of evolutionary algorithms designed to find a collection of diverse and high-performing solutions to a given problem. In robotics, such algorithms can be used for generating a collection of controllers covering most of the possible behaviors of a robot. To do so, these algorithms associate a behavioral descriptor (BD) to each of these behaviors. Each BD is used for estimating the novelty of one behavior compared to the others. In most existing algorithms, the BD needs to be hand-coded, thus requiring prior knowledge about the task to solve. In this article, we introduce: autonomous robots realizing their abilities, an algorithm that uses a dimensionality reduction technique to automatically learn BDs based on raw sensory data. The performance of this algorithm is assessed on three robotic tasks in simulation. The experimental results show that it performs similar to traditional hand-coded approaches without the requirement to provide any hand-coded BD. In the collection of diverse and high-performing solutions, it also manages to find behaviors that are novel with respect to more features than its hand-coded baselines. Finally, we introduce a variant of the algorithm which is robust to the dimensionality of the BD space. [ABSTRACT FROM AUTHOR]

Feature-based algorithm selection aims to automatically find the best one from a portfolio of optimization algorithms on an unseen problem based on its landscape features. Feature-based algorithm selection has recently received attention in the research field of black-box numerical optimization. However, there is still room for the analysis of algorithm selection for black-box optimization. Most previous studies have focused only on whether an algorithm selection system can outperform the single-best solver (SBS) in a portfolio. In addition, a benchmarking methodology for algorithm selection systems has not been well investigated in the literature. In this context, this article analyzes algorithm selection systems on the 24 noiseless black-box optimization benchmarking functions. First, we demonstrate that the first successful performance measure is more reliable than the expected runtime measure for benchmarking algorithm selection systems. Then, we examine the influence of randomness on the performance of algorithm selection systems. We also show that the performance of algorithm selection systems can be significantly improved by using sequential least squares programming as a presolver. We point out that the difficulty of outperforming the SBS depends on algorithm portfolios, cross-validation methods, and dimensions. Finally, we demonstrate that the effectiveness of algorithm portfolios depends on various factors. These findings provide fundamental insights for algorithm selection for black-box optimization. [ABSTRACT FROM AUTHOR]

This work assesses the efficacy of evolutionary algorithms (EAs) using an intuitive multidimensional scaling (MDS) visualization of the evolution of a population. We propose the use of landmark MDS (LMDS) to overcome computational challenges inherent to visualizing many-objective and complex problems with MDS. For the benchmark problems we tested, LMDS is akin to MDS visually, whilst requiring less than 1% of the time and memory necessary to produce an MDS visualization of the same objective space solutions, leading to the possibility of online visualizations for multi- and many-objective optimization evaluation. Using multi- and many-objective problems from the DTLZ and WFG benchmark test suites, we analyze how Landmark MDS visualizations can offer far greater insight into algorithm performance than using traditional algorithm performance metrics such as hypervolume alone, and can be used to complement explicit performance metrics. Ultimately, this visualization allows the visual identification of problem features and assists the decision maker in making intuitive recommendations for algorithm parameters/operators for creating and testing better EAs to solve multi- and many-objective problems. [ABSTRACT FROM AUTHOR]

We introduce an evolutionary algorithm called recombinator- $k$ -means for optimizing the highly nonconvex kmeans problem. Its defining feature is that its crossover step involves all the members of the current generation, stochastically recombining them with a repurposed variant of the $k$ -means++ seeding algorithm. The recombination also uses a reweighting mechanism that realizes a progressively sharper stochastic selection policy and ensures that the population eventually coalesces into a single solution. We compare this scheme with a state-of-the-art alternative, a more standard genetic algorithm with deterministic pairwise-nearest-neighbor crossover and an elitist selection policy, of which we also provide an augmented and efficient implementation. Extensive tests on large and challenging datasets (both synthetic and real word) show that for fixed population sizes recombinator- $k$ -means is generally superior in terms of the optimization objective, at the cost of a more expensive crossover step. When adjusting the population sizes of the two algorithms to match their running times, we find that for short times the (augmented) pairwise-nearest-neighbor method is always superior, while at longer times recombinator- $k$ -means will match it and, on the most difficult examples, take over. We conclude that the reweighted whole-population recombination is more costly but generally better at escaping local minima. Moreover, it is algorithmically simpler and more general (it could be applied even to $k$ -medians or $k$ -medoids, for example). Our implementations are publicly available at https://github.com/carlobaldassi/RecombinatorKMeans.jl. [ABSTRACT FROM AUTHOR]

Identification of variable interaction is essential for an efficient implementation of a divide-and-conquer algorithm for large-scale black-box optimization. In this paper, we propose an improved variant of the differential grouping (DG) algorithm, which has a better efficiency and grouping accuracy. The proposed algorithm, DG2, finds a reliable threshold value by estimating the magnitude of roundoff errors. With respect to efficiency, DG2 reuses the sample points that are generated for detecting interactions and saves up to half of the computational resources on fully separable functions. We mathematically show that the new sampling technique achieves the lower bound with respect to the number of function evaluations. Unlike its predecessor, DG2 checks all possible pairs of variables for interactions and has the capacity to identify overlapping components of an objective function. On the accuracy aspect, DG2 outperforms the state-of-the-art decomposition methods on the latest large-scale continuous optimization benchmark suites. DG2 also performs reliably in the presence of imbalance among contribution of components in an objective function. Another major advantage of DG2 is the automatic calculation of its threshold parameter ( $\epsilon $ ), which makes it parameter-free. Finally, the experimental results show that when DG2 is used within a cooperative co-evolutionary framework, it can generate competitive results as compared to several state-of-the-art algorithms. [ABSTRACT FROM PUBLISHER]

Surrogate-assisted evolutionary algorithms (EAs) have been intensively used to solve computationally expensive problems with some success. However, traditional EAs are not suitable to deal with high-dimensional expensive problems (HEPs) with high-dimensional search space even if their fitness evaluations are assisted by surrogate models. The recently proposed autoencoder-embedded evolutionary optimization (AEO) framework is highly appropriate to deal with high-dimensional problems. This work aims to incorporate surrogate models into it to further boost its performance, thus resulting in surrogate-assisted AEO (SAEO). It proposes a novel model management strategy that can guarantee reasonable amounts of re-evaluations; hence, the accuracy of surrogate models can be enhanced via being updated with new evaluated samples. Moreover, to ensure enough data samples before constructing surrogates, a problem-dimensionality-dependent activation condition is developed for incorporating surrogates into the SAEO framework. SAEO is tested on seven commonly used benchmark functions and compared with state-of-the-art algorithms for HEPs. The experimental results show that SAEO can further enhance the performance of AEO on most cases and SAEO performs significantly better than other algorithms. Therefore, SAEO has great potential to deal with HEPs. [ABSTRACT FROM AUTHOR]

Existing studies in black-box optimization suffer from low generalizability, caused by a typically selective choice of problem instances used for training and testing of different optimization algorithms. Among other issues, this practice promotes overfitting and poor-performing user guidelines. We address this shortcoming by introducing in this work a general-purpose algorithm selection wizard that was designed and tested on a previously unseen breadth of black-box optimization problems, ranging from academic benchmarks to real-world applications, from discrete over numerical to mixed-integer problems, from small to very large-scale problems, from noisy over dynamic to static problems, etc. Not only did we use the already very extensive benchmark environment available in Nevergrad, but we also extended it significantly by adding a number of additional benchmark suites, including Pyomo, Photonics, large-scale global optimization (LSGO), and MuJoCo. Our wizard achieves competitive performance on all benchmark suites. It significantly outperforms previous state-of-the-art algorithms on some of the suites, including YABBOB and LSGO. Its excellent performance is obtained without any task-specific parametrization. The algorithm selection wizard, all of its base solvers, as well as the benchmark suites are available for reproducible research in the open-source Nevergrad platform. [ABSTRACT FROM AUTHOR]

For complex system design [e.g., satellite layout optimization design (SLOD)] in practical engineering, when launching a new optimization instance with another parameter configuration from the intuition of designers, it is always executed from scratch which wastes much time to repeat the similar search process. Inspired by transfer learning which can reuse past experiences to solve relevant tasks, many researchers pay more attention to explore how to learn from past optimization instances to accelerate the target one. In real-world applications, there have been numerous similar source instances stored in the database. The primary question is how to measure the transferability from numerous sources to avoid the notorious negative transferring. To obtain the relatedness between source and target instance, we develop an optimization instance representation method named centroid distribution, which is by the aid of the probabilistic model learned by elite candidate solutions in estimation of distribution algorithm (EDA) during the evolutionary process. Wasserstein distance is employed to evaluate the similarity between the centroid distributions of different optimization instances, based on which, we present a novel framework called multisource selective transfer optimization with three strategies to select sources reasonably. To choose the suitable strategy, four selection suggestions are summarized according to the similarity between the source and target centroid distribution. The framework is beneficial to choose the most suitable sources, which could improve the search efficiency in solving multiobjective optimization problems. To evaluate the effectiveness of the proposed framework and selection suggestions, we conduct two experiments: 1) comprehensive empirical studies on complex multiobjective optimization problem benchmarks and 2) a real-world SLOD problem. Suggestions for strategy selection coincide with the experiment results, based on which, we propose a mixed strategy to deal with the negative transfer in the experiments successfully. The results demonstrate that our proposed framework achieves competitive performance on most of the benchmark problems in convergence speed and hypervolume values and performs best on the real-world applications among all the comparison algorithms. [ABSTRACT FROM AUTHOR]

This paper analyzes the stochastic runtime of the cross-entropy (CE) algorithm for the well-studied standard problems OneMax and LeadingOnes. We prove that the total number of solutions the algorithm needs to evaluate before reaching the optimal solution (i.e., its runtime) is bounded by a polynomial Q(n) in the problem size n with a probability growing exponentially to 1 with n if the parameters of the algorithm are adapted to n in a reasonable way. Our polynomial bound Q(n) for OneMax outperforms the well-known runtime bound of the 1-ANT algorithm, a particular ant colony optimization algorithm. Our adaptation of the parameters of the CE algorithm balances the number of iterations needed and the size of the samples drawn in each iteration, resulting in an increased efficiency. For the LeadingOnes problem, we improve the runtime of the algorithm by bounding the sampling probabilities away from 0 and 1. The resulting runtime outperforms the known stochastic runtime for a univariate marginal distribution algorithm, and is very close to the known expected runtime of variants of max-min ant systems. Bounding the sampling probabilities allows the CE algorithm to explore the search space even for test functions with a very rugged landscape as the LeadingOnes function. [ABSTRACT FROM PUBLISHER]

Over the past few decades, a plethora of computational intelligence algorithms designed to solve multiobjective problems have been proposed in the literature. Unfortunately, it has been shown that a large majority of these optimizers experience performance degradation when tasked with solving problems possessing more than three objectives, referred to as many-objective problems (MaOPs). The downfall of these optimizers is that simultaneously maintaining a uniformly-spread set of solutions along with appropriate selection pressure to converge toward the Pareto-optimal front becomes significantly difficult as the number of objectives increases. This difficulty is further compounded for large-scale MaOPs, i.e., MaOPs with a large number of decision variables. In this paper, insight is given into the current state of many-objective research by investigating scalability of state-of-the-art algorithms using 3–15 objectives and 30–1000 decision variables. Results indicate that evolutionary optimizers are generally the best performers when the number of decision variables is low, but are outperformed by the swarm intelligence optimizers in several large-scale MaOP instances. However, a recently proposed evolutionary algorithm which combines dominance and subregion-based decomposition is shown to be promising for handling the immense search spaces encountered in large-scale MaOPs. [ABSTRACT FROM PUBLISHER]

This paper addresses the problem of finding sparse solutions to linear systems. Although this problem involves two competing cost function terms (measurement error and a sparsity-inducing term), previous approaches combine these into a single cost term and solve the problem using conventional numerical optimization methods. In contrast, the main contribution of this paper is to use a multiobjective approach. The paper begins by investigating the sparse reconstruction problem, and presents data to show that knee regions do exist on the Pareto front (PF) for this problem and that optimal solutions can be found in these knee regions. Another contribution of the paper, a new soft-thresholding evolutionary multiobjective algorithm (StEMO), is then presented, which uses a soft-thresholding technique to incorporate two additional heuristics: one with greater chance to increase speed of convergence toward the PF, and another with higher probability to improve the spread of solutions along the PF, enabling an optimal solution to be found in the knee region. Experiments are presented, which show that StEMO significantly outperforms five other well known techniques that are commonly used for sparse reconstruction. Practical applications are also demonstrated to fundamental problems of recovering signals and images from noisy data. [ABSTRACT FROM AUTHOR]

Multipopulation methods are effective in solving dynamic optimization problems. However, to efficiently track multiple optima, algorithm designers need to address a key issue: how to adapt the number of populations. In this paper, an adaptive multipopulation framework is proposed to address this issue. A database is designed to collect heuristic information of algorithm behavior changes. The number of populations is adjusted according to statistical information related to the current evolving status in the database and a heuristic value. Several other techniques are also introduced, including a heuristic clustering method, a population exclusion scheme, a population hibernation scheme, two movement schemes, and a peak hiding method. The particle swarm optimization and differential evolution algorithms are implemented into the framework, respectively. A set of multipopulation-based algorithms are chosen to compare with the proposed algorithms on the moving peaks benchmark using four different performance measures. The effect of the components of the framework is also investigated based on a set of multimodal problems in static environments. Experimental results show that the proposed algorithms outperform the other algorithms in most scenarios. [ABSTRACT FROM AUTHOR]

Constrained optimization problems (COPs) are frequently encountered in real-world design applications. For some COPs, the evaluation of the objective(s) and/or constraint(s) may involve significant computational/temporal/financial cost. Such problems are referred to as expensive COPs (ECOPs). Surrogate modeling has been widely used in conjunction with optimization methods for such problems, wherein the search is partially driven by an approximate function instead of true expensive evaluations. However, for any true evaluation, nearly all existing methods compute all objective and constraint values together as one batch. Such full evaluation approaches may be inefficient for cases where the objective/constraint(s) can be evaluated independently of each other. In this article, we propose and study a constraint handling strategy for ECOPs using partial evaluations. The constraints are evaluated in a sequence determined based on their likelihood of being violated; and the evaluation is aborted if a constraint violation is encountered. Modified ranking strategies are introduced to effectively rank the solutions using the limited information thus obtained, while saving on significant function evaluations. The proposed algorithm is compared with a number of its variants to establish the utility of its key components systematically. Numerical experiments and benchmarking are conducted on a range of mathematical and engineering design problems to demonstrate the efficacy of the approach compared to state-of-the-art evolutionary optimization approaches. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) have been a popular deep generative model for real-world applications. Despite many recent efforts on GANs that have been contributed, mode collapse and instability of GANs are still open problems caused by their adversarial optimization difficulties. In this article, motivated by the cooperative co-evolutionary algorithm, we propose a cooperative dual evolution-based GAN (CDE-GAN) to circumvent these drawbacks. In essence, CDE-GAN incorporates dual evolution with respect to the generator(s) and discriminators into a unified evolutionary adversarial framework to conduct effective adversarial multiobjective optimization. Thus, it exploits the complementary properties and injects dual mutation diversity into the training, to steadily diversify the estimated density in capturing multimodes and improve generative performance. Specifically, CDE-GAN decomposes the complex adversarial optimization problem into two subproblems (generation and discrimination), and each subproblem is solved with a separated subpopulation (E-Generators and E-Discriminators), evolved by its own evolutionary algorithm. Additionally, we further propose a Soft Mechanism to balance the tradeoff between E-Generators and E-Discriminators to conduct steady training for CDE-GAN. Extensive experiments on one synthetic dataset and three real-world benchmark image datasets demonstrate that the proposed CDE-GAN achieves a competitive and superior performance in generating good quality and diverse samples over baselines. The code and more generated results are available at our project homepage https://shiming-chen.github.io/CDE-GAN-website/CDE-GAN.html. [ABSTRACT FROM AUTHOR]

Kriging models, also known as Gaussian process models, are widely used in surrogate-assisted evolutionary algorithms (SAEAs). However, the cubic time complexity of the standard Kriging models limits their usage in high-dimensional optimization. To tackle this problem, we propose an incremental Kriging model for high-dimensional surrogate-assisted evolutionary computation. The main idea is to update the Kriging model incrementally based on the equations of the previously trained model instead of building the model from scratch when new samples arrive, so that the time complexity of updating the Kriging models can be reduced to quadratic. The proposed incremental learning scheme is very suitable for online SAEAs since they evaluate new samples in each one or several generations. The proposed algorithm is able to achieve competitive optimization results on the test problems compared with the standard Kriging-assisted evolutionary algorithm and is significantly faster than the standard Kriging approach. The proposed algorithm also shows competitive or better performances compared with four fast Kriging-assisted evolutionary algorithms and four state-of-the-art SAEAs. This work provides a fast way of employing Kriging models in high-dimensional surrogate-assisted evolutionary computation. [ABSTRACT FROM AUTHOR]

This article investigates deep neural networks (DNNs)-based lung nodule classification with hyperparameter optimization. Hyperparameter optimization in DNNs is a computationally expensive problem, and a surrogate-assisted evolutionary algorithm has been recently introduced to automatically search for optimal hyperparameter configurations of DNNs, by applying computationally efficient surrogate models to approximate the validation error function of hyperparameter configurations. Different from existing surrogate models adopting stationary covariance functions (kernels) to measure the difference between hyperparameter points, this article proposes a nonstationary kernel that allows the surrogate model to adapt to functions whose smoothness varies with the spatial location of inputs. A multilevel convolutional neural network (ML-CNN) is built for lung nodule classification, and the hyperparameter configuration is optimized by the proposed nonstationary kernel-based Gaussian surrogate model. Our algorithm searches with a surrogate for optimal setting via a hyperparameter importance-based evolutionary strategy, and the experiments demonstrate our algorithm outperforms manual tuning and several well-established hyperparameter optimization methods, including random search, grid search, the tree-structured parzen estimator (TPE) approach, Gaussian processes (GP) with stationary kernels, and the recently proposed hyperparameter optimization via RBF and dynamic (HORD) coordinate search. [ABSTRACT FROM AUTHOR]

Differential evolution (DE) algorithms compose an efficient type of evolutionary algorithm (EA) for the global optimization domain. Although it is well known that the population structure has a major influence on the behavior of EAs, there are few works studying its effect in DE algorithms. In this paper, we propose and analyze several DE variants using different panmictic and decentralized population schemes. As it happens for other EAs, we demonstrate that the population scheme has a marked influence on the behavior of DE algorithms too. Additionally, a new operator for generating the mutant vector is proposed and compared versus a classical one on all the proposed population models. After that, a new heterogeneous decentralized DE algorithm combining the two studied operators in the best performing studied population structure has been designed and evaluated. In total, 13 new DE algorithms are presented and evaluated in this paper. Summarizing our results, all the studied algorithms are highly competitive compared to the state-of-the-art DE algorithms taken from the literature for most considered problems, and the best ones implement a decentralized population. With respect to the population structure, the proposed decentralized versions clearly provide a better performance compared to the panmictic ones. The new mutation operator demonstrates a faster convergence on most of the studied problems versus a classical operator taken from the DE literature. Finally, the new heterogeneous decentralized DE is shown to improve the previously obtained results, and outperform the compared state-of-the-art DEs. [ABSTRACT FROM PUBLISHER]

Having developed multiobjective optimization algorithms using evolutionary optimization methods and demonstrated their niche on various practical problems involving mostly two and three objectives, there is now a growing need for developing evolutionary multiobjective optimization (EMO) algorithms for handling many-objective (having four or more objectives) optimization problems. In this paper, we recognize a few recent efforts and discuss a number of viable directions for developing a potential EMO algorithm for solving many-objective optimization problems. Thereafter, we suggest a reference-point-based many-objective evolutionary algorithm following NSGA-II framework (we call it NSGA-III) that emphasizes population members that are nondominated, yet close to a set of supplied reference points. The proposed NSGA-III is applied to a number of many-objective test problems with three to 15 objectives and compared with two versions of a recently suggested EMO algorithm (MOEA/D). While each of the two MOEA/D methods works well on different classes of problems, the proposed NSGA-III is found to produce satisfactory results on all problems considered in this paper. This paper presents results on unconstrained problems, and the sequel paper considers constrained and other specialties in handling many-objective optimization problems. [ABSTRACT FROM AUTHOR]

This article proposes an evolutionary algorithm using multiple penalties and multiple local surrogates (MPMLS) for expensive constrained optimization. In each generation, MPMLS defines and optimizes a number of subproblems. Each subproblem penalizes the constraints in the original problem using a different penalty coefficient and has its own search subregion. A local surrogate is built for optimizing each subproblem. Two major advantages of MPMLS are: 1) it can maintain good population diversity so that the search can approach the optimal solution of the original problem from different directions and 2) it only needs to build local surrogates so that the computational overhead of the model building can be reduced. Numerical experiments demonstrate that our proposed algorithm performs much better than some other state-of-the-art evolutionary algorithms. [ABSTRACT FROM AUTHOR]

In recent years, numerous efficient and effective multimodal multiobjective evolutionary algorithms (MMOEAs) have been developed to search for multiple equivalent sets of Pareto optimal solutions simultaneously. However, some of the MMOEAs prefer convergent individuals over diversified individuals to construct the mating pool, and the individuals with slightly better decision space distribution may be replaced by significantly better objective space distribution. Therefore, the diversity in the decision space may become deteriorated, in spite of the decision and objective diversities have been taken into account simultaneously in most MMOEAs. Because the Pareto optimal subsets may have various shapes and locations in the decision space, it is very difficult to drive the individuals converged to every Pareto subregion with a uniform density. Some of the Pareto subregions may be overly crowded, while others are rather sparsely distributed. Consequently, many existing MMOEAs obtain Pareto subregions with imbalanced density. In this article, we present a two-stage double niched evolution strategy, namely DN-MMOES, to search for the equivalent global Pareto optimal solutions which can address the above challenges effectively and efficiently. The proposed DN-MMOES solves the multimodal multiobjective optimization problem (MMOP) in two stages. The first stage adopts the niching strategy in the decision space, while the second stage adapts double niching strategy in both spaces. Moreover, an effective decision density self-adaptive strategy is designed for improving the imbalanced decision space density. The proposed algorithm is compared against eight state-of-the-art MMOEAs. The inverted generational distance union (IGDunion) performance indicator is proposed to fairly compare two competing MMOEAs as a whole. The experimental results show that DN-MMOES provides a better performance to search for the complete Pareto Subsets and Pareto Front on IDMP and CEC 2019 MMOPs test suite. [ABSTRACT FROM AUTHOR]

It is particularly challenging for evolutionary algorithms to quickly converge to the Pareto front in large-scale multiobjective optimization. To tackle this problem, this article proposes a large-scale multiobjective evolutionary algorithm assisted by some selected individuals generated by directed sampling (DS). At each generation, a set of individuals closer to the ideal point is chosen for performing a DS in the decision space, and those nondominated ones of the sampled solutions are used to assist the reproduction to improve the convergence in evolutionary large-scale multiobjective optimization. In addition, elitist nondominated sorting is adopted complementarily for environmental selection with a reference vector-based method in order to maintain diversity of the population. Our experimental results show that the proposed algorithm is highly competitive on large-scale multiobjective optimization test problems with up to 5000 decision variables compared to five state-of-the-art multiobjective evolutionary algorithms. [ABSTRACT FROM AUTHOR]

The discrete cross-entropy optimization algorithm iteratively samples solutions according to a probability density on the solution space. The density is adapted to the good solutions observed in the present sample before producing the next sample. The adaptation is controlled by a so-called smoothing parameter. We generalize this model by introducing a flexible concept of feasibility and desirability into the sampling process. In this way, our model covers several other optimization procedures, in particular the ant-based algorithms. The focus of this paper is on some theoretical properties of these algorithms. We examine the first hitting time \( \tau \) of an optimal solution and give conditions on the smoothing parameter for \( \tau \) to be finite with probability one. For a simple test case we show that runtime can be polynomially bounded in the problem size with a probability converging to 1. We then investigate the convergence of the underlying density and of the sampling process. We show, in particular, that a constant smoothing parameter, as it is often used, makes the sample process converge in finite time, freezing the optimization at a single solution that need not be optimal. Moreover, we define a smoothing sequence that makes the density converge without freezing the sample process and that still guarantees the reachability of optimal solutions in finite time. This settles an open question from the literature. [ABSTRACT FROM AUTHOR]

In classification, machine learning algorithms can suffer a performance bias when data sets are unbalanced. Data sets are unbalanced when at least one class is represented by only a small number of training examples (called the minority class), while the other class(es) make up the majority. In this scenario, classifiers can have good accuracy on the majority class, but very poor accuracy on the minority class(es). This paper proposes a multiobjective genetic programming (MOGP) approach to evolving accurate and diverse ensembles of genetic program classifiers with good performance on both the minority and majority of classes. The evolved ensembles comprise of nondominated solutions in the population where individual members vote on class membership. This paper evaluates the effectiveness of two popular Pareto-based fitness strategies in the MOGP algorithm (SPEA2 and NSGAII), and investigates techniques to encourage diversity between solutions in the evolved ensembles. Experimental results on six (binary) class imbalance problems show that the evolved ensembles outperform their individual members, as well as single-predictor methods such as canonical GP, naive Bayes, and support vector machines, on highly unbalanced tasks. This highlights the importance of developing an effective fitness evaluation strategy in the underlying MOGP algorithm to evolve good ensemble members. [ABSTRACT FROM AUTHOR]

The difficulties faced by existing multiobjective evolutionary algorithms (MOEAs) in handling many-objective problems relate to the inefficiency of selection operators, high computational cost, and difficulty in visualization of objective space. While many approaches aim to counter these difficulties by increasing the fidelity of the standard selection operators, the objective reduction approach attempts to eliminate objectives that are not essential to describe the Pareto-optimal front (POF). If the number of essential objectives is found to be two or three, the problem could be solved by the existing MOEAs. It implies that objective reduction could make an otherwise unsolvable (many-objective) problem solvable. Even when the essential objectives are four or more, the reduced representation of the problem will have favorable impact on the search efficiency, computational cost, and decision-making. Hence, development of generic and robust objective reduction approaches becomes important. This paper presents a principal component analysis and maximum variance unfolding based framework for linear and nonlinear objective reduction algorithms, respectively. The major contribution of this paper includes: 1) the enhancements in the core components of the framework for higher robustness in terms of applicability to a range of problems with disparate degree of redundancy; mechanisms to handle input data that poorly approximates the true POF; and dependence on fewer parameters to minimize the variability in performance; 2) proposition of an error measure to assess the quality of results; 3) sensitivity analysis of the proposed algorithms for the critical parameter involved, and the characteristics of the input data; and 4) study of the performance of the proposed algorithms vis-à-vis dominance relation preservation based algorithms, on a wide range of test problems (scaled up to 50 objectives) and two real-world problems. [ABSTRACT FROM AUTHOR]

Multiobjective optimization problems are problems with several, typically conflicting, criteria for evaluating solutions. Without any a priori preference information, the Pareto optimality principle establishes a partial order among solutions, and the output of the algorithm becomes a set of nondominated solutions rather than a single one. Various ant colony optimization (ACO) algorithms have been proposed in recent years for solving such problems. These multiobjective ACO (MOACO) algorithms exhibit different design choices for dealing with the particularities of the multiobjective context. This paper proposes a formulation of algorithmic components that suffices to describe most MOACO algorithms proposed so far. This formulation also shows that existing MOACO algorithms often share equivalent design choices, but they are described in different terms. Moreover, this formulation is synthesized into a flexible algorithmic framework, from which not only existing MOACO algorithms may be instantiated, but also combinations of components that were never studied in the literature. In this sense, this paper goes beyond proposing a new MOACO algorithm, but it rather introduces a family of MOACO algorithms. The flexibility of the proposed MOACO framework facilitates the application of automatic algorithm configuration techniques. The experimental results presented in this paper show that the automatically configured MOACO framework outperforms the MOACO algorithms that inspired the framework itself. This paper is also among the first to apply automatic algorithm configuration techniques to multiobjective algorithms. [ABSTRACT FROM PUBLISHER]

Constrained multiobjective optimization problems (CMOPs) widely exist in real-world applications, and they are challenging for conventional evolutionary algorithms (EAs) due to the existence of multiple constraints and objectives. When the number of objectives or decision variables is scaled up in CMOPs, the performance of EAs may degenerate dramatically and may fail to obtain any feasible solutions. To address this issue, we propose a paired offspring generation-based multiobjective EA for constrained large-scale optimization. The general idea is to emphasize the role of offspring generation in reproducing some promising feasible or useful infeasible offspring solutions. We first adopt a small set of reference vectors for constructing several subpopulations with a fixed number of neighborhood solutions. Then, a pairing strategy is adopted to determine some pairwise parent solutions for offspring generation. Consequently, the pairwise parent solutions, which could be infeasible, may guide the generation of well-converged solutions to cross the infeasible region(s) effectively. The proposed algorithm is evaluated on CMOPs with up to 1000 decision variables and ten objectives. Moreover, each component in the proposed algorithm is examined in terms of its effect on the overall algorithmic performance. Experimental results on a variety of existing and our tailored test problems demonstrate the effectiveness of the proposed algorithm in constrained large-scale multiobjective optimization. [ABSTRACT FROM AUTHOR]

Multimodal multiobjective optimization problems (MMOPs) widely exist in real-world applications, which have multiple equivalent Pareto-optimal solutions that are similar in the objective space but totally different in the decision space. While some evolutionary algorithms (EAs) have been developed to find the equivalent Pareto-optimal solutions in recent years, they are ineffective to handle large-scale MMOPs having a large number of variables. This article thus proposes an EA for solving large-scale MMOPs with sparse Pareto-optimal solutions, i.e., most variables in the optimal solutions are 0. The proposed algorithm explores different regions of the decision space via multiple subpopulations and guides the search behavior of the subpopulations via adaptively updated guiding vectors. The guiding vector for each subpopulation not only provides efficient convergence in the huge search space but also differentiates its search direction from others to handle the multimodality. While most existing EAs solve MMOPs with 2–7 decision variables, the proposed algorithm is shown to be effective for benchmark MMOPs with up to 500 decision variables. Moreover, the proposed algorithm also produces a better result than state-of-the-art methods for the neural architecture search. [ABSTRACT FROM AUTHOR]

To handle different types of many-objective optimization problems (MaOPs), many-objective evolutionary algorithms (MaOEAs) need to simultaneously maintain convergence and population diversity in the high-dimensional objective space. In order to balance the relationship between diversity and convergence, we introduce a Kernel matrix and probability model called determinantal point processes (DPPs). Our MaOEA with DPPs (MaOEADPPs) is presented and compared with several state-of-the-art algorithms on various types of MaOPs with different numbers of objectives. The experimental results demonstrate that MaOEADPPs is competitive. [ABSTRACT FROM AUTHOR]

The radio network design (RND) is an NP-hard optimization problem which consists of the maximization the coverage of a given area while minimizing the base station deployment. Solving RND problems efficiently is relevant to many fields of application and has a direct impact in the engineering, telecommunication, scientific, and industrial areas. Numerous works can be found in the literature dealing with the RND problem, although they all suffer from the same shortfall: noncomparable efficiency. Therefore, the aim of this paper twofold: first, to offer a reliable RND comparison base reference in order to cover a wide algorithmic spectrum, and, second, to offer a comprehensible insight into accurate comparisons efficiency, reliability, and swiftness of the different techniques applied to solve the RND problem. In order to achieve first aim we propose a canonical RND problem formulation driven by two main directives: technology independence a normalized comparison criterion. Following this, we have included an exhaustive behavior comparison between 14 different techniques. Finally, this paper indicates algorithmic trends and different patterns that can be observed through this analysis. [ABSTRACT FROM AUTHOR]

This paper proposes a new multiobjective estimation of distribution algorithm (EDA) based on joint probabilistic modeling of objectives and variables. This EDA uses the multidimensional Bayesian network as its probabilistic model. In this way, it can capture the dependencies between objectives, variables and objectives, as well as the dependencies learned between variables in other Bayesian network-based EDAs. This model leads to a problem decomposition that helps the proposed algorithm find better tradeoff solutions to the multiobjective problem. In addition to Pareto set approximation, the algorithm is also able to estimate the structure of the multiobjective problem. To apply the algorithm to many-objective problems, the algorithm includes four different ranking methods proposed in the literature for this purpose. The algorithm is first applied to the set of walking fish group problems, and its optimization performance is compared with a standard multiobjective evolutionary algorithm and another competitive multiobjective EDA. The experimental results show that on several of these problems, and for different objective space dimensions, the proposed algorithm performs significantly better and on some others achieves comparable results when compared with the other two algorithms. The algorithm is then tested on the set of CEC09 problems, where the results show that multiobjective optimization based on joint model estimation is able to obtain considerably better fronts for some of the problems compared with the search based on conventional genetic operators in the state-of-the-art multiobjective evolutionary algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we present a new algorithm for calculating exact hypervolumes. Given a set of $d$ -dimensional points, it computes the hypervolume of the dominated space. Determining this value is an important subroutine of multiobjective evolutionary algorithms. We analyze the quick hypervolume (QHV) algorithm theoretically and experimentally. The theoretical results are a significant contribution to the current state of the art. Moreover, the experimental performance is also very competitive, compared with existing exact hypervolume algorithms. [ABSTRACT FROM AUTHOR]

Most evolutionary many-objective optimization (EMaO) algorithms start with a description of a number of the predefined set of reference points on a unit simplex. So far, most studies have used the Das and Dennis’s structured approach for generating well-spaced reference points. Due to the highly structured nature of the procedure, this method cannot produce an arbitrary number of points, which is desired in an EMaO application. Although a layer-wise implementation has been suggested, EMO researchers always felt the need for a more generic approach. Motivated by earlier studies, we introduce a metric for defining well-spaced points on a unit simplex and propose a number of viable methods for generating such a set. We compare the proposed methods on a variety of performance metrics such as hypervolume (HV), deviation in triangularized simplices, distance of the closest point pair, and variance of the geometric means to nearest neighbors in up to 15-D spaces. We show that an iterative improvement based on Riesz $s$ -energy is able to effectively find an arbitrary number of well-spaced points even in higher-dimensional spaces. Reference points created using the proposed Riesz $s$ -energy method for a number of standard combinations of objectives and reference points as well as a source code written in Python are available publicly at https://www.egr.msu.edu/coinlab/blankjul/uniform. [ABSTRACT FROM AUTHOR]

We propose a new black-box complexity model for search algorithms evaluating $\lambda $ search points in parallel. The parallel unary unbiased black-box complexity gives lower bounds on the number of function evaluations every parallel unary unbiased black-box algorithm needs to optimize a given problem. It captures the inertia caused by offspring populations in evolutionary algorithms and the total computational effort in parallel metaheuristics. We present complexity results for LeadingOnes and OneMax. Our main result is a general performance limit: we prove that on every function every $\lambda $ -parallel unary unbiased algorithm needs at least a certain number of evaluations (a function of problem size and $\lambda $) to find any desired target set of up to exponential size, with an overwhelming probability. This yields lower bounds for the typical optimization time on unimodal and multimodal problems, for the time to find any local optimum, and for the time to even get close to any optimum. The power and versatility of this approach is shown for a wide range of illustrative problems from combinatorial optimization. Our performance limits can guide parameter choice and algorithm design; we demonstrate the latter by presenting an optimal $\lambda $ -parallel algorithm for OneMax that uses parallelism most effectively. 1 This article significantly extends preliminary results which appeared in. [ABSTRACT FROM AUTHOR]

The ultimate goal of multiobjective optimization is to help a decision maker (DM) identify solution(s) of interest (SOI) achieving satisfactory tradeoffs among multiple conflicting criteria. This can be realized by leveraging DM’s preference information in evolutionary multiobjective optimization (EMO). No consensus has been reached on the effectiveness brought by incorporating preference in EMO (either a priori or interactively) versus a posteriori decision making after a complete run of an EMO algorithm. Bearing this consideration in mind, this article: 1) provides a pragmatic overview of the existing developments of preference-based EMO (PBEMO) and 2) conducts a series of experiments to investigate the effectiveness brought by preference incorporation in EMO for approximating various SOI. In particular, the DM’s preference information is elicited as a reference point, which represents her/his aspirations for different objectives. The experimental results demonstrate that preference incorporation in EMO does not always lead to a desirable approximation of SOI if the DM’s preference information is not well utilized, nor does the DM elicit invalid preference information, which is not uncommon when encountering a black-box system. To a certain extent, this issue can be remedied through an interactive preference elicitation. Last but not the least, we find that a PBEMO algorithm is able to be generalized to approximate the whole PF given an appropriate setup of preference information. [ABSTRACT FROM AUTHOR]

We theoretically analyze the information geometric optimization (IGO), which is a unified framework of stochastic search algorithms for black-box optimization. The IGO framework has two parameters: 1) the learning rate and 2) the sample size, and they influence the behavior of the algorithm. We investigate the strategy parameters of the IGO with the family of isotropic Gaussian distributions on a general convex quadratic function. Compared to the previous theoretical works, where an infinite sample size is assumed and the deterministic algorithm dynamics is studied, we investigate the expected improvement of the algorithm with a finite sample size. The analysis finds that the relative decrease rates of the distance from the distribution mean to the landscape optimum and the distribution standard deviation must be the same, which we observe in practice, while the analysis based on an infinite sample size failed to obtain. We derive these rates explicitly as a function of the eigenvalues of the Hessian of the objective function and the strategy parameters. We also derive the stable value of the ratio of the square distance to the optimum over the distribution variance, as well as the conditions that the stable value exists. These theoretical values coincide with our numerical simulations. [ABSTRACT FROM AUTHOR]

By efficiently building and exploiting surrogates, data-driven evolutionary algorithms (DDEAs) can be very helpful in solving expensive and computationally intensive problems. However, they still often suffer from two difficulties. First, many existing methods for building a single ad hoc surrogate are suitable for some special problems but may not work well on some other problems. Second, the optimization accuracy of DDEAs deteriorates if available data are not enough for building accurate surrogates, which is common in expensive optimization problems. To this end, this article proposes a novel DDEA with two efficient components. First, a boosting strategy (BS) is proposed for self-aware model managements, which can iteratively build and combine surrogates to obtain suitable surrogate models for different problems. Second, a localized data generation (LDG) method is proposed to generate synthetic data to alleviate data shortage and increase data quantity, which is achieved by approximating fitness through data positions. By integrating the BS and the LDG, the BDDEA-LDG algorithm is able to improve model accuracy and data quantity at the same time automatically according to the problems at hand. Besides, a tradeoff is empirically considered to strike a better balance between the effectiveness of surrogates and the time cost for building them. The experimental results show that the proposed BDDEA-LDG algorithm can generally outperform both traditional methods without surrogates and other state-of-the-art DDEA son widely used benchmarks and an arterial traffic signal timing real-world optimization problem. Furthermore, the proposed BDDEA-LDG algorithm can use only about 2% computational budgets of traditional methods for producing competitive results. [ABSTRACT FROM AUTHOR]

In this article, a new hypervolume-based evolutionary multiobjective optimization algorithm (EMOA), namely, R2HCA-EMOA (R2-based hypervolume contribution approximation EMOA), is proposed for many-objective optimization. The core idea of the algorithm is to use an R2 indicator variant to approximate the hypervolume contribution. The basic framework of the proposed algorithm is the same as SMS-EMOA. In order to make the algorithm computationally efficient, a utility tensor structure is introduced for the calculation of the R2 indicator variant. Moreover, a normalization mechanism is incorporated into R2HCA-EMOA to enhance the performance of the algorithm. Through experimental studies, R2HCA-EMOA is compared with three hypervolume-based EMOAs and several other state-of-the-art EMOAs on 5-, 10-, and 15-objective DTLZ, WFG problems, and their minus versions. Our results show that R2HCA-EMOA is more efficient than the other hypervolume-based EMOAs, and is superior to all the compared state-of-the-art EMOAs. [ABSTRACT FROM AUTHOR]