Search

For several real-world optimization problems, the evaluation of response functions may be expensive, computationally or otherwise. The number of design evaluations one can afford for such problems are therefore severely limited. Surrogate models are commonly used to guide the search for such computationally expensive optimization problems (CEOP). The surrogate models built using a limited number of true evaluations are used to identify the next infill/sampling location. Expected improvement (EI) is a well known infill criteria which balances exploration and exploitation by accounting for both mean and uncertainties in the current model. However, recent studies have shown that, somewhat counter-intuitively, a greedy ("believer") strategy can compete well with EI in solving CEOPs. In this study, we are interested in examining the relative performance of the two infill methods across a range of problems, and identify the influencing factors affecting their performance. Based on the empirical analysis, we further propose an algorithm incorporating the strengths of the two strategies. The numerical experiments demonstrate that the proposed algorithm is able to achieve a competitive performance across a range of problems with diverse characteristics; making it a strong candidate for solving black-box CEOPs.

Previous theory work on multi-objective evolutionary algorithms considers mostly easy problems that are composed of unimodal objectives. This paper takes a first step towards a deeper understanding of how evolutionary algorithms solve multi-modal multi-objective problems. We propose the OneJumpZeroJump problem, a bi-objective problem with single objectives isomorphic to the classic jump function benchmark. We prove that the simple evolutionary multi-objective optimizer (SEMO) cannot compute the full Pareto front. In contrast, for all problem sizes n and all jump sizes [EQUATION], the global SEMO (GSEMO) covers the Pareto front in Θ((n-2k)nk) iterations in expectation. To improve the performance, we combine the GSEMO with two approaches, a heavy-tailed mutation operator and a stagnation detection strategy, that showed advantages in single-objective multi-modal problems. Runtime improvements of asymptotic order at least kΩ(k) are shown for both strategies. Our experiments verify the substantial runtime gains already for moderate problem sizes. Overall, these results show that the ideas recently developed for single-objective evolutionary algorithms can be effectively employed also in multi-objective optimization. This Hot-off-the-Press paper summarizes "Theoretical Analyses of Multi-Objective Evolutionary Algorithms on Multi-Modal Objectives" by B. Doerr and W. Zheng, which has been accepted for publication in AAAI 2021 [9].

In the era of the big data and e-science revolution, the execution of such applications known as High Performance Computing (HPC) is becoming a challenging issue. In order to face these challenges, a new promising Large Scale Distributed Systems (LSDS) has emerged suchlike Grid and Cloud Computing. As a matter of fact, these HPC applications are commonly arranged as a form of interdependent tasks named workflows. Nevertheless, the new challenging topic is that the scheduling of these scientific workflows in the LSDS is a well-known NP-hard problem. The goal of this work is to design an Non-dominated Sorting Genetic Algorithm Version II (NSGA-II)-based approach for optimizing a multi objective scheduling of scientific workflows in hybrid distributed systems. This paper work deals with the proposition of two execution models: i) A Cumulative one aiming to improve the Pareto front quality in term of Makespan-Cost trade-off; ii) An Incremental execution fashion, what kind of Cost-driven approach leading to a solution diversity of the Pareto front in the objective space. Experiments conducted with multiple common scientific workflows point out significant improvement against the classic NSGA-II algorithm.

In this paper, we first explain why many-objective problems are difficult for Pareto dominance-based evolutionary multiobjective optimization algorithms such as NSGA-II and SPEA. Then we explain recent proposals for the handling of many-objective problems by evolutionary algorithms. Some proposals are examined through computational experiments on multiobjective knapsack problems with two, four and six objectives. Finally we discuss the viability of many-objective genetic fuzzy systems (i.e., the use of many-objective genetic algorithms for the design of fuzzy rule-based systems).

Previous work presented a technique called evolving self-taught neural networks - neural networks that can teach themselves, intrinsically motivated, without external supervision or reward [3]. In an autonomous multi-agent setting in which the agent is primitively set to know little or nothing about its environment, self-teaching was shown to give rise to intelligence, whereas an evolutionary algorithm alone fails since it has no way to search without gradient information. In this paper, we conduct another comparative experiment in which the foraging agent is built more conscious of its environment beforehand. Experimental results show that the more conscious primitive design can let evolution alone be able to search. Yet the combination of evolution and self-teaching still outperforms the alternative. Indications for future work on evolving intelligence are also presented.

Inferring models of dynamic systems from their time series data is a challenging task for optimization algorithms due to its potentially expensive computational cost and underlying large search space. In this study we aim to infer both the structure and parameters of a dynamic system model simultaneously by Particle Swarm Optimization (PSO), enhanced by effective stratified sampling strategies. More specifically we apply Latin Hyper Cube Sampling (LHS) with PSO. This leads to two novel swarm-inspired algorithms, LHS-PSO which can be used efficiently to learn the structure and parameters of simple and complex dynamic system models. We used a complex biological cancer model called Kinetochores, for assessing the performance of PSO and LHS-PSO. The experimental results demonstrate that LHS-PSO can find promising solutions with corresponding structure and parameters, and it outperforms PSO during our experiments.

An important benefit of multi-objective search is that it maintains a diverse population of candidates, which helps in deceptive problems in particular. Not all diversity is useful, however: candidates that optimize only one objective while ignoring others are rarely helpful. A recent solution is to replace the original objectives by their linear combinations, thus focusing the search on the most useful tradeoffs between objectives. To compensate for the loss of diversity, this transformation is accompanied by a selection mechanism that favors novelty. This paper improves this approach further by introducing novelty pulsation, i.e. a systematic method to alternate between novelty selection and local optimization. In the highly deceptive problem of discovering minimal sorting networks, it finds state-of-the-art solutions significantly faster than before. In fact, our method so far has established a new world record for the 20-lines sorting network with 91comparators. In the real-world problem of stock trading, it discovers solutions that generalize significantly better on unseen data. Composite Novelty Pulsation is therefore a promising approach to solving deceptive real-world problems through multi-objective optimization.

With improvements in selection methods and genetic operators, Genetic Programming (GP) has been able to solve many software synthesis problems. However, so far, the primary focus of GP has been on improving success rates (fraction of the runs that succeeds in finding a solution). Less attention has been paid to other important characteristics and quality measures of human-written programs. One such quality measure is modularity. Since the introduction of Automatically Defined Functions (ADFs) by John Koza, most of efforts involving modularity in GP have been directed towards pre-programming modularity into the GP system, rather than measuring it for evolved programs. Modularity has played a central role in evolutionary biology. To study its effects on the evolution of software, however, we need a quantitative formulation of modularity. In this paper, we present two platform-independent modularity metrics, namely, reuse and repetition, that make use of the information contained in the execution traces of the programs. We describe the process of calculating these metrics for any evolved program, using problems that have been solved with the PushGP system as examples. We also discuss some mechanisms for integrating these metrics into the evolution framework itself.

1With the implementation of dynamic tariffs, the electricity retailer may define distinct energy prices along the day. These tariff schemes encourage consumers to adopt different patterns of consumption with potential savings and enable the retailer to manage the interplay between wholesale and retail prices. In this work, the interaction between retailer and consumers is hierarchically modelled as a bi-level (BL) programming problem. However, if the lower level (LL) problem, which deals with the optimal operation of the consumer's appliances, is difficult to solve, it may not be possible to obtain its optimal solution, and therefore the solution to the BL problem is not feasible. Considering a computation budget to solve LL problems, a hybrid particle swarm optimization (PSO) - mixed-integer programming (MLP) approach is proposed to estimate good quality bounds for the upper level (UL) objective function. This work is based on [4].

Random drift particle swarm optimization (RDPSO) is a swarm intelligence algorithm inspired by the trajectory analysis of the canonical particle swarm optimization (PSO) and the free electron model in metal conductors placed in an external electric field. However, the RDPSO algorithm may easily encounter premature convergence when solving multimodal optimization problems. In order to deal with this issue, a new collaborative diversity control strategy for RDPSO is presented in this paper. Within this strategy, two kinds of diversity measures are used and changed in a collaborative manner to make the evolving process of the RDPSO controllable, so that premature convergence can be avoided and a final good solution can be found. Experimental results, when comparing with the canonical RDPSO and the canonical RDPSO using ring neighborhood topology, show that the proposed collaborative diversity control strategy can significantly improve the performance of the RDPSO algorithm for multimodal optimization problems in most cases.

This paper presents a new multi-gene genetic programming (MGGP) approach for prediction of bond strength of Fiber Reinforced Polymers (FRP) bars to concrete. The MGGP technique models the bond behavior of FRP bars by integrating the capabilities of standard genetic programming and classical regression. The factors that affect the bond strength of FRP bars were identified from the test data of 223 beam-test specimens in the literature and fed into the GP system to produce a model. Our study concludes that the proposed MGGP model predicts bond strength of FRP bars in concrete quite accurately. Indeed, the proposed MGGP model equation has better performance than the code equation currently used by the American Concrete Institute (ACI).

A new framework1 mixing evolutionary approach, discrete-event simulation and deep neural networks is proposed to achieve multi-asset collection/image acquisition scheduling in a surveillance context. It combines an extended graph-based hybrid genetic algorithm (GA) used for satellite image acquisition scheduling, with a predictive simulation-based deep neural network and knowledge-based capabilities to solve an heterogeneous collection asset scheduling problem. Plan execution simulation and neural networks predict track trajectories target behaviors. In contrast, a knowledge-based approach is used to estimate target identification. Both assessments are exploited to instantiate key solution quality parameters of a generalized decision model aimed at maximizing task collection value subject to a variety of collector capacity constraints. The mixed framework departs from basic point target/area coverage task modeling, introducing tracking and identification tasks while expanding resource allocation to various space, air and ground-based deployable image acquisition/collection asset types.

Support Vector Machines (SVMs) deliver state-of-the-art performance in real-world applications and are established as one of the standard tools for machine learning and data mining. A key problem of these methods is how to choose an optimal kernel function. The real-world applications have also emphasized the need to adapt the kernel to the characteristics of heterogeneous data in order to boost the classification accuracy. Therefore, our goal is to automatically search a task specific kernel function. We use reinforcement learning based search mechanisms to discover custom kernel functions and verify the effectiveness of our approach by conducting an empirical evaluation with the discovered kernel function on MNIST classification. Our experiments show that the discovered kernel function shows significantly better classification performance than well-known classic kernels. Our solution will be very effective for resource constrained systems with low memory footprint which rely on traditional machine learning algorithms like SVMs for classification tasks.

The real-world implementation of Underwater Glider Path Planning (UGPP) over the dynamic and changing environment in deep ocean waters requires complex mission planning under very high uncertainties. Such a mission is also influenced to a large extent by remote sensing for forecasting weather models outcomes used to predict spatial currents in deep sea, further limiting the available time for accurate run-time decisions by the pilot, who needs to re-test several possible mission scenarios in a short time, usually a few minutes. Hence, this paper presents the recently proposed UGPP mission scenarios' optimization with a recently well performing algorithm for continuous numerical optimization, Success-History Based Adaptive Differential Evolution Algorithm (SHADE) including Linear population size reduction (L-SHADE). An algorithm for path optimization considering the ocean currents' model predictions, vessel dynamics, and limited communication, yields potential way-points for the vessel based on the most probable scenario; this is especially useful for short-term opportunistic missions where no reactive control is possible. The newly obtained results with L-SHADE outperformed existing literature results for the UGPP benchmark scenarios. Thereby, this new application of Evolutionary Algorithms to UGPP contributes significantly to the capacity of the decision-makers when they use the improved UGPP expert system yielding better trajectories.

This work presents interesting multi-point search algorithms exploiting several surrogate models, implemented in Minamo, the multi-disciplinary optimization platform of Cenaero. Many types of surrogate models are used in the literature with their own forces and weaknesses. More generally, each one models differently a given problem and provides its own representation of the reality. The idea of this paper is to exploit simultaneously different types of surrogate models in order to catch automatically their forces and to outshine some of their weaknesses. This strategy is based on a multi-point enrichment at each iteration, each candidate point being provided by one kind of surrogate model and/or criterion. This strategy can be tuned by selecting different infill criteria, based on different surrogate models, in order to improve more specifically different aspects such as feasibility, exploration and/or exploitation. The performance of this surrogate-based optimization framework is illustrated on well-known constrained benchmark problems available in the literature (such as GX-functions and MOPTA08 test cases). Good performance both in terms of identification of feasible regions and objective gains is demonstrated.

For evolutionary algorithms (EAs), selection is one of the main components which decides solutions for the new population. Most selection strategies are fitness-based and prodigal in fitness evaluations, since many evaluated solutions are discarded immediately due to their worse values. It is desirable to predict the quality of new solutions without the evaluations before selection, thus the efficiency of EAs can be improved. Naturally selection can be considered as a classification problem: selected solutions belong to the 'good' class and the discarded ones belong to the 'bad' class. This paper demonstrates this idea by introducing a classification-based selection (CBS) strategy for EAs. The CBS is integrated into a state-of-the-art algorithm and studied on a test suite. The experimental results evidence the efficiency of CBS on saving the number of fitness evaluations when compared with the original algorithm.

Legends of the Three Kingdoms is a Chinese card game based on the novel Romance of the Three Kingdoms. As a turn-based strategy game, Legends of the Three Kingdoms is an imperfect information game where agents interact with one another to deduce the unknown information. The most famous solution for an imperfect information game is to find a Nash equilibrium where no agent can benefit from changing strategies. Tencent AI Lab has recently developed a novel learning-based Hierarchical Macro Strategy model for AI to master MOBA games, a sub-genre of RTS games. In this paper, we adopt this method on our AI to solve Legends of the Three Kingdoms in a shrunken size of 4 players.

The Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation problem. Genetic Programming (GP) has shown effectiveness in automatically evolving routing policies to handle the uncertain environment in UCARP. However, when the scenario changes, the current routing policy can no longer work effectively, and one has to retrain a new policy for the new scenario which is time consuming. On the other hand, knowledge from solving the previous similar scenarios may be helpful in improving the efficiency of the retraining process. In this paper, we propose different knowledge transfer methods from a source scenario to a similar target scenario and examine them in different settings. The experimental results showed that by knowledge transfer, the retraining process is made more efficient and the same performance can be obtained within a much shorter time without having any negative transfer.

We extend the context-free grammar mapping method in the Grammatical Evolution search heuristic. Grammatical Evolution guarantees the generation of transparent and syntactically correct sentences(phenotypes), but not necessarily semantically correct or feasible ones. Generating syntactically valid phenotypes with postprocessing to filter out semantically invalid ones suffers from some issues, e.g. introduction of bias toward short phenotypes and loss in search efficiency. These issues become significant in legal application domains. We first demonstrate that applying Grammatical Evolution with a context free grammar to legal non-compliance detection problems might not be a tenable solution. Then we demonstrate how the addition of context sensitivity improves both the search efficiency and achieves a greater diversity in the case of the iBoB problem regarding legal non-compliance.

Real-world problems often involve optimization of multiple conflicting objectives. Significant research has been directed recently towards development of multi-objective evolutionary algorithms that are scalable, i.e., able to deal with problems involving more than 3 objectives, commonly referred to as many-objective optimization problems. This has led to the emergence several new techniques that can deliver a set of trade-off solutions to approximate the Pareto optimal front of the problem. However, means to select solution(s) from this large trade-off set for final implementation/decision making has received relatively scarce attention. This paper aims to study and demonstrate the performance of recursive expected marginal utility (EMUr) approach for informed decisionmaking. Towards this goal, we apply the EMUr approach to identify solutions of interest for two practical examples and analyze the obtained set of solutions. The study highlights the desirable trade-off characteristics that the chosen solutions have over the rest of the trade-off set, highlighting its potential as a decision-making tool, especially in cases where other preference information or domain knowledge is unavailable.

Dynamic graphs are an essential tool for representing a wide variety of concepts that change over time. Examples include modeling the evolution of relationships and communities in a social network or tracking the activity of users within an enterprise computer network. In the case of static graph representations, random graph models are often useful for analyzing and predicting the characteristics of a given network. Even though random dynamic graph models are a trending research topic, the field is still relatively unexplored. The selection of available models is limited and manually developing a model for a new application can be difficult and time-consuming. This work leverages hyper-heuristic techniques to automate the design of novel random dynamic graph models. A genetic programming approach is used to evolve custom heuristics that emulate the behavior of a variety of target models with high accuracy. Results are presented that illustrate the potential for the automated design of custom random dynamic graph models.

Web applications have become increasingly essential in many domains that operate on confidential data related to business. SQL injection attack is one of the most significant web application security risks. Detecting SQL injection vulnerabilities is essential for protecting the underlying web application. However, manually enumerating test cases is extremely challenging, if not impossible, given the potentially infinite number of user inputs and the likely nonexistence of one-to-one mapping between user inputs and malicious SQL statements. This paper proposes an automatic security test case generation approach to detect SQL injection vulnerabilities for web applications, following a search-based software engineering (SBSE) paradigm. Particularly, we propose a novel fitness function that evaluates the similarity between the SQL statements produced by feeding user inputs in the system under test and a known malicious SQL statement. For the search algorithm, we exploit differential evolution, which is robust in continuous optimization but it is under-investigated in SBSE. Based on three real-world web applications, we conduct experiments on 19 configurations that are of diverse forms of SQL statements and types of attacks. Results demonstrate that our approach is more effective, with statistical significance and high effect sizes, than the state-of-the-art.

The use of well-established statistical testing procedures to compare the performance of evolutionary algorithms often yields pessimistic results. This requires increasing the number of independent samples, and thus the computation time, in order to get results with the necessary precision. We aim at improving this situation by developing statistical tests that are good in answering typical questions coming from benchmarking of evolutionary algorithms. Our first step, presented in this paper, is a procedure that determines whether the performance distributions of two given algorithms are identical for each of the benchmarks. Our experimental study shows that this procedure is able to spot very small differences in the performance of algorithms while requiring computational budgets which are by an order of magnitude smaller (e.g. 15x) compared to the existing approaches.

Genetic improvement uses automated search to find improved versions of existing software. Software can either be evolved with general-purpose intentions or with a focus on a specific application (e.g., to improve it's efficiency for a particular class of problems). Unfortunately, software specialisation to each problem application is generally performed independently, fragmenting and slowing down an already very time-consuming search process. We propose to incorporate specialisation as an online mechanism of the general search process, in an attempt to automatically devise application classes, by benefiting from past execution history.

ECJ is now 20 years old. Begun as a genetic programming and evolutionary computation library in Java, it has since established itself as historically one of the most popular EC toolkits worldwide. In 2016 we received a National Science Foundation grant to improve ECJ in many ways with an eye toward making it a useful toolkit not just for EC but for the broader metaheuristics community. This paper is a report on our efforts to this end. We discuss new metaheuristics frameworks and representations added to ECJ and the design challenges that they raise for a general-purpose framework, as well as testing facilities and other support tools. We conclude with our future directions for the library.

An appropriate bus timetable is vital for bus enterprises to improve service quality and save operational cost. Most existing literature on bus timetable optimization divide a whole day into several periods and assume that departure time intervals are the same for each period. As passengers' flow varies over time in a period, giving the same time interval for each period cannot really meet the demand of passengers. In this paper, we study the optimization of bus timetable with unequal time intervals. Aiming at characteristics of this problem, a memetic algorithm is devised that combines a genetic algorithm with elite strategy and a local search. To handle infeasible solutions, a repair method is proposed to repair solutions that do not meet the constraint. A new metric reflecting the degree of bus carrying capability matching passengers' need is introduced. The metric together with passengers' waiting time is used to evaluate a bus timetable. Experiments show that compared to the actually used timetables, timetables optimized by the proposed approach are able to save about 4.54-12.84% cost and 11.87-37.76% passengers' waiting time.

Malware is a malicious code which intends to harm computers and networks. Each year, a huge number of malicious programs are released. Therefore, detecting malware has become one of the most important challenges for the security of computer systems. Various methods have been defined for detecting and classifying malware, such as signature-based and heuristic-based techniques. This paper proposes a new malware detection method based on the operational codes (OpCodes) within an executable file by using the evolutionary algorithm. There are several steps in the proposed method, which includes disassembling the executable files, generating a graph of OpCodes and using the evolutionary algorithm to find the most similar graph to each suspicious instance. Finally, the label of each suspicious instance is detected based on the most similar graph obtained from the evolutionary algorithm with each class (family of malware and benign). The results show that, the proposed method can be used as a method for malware detection and malware category.

This extended abstract previews the usage of Gaussian processes in a surrogate-model version of the CMA-ES, a state-of-the-art black-box continuous optimization algorithm. The proposed algorithm DTS-CMA-ES exploits the benefits of Gaussian process uncertainty prediction, especially during the selection of points for the evaluation with the surrogate model. Very brief results are presented here, while much more elaborate description of the methods, parameter settings and detailed experimental results can be found in the original article Gaussian Process Surrogate Models for the CMA Evolution Strategy [2], to appear in the Evolutionary Computation1.

Aiming at the problem of oversubscription of data relay access request of user stars in future Space-Based Information System, the problem of resource scheduling optimization for data relay satellite system with microwave and laser hybrid links is studied. The characteristics of the hybrid links are analyzed. A multi-objective programming model on static resource scheduling constraint satisfaction problem is established, and a hybrid optimization algorithm integration of artificial immune strategies, niche ideas and improved genetic algorithm is put forward to solve the scheduling model. Simulation results show that the hybrid optimization algorithm optimizes the model quickly, and performs well in the ability of global optimization and convergence. The results validate that the static resource scheduling model could accurately describe the microwave and laser hybrid links relay satellite system resource scheduling problem with multi-tasking and multi-type antenna1.

This work discusses the solution of a Large-scale global optimization problem named Security Constrained Optimal Power Flow (SCOPF) using a method based on Cross Entropy (CE) and Evolutionary Particle Swarm Optimization (EPSO). The obtained solution is compared to the Entropy Enhanced Covariance Matrix Adaptation Evolution Strategy (EE-CMAES) and Shrinking Net Algorithm (SNA). Experiments show the approach reaches competitive results.

Swarm Intelligence (SI) is a behavior, used first by Beni and Wang, corresponding to a system working with single and self-organized agents, interacting the ones with each other. This operating concept is implemented in many algorithms. Developed by Kennedy, Eberhart and Shi, Particle Swarm Optimization (PSO) is one of them. Its behavior is based on the movements of birds swarm, and its effectiveness, for looking for the optimal solution of a given problem, is well established. Nevertheless, PSO is known for its weakness in local search. Moreover, the behavior of PSO strongly depends on internal parameters settings. In order to address these problems, we propose a new type of self-adaptive Quantum PSO (QPSO), called QUAntum Particle Swarm Optimization (QUAPSO), based on quantum superposition to set the velocity parameters and hybridized with a Kangaroo Algorithm in order to optimize its efficiency in local search. QUAPSO was compared with five known algorithms from the literature (classical PSO, Kangaroo Algorithm, Simulated Annealing Particle Swarm Optimization, Bat Algorithm and Simulated Annealing Gaussian Bat Algorithm) on a set of 19 test functions. The results show that QUAPSO outperforms the competing algorithms.

Model-based evolutionary algorithms (MBEAs) are praised for their broad applicability to black-box optimization problems. In practical applications however, they are mostly used to repeatedly optimize different instances of a single problem class, a setting in which specialized algorithms generally perform better. In this paper, we introduce the concept of a new type of MBEA that can automatically specialize its behavior to a given problem class using tabula rasa self-learning. For this, reinforcement learning is a naturally fitting paradigm. A proof-of-principle framework, called SL-ENDA, based on estimation of normal distribution algorithms in combination with reinforcement learning is defined. SL-ENDA uses an RL-agent to decide upon the next population mean while approaching the rest of the algorithm as the environment. A comparison of SL-ENDA to AMaLGaM and CMA-ES on unimodal noiseless functions shows mostly comparable performance and scalability to the broadly used and carefully manually crafted algorithms. This result, in combination with the inherent potential of self-learning model-based evolutionary algorithms with regard to specialization, opens the door to a new research direction with great potential impact on the field of model-based evolutionary algorithms.

Stochastic synthesis of recursive functions has historically proved difficult, not least due to issues of non-termination and the often ad hoc methods for addressing this. We propose a general method of implicit recursion which operates via an automatically-derivable decomposition of datatype structure by cases, thereby ensuring well-foundedness. The method is applied to recursive functions of long-standing interest and the results outperform recent work which combines two leading approaches and employs 'human in the loop' to define the recursion structure. We show that stochastic synthesis with the proposed method on benchmark functions is effective even with random search, motivating a need for more difficult recursive benchmarks in future. This paper summarizes work that appeared in [1].

Modern gradient based optimization methods for deep neural networks demonstrate outstanding results on image classification tasks. However, methods that do not rely on gradient feedback fail to tackle deep network optimization. In the held of evolutionary computation, applying evolutionary algorithms directly to network weights remains to be an unresolved challenge. In this paper we examine a new framework for the evolution of deep nets. Based on the empirical analysis, we propose the use of linear sub-spaces of problems to search for promising optimization trajectories in parameter space, opposed to weight evolution. We show that linear sub-spaces of loss functions are sufficiently well-behaved to allow trajectory evaluation. Furthermore, we introduce fitness measure to show that it is possible to correctly categorize trajectories according to their distance from the optimal path. As such, this work introduces an alternative approach to evolutionary optimization of deep networks.

In this paper we propose a tabu search-based memetic algorithm (TSM) for the multi-objective flexible job shop scheduling problem (FJSSP), with the objectives to minimize the makespan, the total workload and the critical workload. The problem is addressed in a Pareto manner, which targets a set of Pareto optimal solutions. The novelty of our method lies in the use of tabu search (TS) as the local search method as well as a mutation operator and the use of the hypervolume indicator to avoid stagnation by increasing the flow of individuals in the local search. To the best of our knowledge, the use of TS in the context of multi-objective FJSSP has not been reported so far. We apply our algorithm on well known test instances and compare our results to state-of-the art algorithms. The results show that our approach yields competitive solutions in 6 of the 10 instances against two of their algorithms proving that the use of TS as a local search method can provide competitive results.

Multi-objective problems (MOP) are of significant interest to both multi-criteria decision making (MCDM) and evolutionary multi-objective (EMO) research communities. A core technique common in both is scalarization, which combines multiple objectives into one in a way that solving it provides a solution to the original MOP. In this paper, we look closely at two scalarization methods --- augmented achievement scalarization function (AASF) and penalty boundary intersection (PBI). While the former has its roots in MCDM literature, the latter was developed in EMO field with focus on decomposition-based algorithms. We observe the conventional limits of the parameters involved in these methods and then demonstrate that by relaxing those limits one could be made to behave like the other. The aim is to gain a deeper understanding of both these measures, as well as expand their parametric range to provide more control over the search behavior of EMO algorithms. It also lays groundwork for further development of complete analytical derivations of equivalence conditions between the two metrics.

This paper proposes a hybrid evolutionary approach to feature engineering for mining frequent patterns from multidimensional financial ultra-high frequency time series. Experiments performed on real-world data from the London Stock Exchange Rebuilt Order Book database confirms that the evolutionary algorithm is capable of improving significantly the results of classification.

Automated search in the form of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), plus manual code changes, transforms 512 Newton-Raphson floating point start numbers from an open source GNU C library, glibc, table driven square root function to create a new bespoke custom mathematical implementation of double precision binary logarithm log2 for C in seconds.

This paper surveys and compares a wide range of derivative-free optimization algorithms in an open source context. We also propose a genetic variant of differential evolution, an adaptation of population control for the multimodal noise-free case, new multiscale deceptive functions, and as a contribution to the debate on genetic crossovers, a test function with useless variables on which 2-points crossover achieves great performance. We include discrete and continuous and mixed settings; sequential and parallel optimization; rotated, separable and partially rotated settings; noisy and noise-free setting. We include real world applications and compare with recent optimizers which have not yet been extensively compared to the state of the art.

Various EA-based methods have been applied to design and optimize logic circuits since the early nineties. The unconventional methods, however, typically suffer from various scalability issues preventing them to be adopted in practice. Recent improvement in the fitness computation procedure connected with the introduction of formal methods in the fitness evaluation such as SAT solvers or BDDs enabled pushing of the limits forward and approaching the complexity of industrial problems. It was demonstrated that EAs can be applied to optimize gate-level circuits consisting of thousands of gates without introducing any decomposition technique. Despite that, the efficiency decreases with increasing the circuit complexity. This problem can be managed by adopting the concept of the so-called iterative resynthesis based on the extraction of smaller sub-circuits from a complex circuit, their local optimization followed by the implantation back to the original circuit. Recently, a method based on the computation of so-called cuts was proposed. In this paper, we propose an alternative approach which is able to select more complex sub-graphs consisting of more nodes and more inputs. Compared to the previous method, the proposed approach allows to improve the efficiency of the optimization. More than 9% and 20% reduction was observed on the highly optimized logic and arithmetic circuits, respectively.

This paper proposes a region-learning based JADE algorithm, namely RL-JADE, to solve numerical optimization problems. To exploit as much as possible the most promising areas known by the current population, the worst parts of population are eliminated, and some new individuals are regenerated in the area where the best parts of population locate. RL-JADE is tested based on COCO benchmarks, and compared with other DE-variants. Experimental results show that RL-JADE has a better performance than JADE on the tested 5-D problems.

We propose a hybridization approach called Regularized-Surrogate- Optimization (RSO) aimed at overcoming difficulties related to high- dimensionality. It combines standard Kriging-based SMBO with regularization techniques. The employed regularization methods use the least absolute shrinkage and selection operator (LASSO). An extensive study is performed on a set of artificial test functions and two real-world applications: the electrostatic precipitator problem and a multilayered composite design problem. Experiments reveal that RSO requires significantly less time than Kriging to obtain comparable results. The pros and cons of the RSO approach are discussed and recommendations for practitioners are presented.

In machine learning, transfer learning is concerned with utilising prior knowledge as a way to improve the process of training a new model in a different, but related, domain. Transfer learning has been shown to be beneficial across a large set of problems. One of the main questions any transfer learning approach must address is "What to transfer?". This paper proposes a new transfer learning method in genetic programming (GP) to improve solving symbolic regression problems by extracting all potentially good and unique building blocks from a source problem. The proposed method is compared against standard GP and a state-of-the-art GP method on ten regression datasets. The experimental results show that the proposed method has achieved significantly better or comparable performance to that of the competitive methods. Furthermore, the proposed method shows better initial population and convergence compared to the other methods.

This work presents a multi-population biased random-key genetic algorithm (BRKGA) for the electric distribution network reconfiguration problem (DNR). DNR belongs to the class of network design problems which include transportation problems, computer network restoration and telecommunication network design and can be used for loss minimization and load balancing, being an important tool for distribution network operators. A BRKGA is a class of genetic algorithms in which solutions are encoded as vectors of random keys, i.e. randomly generated real numbers from a uniform distribution in the interval [0, 1). A vector of random keys is translated into a solution of the optimization problem by a decoder. The decoder used generates only feasible solutions by using an efficient codification based upon the fundamentals of graph theory, restricting the search space. The parallelization is based on the single program multiple data paradigm and is executed on the cores of a multi-core processor. Time to target plots, which characterize the running times of stochastic algorithms for combinatorial optimization, are used to compare the performance of the serial and parallel algorithms. The proposed method has been tested on two standard distribution systems and the results show the effectiveness and performance of the parallel algorithm.

In the area of multi-objective optimization, a special challenge is dynamic optimization problems. These problems change their optimal values or optimal configurations of input variables over time, making it harder for metaheuristic algorithms to track these changes and find the new optima. To test the search ability of such dynamic multi-objective algorithms, a dynamic benchmark called the Dynamic Distance Minimization Problem (dDMP) was proposed in the literature. The dDMP implements multiple changes, not only in location and fitness values of the Pareto-optimal sets, but also in the complexity of the problem. This work aims to test the performance of two well-known dynamic multi-objective algorithms on different instances of the dDMP with varying complexity. This involves changes in the number of objectives and changes of the distance metric at runtime, which has not been done before with this problem in the literature. The results show that both algorithms struggled to recover after the number of objectives was reduced and then increased again. When the distance metric was changed over time both algorithms performed reasonable well. However, there were gaps in the found Pareto fronts when switching between the Euclidean and the Manhattan distance metrics.

Electrical wires are a very important part of a vehicle as well of many other objects with electrical functions. These wires are summed up to harnesses. Engineers often manually create the virtual paths for wires by considering their expert knowledge. More variants of a product often require different wires and the criteria for cables do vary. Criteria, e.g. heat or the length, define the wire path and configuration and have to be taken into account. Engineers may not be able to consider all criteria, notably when there is an extensive amount of product variants, or when the object configuration is changing. This paper will evaluate if a multi-objective evolutionary algorithm can compute feasible wire paths where two of these criteria are optimised. We are showing that discretised free-space in a graph structure enables an evolutionary algorithm to compute wire suggestions by using customised reproduction operators for path planning. The developed crossover and mutation operators are suitable for multi-objective path finding, may be used in other fields as well and will be extended in future research.