Search

Distributed optimization algorithms are widely used for solving science and engineering problems in recent years. However, these algorithms become vulnerable in the face of external attacks, so it is urgent to develop novel resilient distributed optimization algorithms against attacks. In this paper, we design a novel median-based resilient distributed optimization algorithm to defend the Byzantine attack. The existence of transition matrix is proved, and the convergence and optimality of the proposed algorithm are analyzed. Simulation experiments are performed to verify the effectiveness of the proposed algorithm and the results show that it is more effective than the compared one and can get the best possible results against the attack. [ABSTRACT FROM AUTHOR]

This paper presents an optimal feeder-switch relocation that is applied by modified Particle Swarm Optimization (MPSO). An inertia weight in Particle Swarm Optimization (PSO) is modified to find the best patterns of distribution configuration. MPSO performance is evaluated by comparison with the conventional inertia weight method. Six different benchmark functions with asymmetric initial range settings are selected as testing functions. A nonlinear inertia weight function is applied in this paper. The results of the experiment illustrate the advantage of MPSO. The optimal feeder-switch relocation in a radial distribution system is used to evaluate the MPSO performance. The results show that MPSO can identify suitable switch locations, based on minimum customer interruption costs. [ABSTRACT FROM AUTHOR]

In the last decade a new variety of nature inspired optimization algorithms has been appeared. After the swarm based models, researchers turned their inspiration in nature phenomena and laws of science. In this way a new category of algorithms was born, equally effective or even sometimes superior to known algorithms for optimization problems, like genetic algorithms and swarm intelligence schemes. The present survey depicts the evolution of research on nature inspired optimization algorithms related to physical phenomena and laws of science and discusses the possibilities of using the presented approaches in a number of different applications. An attempt has been made to draw conclusions on what algorithm could be used in which different problem areas, for those approaches that this information could be extracted from the related papers studied. The paper also underlines the usage of this kind of nature inspired algorithms in industrial research problems, due to their better confrontation with optimization problems represented with nodes and edges. [ABSTRACT FROM AUTHOR]

In this paper, an improved imperialistic competitive algorithm is presented for real-valued optimization problems. A new method is introduced for the movement of colonies towards their imperialist, which is called assimilation. The proposed method uses Euclidean distance along with Pearson correlation coefficient as an operator for assimilating colonies with respect to their imperialists. Applications of the proposed algorithm to classical and recently published hard benchmark problems, and statistical analysis associated with the corresponding experimental results illustrated that the achieved success is significantly better than a number of state-of-the art methods. [ABSTRACT FROM AUTHOR]

The effectiveness of cross entropy (CE) method has been investigated on both combinatorial and continuous optimization problems, though it lacks exploitative search to refine solutions. Hybrid with local search (LS) method can greatly improve the performance of evolutionary algorithm. This paper proposes a parameter-less framework combining CE with LS method. Four LS methods are chosen and four combination algorithms are obtained after combining them with the CE method. We first study the performance of the four combinations on a set of twenty eight mathematical functions including both unimodal and multimodal functions. CE hybrid with Powell's method (CE-Pow) is identified as the most effective algorithm. Then the CE-Pow algorithm is applied to resolve proportional, integral, and derivative (PID) controller design problem and Lennard-Jones potential problem. Its performance has been verified by comparing with four state of the art evolutionary algorithms. Experimental results show that CE-Pow significantly outperforms other benchmark algorithms. [ABSTRACT FROM AUTHOR]

The problem of finding the best quantum time in multi-level processor scheduling is addressed in this paper. Processor scheduling is one of the most important issues in operating systems design. Different schedulers are introduced to solve this problem. In one scheduling approach, processes are placed in different queues according to their properties, and the processor allocates time to each queue iteratively. One of the most important parameters of a processor's efficiency in this approach is the amount of time slices associated to each processor queue. In this paper, an ant colony optimization (ACO) algorithm is presented to solve the problem of finding appropriate time slices to assign to each processor queue. In this technique, each ant tries to find an appropriate scheduling. Ant algorithm searches the problem space to find the best scheduling. The quality of each ant's solution is evaluated using a new fitness function. This fitness function is designed according to the evaluation parameters of each processor queue and also according to the queue theory's relations. Also a heuristic function is presented which prompts ant to select better solutions. Computational tests are presented and the comparisons made with genetic algorithm (GA) and particle swarm optimization (PSO) algorithms which try to solve same problem. The results show the efficiency of this algorithm. [ABSTRACT FROM AUTHOR]

Artificial bee colony (ABC) algorithm invented by Karaboga has been proved to be an efficient technique compared with other biological-inspired algorithms for solving numerical optimization problems. Unfortunately, convergence speed of ABC is slow when working with certain optimization problems and some complex multimodal problems. Aiming at the shortcomings, a hybrid artificial bee colony algorithm is proposed in this paper. In the hybrid ABC, an improved search operator learned from Differential Evolution (DE) is applied to enhance search process, and a not-so-good solutions selection strategy inspired by free search algorithm (FS) is introduced to avoid local optimum. Especially, a reverse selection strategy is also employed to do improvement in onlooker bee phase. In addition, chaotic systems based on the tent map are executed in population initialization and scout bee's phase. The proposed algorithm is conducted on a set of 40 optimization test functions with different mathematical characteristics. The numerical results of the data analysis, statistical analysis, robustness analysis and the comparisons with other state-of-the-art-algorithms demonstrate that the proposed hybrid ABC algorithm provides excellent convergence and global search ability. [ABSTRACT FROM AUTHOR]

Combinatorial Optimization lies at the heart of many Artificial Intelligence tasks. While existing techniques are still improved year after year, we advocate that recent sub-symbolic approaches may change the game if they can be successfully integrated into combinatorial solvers. In this paper, we present a state of the art of current approaches and discuss the role of new learning techniques to improve the current results. [ABSTRACT FROM AUTHOR]

In June 2008, the Eclipse open platform released a new dependency management system called p2. That system was based on the translation of the dependency management problem into a pseudo-Boolean optimization problem, to be handled by the Sat4j solver. Since then, p2 has been more tightly integrated with Sat4j, the platform opened a public plugin repository (the Eclipse marketplace) which relies on p2 to install the available plugins and their dependencies, and became the favorite way to install plugins in the Eclipse community. This paper summarizes the issues raised by Eclipse dependency management, its pseudo-Boolean encoding within p2, its extension for Linux package management with p2cudf, and concludes with lessons learned on using research software in production systems. [ABSTRACT FROM AUTHOR]

The successful handling of numerous real–world complex problems has increased the popularity of nature–inspired intelligent (NII) algorithms and techniques. Their successful implementation primarily on difficult and complicated optimization problems, stresses their upcoming importance in the broader area of artificial intelligence. NII techniques take advantage of the way that biological systems deal with real–world situations. Specifically, they simulate the way real biological systems, such as the human brain, ant colonies and human immune system work, when solving complex real–world situations. In this survey paper, we briefly present a number of selected NII approaches and we point particular suitable areas of application for each of them. Specifically, five major categories of nature inspired approaches are presented, namely, Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), DNA computing, artificial immune systems and membrane computing. Applications include problems related to optimization (financial, industrial and medical), task scheduling, system design (optimization of the system's parameters), image processing and data processing (feature selection and classification). We also refer to collaboration between NII techniques and classical AI methodologies, such as neural networks, genetic algorithms, fuzzy logic, etc. The current survey states that NII techniques are likely to become the next step in the rapid evolution of artificial intelligence tools. [ABSTRACT FROM AUTHOR]

When a practical problem can be modeled as a constraint satisfaction problem (CSP), which is a set of constraints that need to be satisfied, it can be solved using many constraint programming techniques. In many practical applications, while users can recognize examples of where a CSP should be satisfied or violated, they cannot articulate the specification of the CSP itself. In these situations, it can be helpful if the computer can take an active role in learning the CSP from examples of its solutions and non-solutions. This is called constraint acquisition. This paper introduces a framework for constraint acquisition in which one can uniformly define and formulate constraint acquisition problems of different types as optimization problems. The difference between constraint acquisition problems within the framework is not only in the type of constraints that need to be acquired but also in the learning objective. The generic framework can be instantiated to obtain specific formulations for acquiring classical, fuzzy, weighted or probabilistic constraints. The paper shows as an example how recent techniques for acquiring classical constraints can be directly obtained from the framework. Specifically, the formulation obtained from the framework to acquire classical CSPs with the minimum number of violated examples is equivalent to a simple pseudo-boolean optimization problem, thus being efficiently solvable by using many available optimization tools. The paper also reports empirical results on constraint acquisition methods to show the utility of the framework. [ABSTRACT FROM AUTHOR]

Case retrieval, case reuse and case retention are critical to the reasoning performance of the traditional case-based reasoning (CBR) model. In this paper, the integrated use of template reduction technology (TR), genetic algorithms (GA), nearest neighbor (NN) rules and group decision-making (GDM) establishes the CBR-GDM model. First, the TR method of the case base is introduced. Then, an attribute weights optimization using GA is discussed in the case retrieval phase. After that, a case reuse method is carried out with NN and GDM. Finally, 10 data sets from UCI are used to carry out a comparison experiment by 5-fold cross-validation. The classification accuracy rate and the classification efficiency are analyzed under the small samples, before and after the data reduction. The results show that, combined with TR, GA and GDM, the pattern classification performance by CBR can be improved. [ABSTRACT FROM AUTHOR]

In a typical supervised data analysis task, one needs to perform the following two tasks: (a) select an optimal combination of learning methods (e.g., for variable selection and classifier) and tune their hyper-parameters (e.g., K in K-NN), also called model selection, and (b) provide an estimate of the performance of the final, reported model. Combining the two tasks is not trivial because when one selects the set of hyper-parameters that seem to provide the best estimated performance, this estimation is optimistic (biased/overfitted) due to performing multiple statistical comparisons. In this paper, we discuss the theoretical properties of performance estimation when model selection is present and we confirm that the simple Cross-Validation with model selection is indeed optimistic (overestimates performance) in small sample scenarios and should be avoided. We present in detail and investigate the theoretical properties of the Nested Cross Validation and a method by Tibshirani and Tibshirani for removing the estimation bias. In computational experiments with real datasets both protocols provide conservative estimation of performance and should be preferred. These statements hold true even if feature selection is performed as preprocessing. [ABSTRACT FROM AUTHOR]

Automatic facial landmark localization is a challenging problem for real world images because of face pose variations and occlusions. This paper proposes a unified framework to robustly locate facial landmarks under different poses and occlusions. Instead of explicitly modeling the statistical point distribution, we use a sparse linear combination to approximate the observed shape, and hence alleviate the multi-pose problem. In addition, we use the sparsity constraint to handle outliers caused by occlusions. We also model the initial misalignment and use convex optimization techniques to solve them simultaneously and efficiently. We evaluated the proposed method extensively on both synthetic and real data. The experimental results are promising on handling pose variations and occlusions. [ABSTRACT FROM AUTHOR]

One of the most important types of signal found in the area of machine condition monitoring/prognostic health monitoring (MCM/PHM) is the vibration signal, a type of waveform. Many time-frequency domain techniques have been proposed to interpret such signals, including wavelet packet decomposition (WPD). Previous work has shown how to extend the WPD algorithm to operate on streaming signals, but the number of output variables becomes exponential in the number of levels of decomposition, hindering data mining in limited-memory environments. Feature selection techniques, well understood in other areas of data mining, can be used to greatly reduce the number of output variables and speed up the machine learning algorithms. This paper presents a case study comparing two versions of WPD both with and without feature selection, demonstrating that removing most of the features produced by the WPD does not impair its performance within the context of MCM/PHM. [ABSTRACT FROM AUTHOR]

Choosing the optimal number of clusters is a key issue in cluster analysis. Especially when dealing with more spatial clustering, things tend to be more complicated. Cluster validation helps to determine the appropriate number of clusters present in a dataset. Furthermore, cluster validation evaluates and assesses the results of clustering algorithms. There are numerous methods and techniques for choosing the optimal number of clusters via crisp and fuzzy clustering. In this paper, we introduce a new index for fuzzy clustering to determine the optimal number of clusters. This index is not another metric for calculating compactness or separation among partitions. Instead, the index uses several existing indices to give a degree, or fuzziness, to the optimal number of clusters. In this way, not only do the objects in a fuzzy cluster get a membership value, but the number of clusters to be partitioned is given a value as well. The new index is used in the fuzzy c-means algorithm for the geodemographic segmentation of 285 postal codes. [ABSTRACT FROM AUTHOR]

Polynomial mutation is widely used in evolutionary optimization algorithms as a variation operator. In previous work on the use of evolutionary algorithms for solving multi-objective problems, two versions of polynomial mutations were introduced. The first is non-highly disruptive that is not prone to local optima and the second is highly disruptive polynomial mutation. This paper looks at the two variants and proposes a dynamic version of polynomial mutation. The experimental results show that the proposed adaptive algorithm is doing well for three evolutionary multiobjective algorithms on well known multiobjective optimization problems in terms of convergence speed, generational distance and hypervolume performance metrics. [ABSTRACT FROM AUTHOR]

Many real world industrial applications involve the Traveling Salesman Problem (TSP), which is a problem that finds a Hamiltonian path with minimum cost. Examples of problems that belong to this category are transportation routing problem, scan chain optimization and drilling problem in integrated circuit testing and production. This paper presents a Bee Colony Optimization (BCO) algorithm for symmetrical TSP. The BCO model is constructed algorithmically based on the collective intelligence shown in bee foraging behaviour. The algorithm is integrated with a fixed-radius near neighbour 2-opt (FRNN 2-opt) heuristic to further improve prior solutions generated by the BCO model. To limit the overhead incurred by the FRNN 2-opt, a frequency-based pruning strategy is proposed. The pruning strategy allows only a subset of the promising solutions to undergo local optimization. Experimental results comparing the proposed BCO algorithm with existing approaches on a set of benchmark problems are presented. For 84 benchmark problems, the BCO algorithm is able to obtain an overall average solution quality of 0.31% from known optimum. The results also show that it is comparable to other algorithms such as Ant Colony Optimization and Particle Swarm Optimization. [ABSTRACT FROM AUTHOR]

This paper presents a parallel framework for the solution of multi-objective optimization problems. The framework implements some of the best known multi-objective evolutionary algorithms. The plugin-based architecture of the framework minimizes the end user effort required to incorporate their own problems and evolutionary algorithms, and facilitates tool maintenance. A wide variety of configuration options can be specified to adapt the software behavior to many different parallel models. An innovation of the framework is that it provides a self-adaptive parallel model that is based on the cooperation of a set of evolutionary algorithms. The aim of the new model is to raise the level of generality at which most current evolutionary algorithms operate. This way, a wider range of problems can be tackled since the strengths of one algorithm can compensate for the weaknesses of another. The model proposed is a hybrid algorithm that combines a parallel island-based scheme with a hyperheuristic approach. The model grants more computational resources to those algorithms that show a more promising behavior. The flexibility and efficiency of the framework were tested and demonstrated by configuring standard and self-adaptive models for test problems and real-world applications. [ABSTRACT FROM AUTHOR]

The current paper addresses the design of complex distributed systems consisting of many components by using Hierarchical Cooperative CoEvolution (HCCE), an optimization mechanism that also follows a distributed organization. The proposed coevolutionary scheme is capable of optimizing complex distributed systems, taking also into account the specialized roles of substructures. Here, we present HCCE and we compare it with ordinary Unimodal evolution and Enforced SubPopulation coevolution. The current study aims at highlighting the internal dynamics of HCCE that give rise to its effectiveness in addressing difficult distributed design problems. The results obtained attest to the validity and effectiveness of HCCE, showing that it outperforms both other schemes. [ABSTRACT FROM AUTHOR]

In this paper we present a radical approach to obtaining a backtrack-free representation for a constraint satisfaction problem: remove values that lead to dead-ends. This technique does not require additional space but has the drawback of removing solutions. We investigate a number of variations on the basic algorithm including the use of seed solutions, consistency techniques, and a variety of pruning heuristics. Our experimental results indicate that a significant proportion of the solutions to the original problem can be retained especially when an optimization algorithm that specifically searches for such “good” backtrack-free representations is employed. Further extensions increase solution retention by searching for high-coverage backtrack-free representations, by removing tuples rather than values, and by combining multiple backtrack-free representations. Our approach elucidates, for the first time, a three-way trade-off between space complexity, potential backtracks, and solution loss and enables algorithms that can actively reason about the trade-off between space, backtracks, and solution loss. [ABSTRACT FROM AUTHOR]

During the last years nature inspired intelligent techniques have become attractive for analyzing large data sets and solving complex optimization problems. In this paper, one of the most interesting of them, the Ant Colony Optimization (ACO), is used for the construction of a hybrid algorithmic scheme which effectively handles the Pap Smear Cell classification problem. This algorithmic approach is properly combined with a number of nearest neighbor based approaches for performing the requested classification task, through the solution of the so-called optimal feature subset selection problem. The proposed complete algorithmic scheme is tested in two sets of data. The first one consists of 917 images of pap smear cells and the second set consists of 500 images, classified carefully by expert cyto-technicians and doctors. Each cell is described by 20 numerical features, and the cells fall into seven (7) classes, four (4) representing normal cells and three (3) abnormal cases. Nevertheless, from the medical diagnosis viewpoint, a minimum requirement corresponds to the general two-class problem of correct separation between normal from abnormal cells. [ABSTRACT FROM AUTHOR]

In this paper, we study the unrelated parallel machine problem for minimizing the makespan, which is NP-hard. We used Simulated Annealing (SA) and Tabu Search (TS) with Neighborhood Search (NS) based on the structure of the problem. We also used a modified SA algorithm, which gives better results than the traditional SA and developed an effective heuristic for the problem: Squeaky Wheel Optimization (SWO) hybrid with TS. Experimental results average 2.52% from the lower bound and are within acceptable timescales improving current best results for the problem. [ABSTRACT FROM AUTHOR]

A temporal flexible planning problem that involves contingent and requirement events can be formulated as a simple temporal network with uncertainty (STNU). An STNU is controllable when there is a strategy for executing the requirement events (or actions) in such a way that all the conditions involving contingent events can be satisfied in all situations. The most interesting and useful controllability property is dynamic controllability in which the remaining actions in an STNU can always be scheduled under all possible feasible durations of future contingent events when all the past contingent events are known. In this paper, we propose and study a novel problem of assigning bounds on the duration of each requirement link in order for the resulting STNU to be dynamically controllable and to minimize the total cost over the allowed durations of all requirement links. We first prove the NP hardness of the problem with a linear cost function. We then formulate the dynamic controllability of an STNU as the constraints in a nonlinear optimization problem. Finally, we present methods for reducing the number of constraints in order to make the problem tractable and to demonstrate the computational performance of our methods. [ABSTRACT FROM AUTHOR]

A complex model for evolving the update strategy of a Particle Swarm Optimisation (PSO) algorithm is described in this paper. The model is a hybrid technique that combines a Genetic Algorithm (GA) and a PSO algorithm. Each GA chromosome is an array encoding a meaning for updating the particles of the PSO algorithm. The Evolved PSO algorithm is compared to several human-designed PSO algorithms by using ten artificially constructed functions and one real-world problem. Numerical experiments show that the Evolved PSO algorithm performs similarly and sometimes even better than the Standard approaches for the considered problems. The Evolved PSO is highly scalable (regarding the size of the problem's input), being able to solve problems having different dimensions. [ABSTRACT FROM AUTHOR]

The complexity and frequent fluctuations of economic data pose a significant challenge to forecasting studies. In order to predict financial data more accurately, we build a fusion model for predicting financial data based on the idea of decomposition-recombination by combining the Sooty Tern Optimization Algorithm (STOA), Variational Mode Decomposition (VMD), Support Vector Machine (SVM) and Back Propagation Neural Network (BPNN). Firstly, after finding the optimal parameters of the VMD by the STOA, adjust the parameters of the VMD and decompose the financial data, remove the residual from the series, and constitute the remaining necessary information within the financial data into new series modeled by applying the SVM. Finally, calculate the error between the predicted and actual values. Notably, the residual is treated specially in the modeling. A Double-layer BPNN model is used to establish a mechanism to increase the sensitivity of the model fluctuations. The influence factor series and the residual are introduced as input variables to the BPNN to establish a mapping relationship between them and the error series. The results show that the model improves data utilization through five experiments, solves the problem of the VMD insensitivity to fluctuations, and improves the prediction accuracy of financial time series effectively. [ABSTRACT FROM AUTHOR]

In this paper, we consider optimal scheduling algorithms for scientific workows with two typical structures, fork&join and tree, on a set of provisioned (virtual) machines under budget and deadline constraints in cloud computing. First, given a total budget B, by leveraging a bi-step dynamic programming technique, we propose optimal algorithms in pseudo-polynomial time for both workows with minimum scheduling length as a goal. Our algorithms are efficient if the total budget B is polynomially bounded by the number of jobs in respective workows, which is usually the common case in practice. Second, we consider the dual of this optimization problem to minimize the cost when the deadline of the computation D is fixed. We change this problem into the standard multiple-choice knapsack problem via a parallel transformation. [ABSTRACT FROM AUTHOR]

Mixed-initiative systems for interactive optimization allow the user to add preferences to find satisfactory solutions. Industries use them as decision-support tools in order to take into consideration unexpected events or characteristics not stated in the optimization model. We present a design of an interactive optimization system that allows users to quickly find a new optimal solution when the user changes the values of decision variables. It anticipates the actions of the user and precomputes the solutions that satisfy the preferences by using both the capacities of the optimization software and a novel custom technique based on the kernel of the constraint matrix. This system is tested on a softwood lumber-drying planning problem and returns promising results for future use in wood industries. [ABSTRACT FROM AUTHOR]

Optimization Solution Task is a typical and important task in many applications. Many optimization problems have been proved to be NP-hard problems, which cannot be solved by some predefined mathematic formulae. In this case, computer aided method is very helpful. While some local search algorithms are easily to fall into a local optimum solution. On contrast, the population based methods, such as Genetic Algorithms, Artificial Immune System, Autonomy Oriented Computing, are global search algorithms. However, they are not good at the local search. In this paper, an improved method is proposed by combining the local and global search ability, so as to improve the performance in terms of the convergence speed and the convergence reliability. We construct a generic form to deal with the common objective function space or the objective function with the partial derivative. In addition, we present an n-hold method in population based evolution method. The experiments indicate that our approach can effectively improve the convergence reliability, which is much concerned in some applications with the expensive executing expense. [ABSTRACT FROM AUTHOR]

Drawing upon the past research on neuromorphic implementation of an emotionally intelligent control system, a developed version of the brain emotional learning based intelligent controller, BELBIC, is proposed in this paper. The modified system, which essentially realizes direct adaptive control scheme, deals very well for SISO plants, and does not need any pre-training. It is adaptive and robust with respect to changes in the environment since it learns to produce appropriate control actions on-line. An additional advantage of BELBIC is its simplicity as well as low computational load. The performance of the system is demonstrated via application on previously studied benchmarks involving optimization and adaptive control in nonlinear systems with MLP-backpropagation (BP) nets. [ABSTRACT FROM AUTHOR]

The use of evolutionary algorithms (EAs) to solve problems with multiple objectives (known as Vector Optimization Problems (VOPs)) has attracted much attention recently. Being population based approaches, EAs offer a means to find a group of pareto-optimal solutions in a single run. Differential Evolution (DE) is an EA that was developed to handle optimization problems over continuous domains. The objective of this paper is to introduce a novel Pareto Differential Evolution (PDE) algorithm to solve VOPs. The solutions provided by the proposed algorithm for five standard test problems, is competitive to nine known evolutionary multiobjective algorithms for solving VOPs. [ABSTRACT FROM AUTHOR]

In this paper we introduce a new method based on Russian Doll Search (RDS) for solving optimization problems expressed as Valued Constraint Satisfaction Problems (VCSPs). The RDS method solves problems of size n (where n is the number of variables) by replacing one search by n successive searches on nested subproblems using the results of each search to produce a better lower bound. The main idea of our method is to introduce the variables through the successive searches not one by one but by sets of k variables. We present two variants of our method: the first one where the number k is fixed, noted k[sub f] RDS; the second one, k[sub v] RDS, where k can be variable. Finally, we show that our method improves RDS on daily management of an earth observation satellite. [ABSTRACT FROM AUTHOR]

Many new approaches to solve optimization problems have been proposed with the appearance of methods based on analogies to natural processes. For the user in practice, it is not only difficult to assess which method is the "best" one but also to judge how easy or complicated it is to implement and apply the methods. This is mainly due to the fact that most methods have interacting parameters which need to be tuned before a method can be used. The paper discusses ease of use and implementation as well as performance aspects based on computational tests. Three iterative improvement methods (simulated annealing, tabu search, threshold accepting) and two evolutionary methods (genetic algorithms, PRSA) are compared with regard to an optimization problem (standard cell placement in chip design). Based on results from a large number of test runs, we discuss how easy or difficult it is to prepare the methods for a specific application. In summary, iterative improvement methods were found easier to implement and to use than evolutionary methods. They also outperformed the latter ones in terms of solution quality and computing time. [ABSTRACT FROM AUTHOR]

This paper studies various strategies in constrained simulated annealing (CSA), a global optimization algorithm that achieves asymptotic convergence to constrained global minima (CGM) with probability one for solving discrete constrained nonlinear programming problems (NLPs). The algorithm is based on the necessary and sufficient condition for discrete constrained local minima (CLM) in the theory of discrete Lagrange multipliers and its extensions to continuous and mixed-integer constrained NLPs. The strategies studied include adaptive neighborhoods, distributions to control sampling, acceptance probabilities, and cooling schedules. We report much better solutions than the best-known solutions in the literature on two sets of continuous benchmarks and their discretized versions. [ABSTRACT FROM AUTHOR]

Regional Knowledge is useful in identifying patterns of relationships between variables, and it is particularly important in solving constrained global optimization problems. However, regional knowledge is generally unavailable prior to the optimization search. The questions here are: 1) Is it possible for an evolutionary system to learn regional knowledge during the search instead of having to acquire it beforehand? and 2) How can this regional knowledge be used to expedite evolutionary search? This paper defines regional schemata to provide an explicit mechanism to support the acquisition, storage and manipulation of regional knowledge. In a Cultural Algorithm framework, the belief space "contains" a set of these regional schemata, arranged in a hierarchical architecture, to enable the knowledge-based evolutionary system to learn regional knowledge during the search and apply the learned knowledge to guide the search. This mechanism can be used to guide the optimization search in a direct way, by "pruning" the infeasible regions and "promoting" the promising regions. Engineering problems with nonlinear constraints are tested and the results are discussed. It shows that the proposed mechanism is potential to solve complicated non-linear constrained optimization problems, and some other hard problems, e.g. the optimization problems with "ridges" in landscapes. [ABSTRACT FROM AUTHOR]

This article considers a problem of solving the optimal solution of the sum of locally convex cost functions over an undirected network. Each local convex cost function in the network is accessed only by each unit. To be able to reduce the amount of computation and get the desired result in an accelerated way, we put forward a fresh accelerated decentralized event-triggered algorithm, named as A-DETA, for the optimization problem. A-DETA combines gradient tracking and two momentum accelerated terms, adopts nonuniform step-sizes and emphasizes that each unit interacts with neighboring units independently only at the sampling time triggered by the event. On the premise of assuming the smoothness and strong convexity of the cost function, it is proved that A-DETA can obtain the exact optimal solution linearly in the event of sufficiently small positive step-size and momentum coefficient. Moreover, an explicit linear convergence speed is definitely shown. Finally, extensive simulation example validates the usability of A-DETA. [ABSTRACT FROM AUTHOR]

Deep neural networks become more popular as its ability to solve very complex pattern recognition problems. However, deep neural networks often need massive computational and memory resources, which is main reason resulting them to be difficult efficiently and entirely running on embedded platforms. This work addresses this problem by saving the computational and memory requirements of deep neural networks by proposing a variance reduced (VR)-based optimization with regularization techniques to compress the requirements of memory of models within fast training process. It is shown theoretically and experimentally that sparsity-inducing regularization can be effectively worked with the VR-based optimization whereby in the optimizer the behaviors of the stochastic element is controlled by a hyper-parameter to solve non-convex problems. [ABSTRACT FROM AUTHOR]

Voting is a commonly used ensemble method aiming to optimize classification predictions by combining results from individual base classifiers. However, the selection of appropriate classifiers to participate in voting algorithm is currently an open issue. In this study we developed a novel Dissimilarity-Performance (DP) index which incorporates two important criteria for the selection of base classifiers to participate in voting: their differential response in classification (dissimilarity) when combined in triads and their individual performance. To develop this empirical index we firstly used a range of different datasets to evaluate the relationship between voting results and measures of dissimilarity among classifiers of different types (rules, trees, lazy classifiers, functions and Bayes). Secondly, we computed the combined effect on voting performance of classifiers with different individual performance and/or diverse results in the voting performance. Our DP index was able to rank the classifier combinations according to their voting performance and thus to suggest the optimal combination. The proposed index is recommended for individual machine learning users as a preliminary tool to identify which classifiers to combine in order to achieve more accurate classification predictions avoiding computer intensive and time-consuming search. [ABSTRACT FROM AUTHOR]

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets. [ABSTRACT FROM AUTHOR]

A multi-stage krill herd (MSKH) algorithm is presented to fully exploit the global and local search abilities of the standard krill herd (KH) optimization method. The proposed method involves exploration and exploitation stages. The exploration stage uses the basic KH algorithm to select a good candidate solution set. This phase is followed by fine-tuning a good candidate solution in the exploitation stage with a focused local mutation and crossover (LMC) operator in order to enhance the reliability of the method for solving global numerical optimization problems. Moreover, the elitism scheme is introduced into the MSKH method to guarantee the best solution. The performance of MSKH is verified using twenty-five standard and rotated and shifted benchmark problems. The results show the superiority of the proposed algorithm to the standard KH and other well-known optimization methods. [ABSTRACT FROM AUTHOR]

Harmony search algorithm (HSA) is a music-inspired population-based meta-heuristic search and optimization algorithm. In order to improve exploration or global search ability, exploit local search more effectively, increase convergence speed, improve solution quality, and minimize computational cost, researchers have advanced the concept of hybridizing HSA with other algorithms. This article presents a review of hybrid harmony search algorithms. [ABSTRACT FROM AUTHOR]

The NP-hard problem of scheduling jobs on unrelated parallel machines with the objective of minimizing the makespan is addressed. A variable neighborhood descent heuristic hybridized with mathematical programming is proposed. A new constructive heuristic is also introduced to seed the search with a solution residing in a promising region. The strength of the approach lies in the adoption of appropriately chosen optimization criteria that avoid early local optimum entrapment and in the hybridization of the heuristic with mathematical programming for the exploration of the large neighborhood structures. Experimental results on a large set of benchmark problem instances attest to the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

Foraging behavior of bees in finding food resource is one of the useful patterns to develop algorithms for solving complex problems. This article by simulation of such behavior and consider a memory for them proposed a method in discrete spaces. The proposed method is applied to Travel Salesman Problem (TSP) and successfully solved it. Simulation results have been proved the performance of our algorithm compared to similar strategies. [ABSTRACT FROM AUTHOR]

Support Vector Machines (SVMs) are based on the concept of decision planes that define decision boundaries, and Least Squares Support Vector (LS-SVM) Machine is the reformulation of the principles of SVM. In this study a diagnosis on a BUPA liver disorders dataset, is conducted LS-SVM with the Taguchi method. The BUPA Liver Disorders dataset includes 345 samples with 6 features and 2 class labels. The system approach has two stages. In the first stage, in order to effectively determine the parameters of the kernel function, the Taguchi method is used to obtain better parameter settings. In the second stage, diagnosis of the BUPA liver disorders dataset is conducted using the LS-SVM classifier; the classification accuracy is 95.07%; the AROC is 99.12%. Compared with the results of related research, our proposed system is both effective and reliable. [ABSTRACT FROM AUTHOR]

The Lexicographic Path Ordering (LPO) poses an interesting problem to the implementor: How to achieve a version that is both efficient and correct? The method of program transformation helps us to develop an efficient version step-by-step, making clear the essential ideas, while retaining correctness. By theoretical analysis we show that the worst-case behavior is thereby changed from exponential to polynomial. Detailed measurements show the practical improvements of the different variants. They allow us to assess experimentally various optimizations suggested for LPO. [ABSTRACT FROM AUTHOR]