Search

K-nearest neighbors search (K-NNS) is a fundamental problem in many areas of machine learning and data mining. In an attempt to solve NNS problems by locality-sensitive hashing (LSH)-based algorithms and avoid the F1-trap, in this paper, we theoretically prove the necessity of maintaining close edges of each dimension, the independence of the partition location and the precision of retrieval. Based on the theories, a new method is proposed for constructing the hash function family. Then, a volume-balancing locality-sensitive hashing (VBLSH) algorithm is implemented. In this algorithm, the problem of constructing the hash function family is transformed into an optimization problem under a fixed number of hypercubes. By solving the optimization problem, different hyperplane intervals are set for each dimension to ensure that the hash functions can evenly segment the data space, where each segmented hypercube has a close volume and close side lengths. Finally, the analysis of the algorithm shows that the VBLSH algorithm has a high index coding efficiency and hash code calculation efficiency. The experimental results show that the VBLSH algorithm can simultaneously take into account the efficiency, precision and recall rate, and avoid the F1-trap; it shows excellent effectiveness on 4 datasets.

This paper presents a new Reinforcement Learning (RL)-based control approach that uses the Policy Iteration (PI) and a metaheuristic Grey Wolf Optimizer (GWO) algorithm to train the Neural Networks (NNs). Due to an efficient tradeoff to exploration and exploitation, the GWO algorithm shows good results in NN training and solving complex optimization problems. The proposed approach is compared to the classical PI RL-based control approach using the Gradient Descent (GD) algorithm, and with the RL-based control approach which uses the metaheuristic Particle Swarm Optimization (PSO) algorithm. The experiments are conducted using a nonlinear servo system laboratory equipment. Each approach evaluated on how well it solves the optimal reference tracking control for an experimental servo system position control system. The policy NNs specific to all three approaches are implemented as state feedback with integrator controllers to remove the steady-state control errors and thus ensure the convergence of the objective function. Because of the random nature of metaheuristic algorithms, the experiments for GWO and PSO algorithms are run multiple times and the results are averaged before the conclusions are presented. The experimental results shows that for the control objective considered in this paper, the GWO algorithm represents a better solution compared to GD and PSO algorithms.

Multi-modal multi-objective optimization problem (MMOPs) has attracted more and more attention in evolutionary computing recently. It is not easy to solve these problems using the existing evolutionary algorithms. The non-dominated solution sorting genetic algorithm (NSGA-II) has poor PS distribution and convergence. In this paper, an enhanced fast NSGA-II based on a special congestion strategy and adaptive crossover strategy, namely ASDNSGA-II is proposed. In the ASDNSGA-II, the strategy with a special congestion degree is used to improve the selection strategy. Then a new adaptive crossover strategy is designed by evaluating the advantages and disadvantages of the SBX crossover strategy with the ability to solve high dimensions and the BLX-α with the ability of Pareto solution to produce offspring solutions. These can ensure the generation of offspring solutions around individuals with large crowding degrees and balance the convergence and diversity of decision space and object space. It can improve PS distribution and convergence and maintain PF precision. Eight functions of MMF1-MMF8 from the CEC2020 are selected to prove the effectiveness of the ASDNSGA-II. By comparing several latest multi-modal multi-objective evolutionary algorithms, the results show that the ASDNSGA-II can effectively find the global Pareto solution set and improve the distribution and convergence of PS.

In the constrained multi-objective optimization problems, the pursuit of feasibility could improve convergence but will lead to the loss of diversity. For optimization algorithm, balancing the weight between convergence and diversity dynamically is a challenge, especially in problems with low proportion of feasible regions. In this paper, a constrained multi-objective optimization algorithm is proposed based on a hybrid driven strategy to enhance both the feasibility and diversity performance of the approximate Pareto solutions. The proposed algorithm contains two archives, that one is driven by feasibility information and the other is driven by diversity information. A self-adaptive archive selection mechanism and a conditional tournament selection strategy are proposed to provide mating parent solutions according to the evolutionary stage. Moreover, in the update of the feasibility archive, an evolutionary direction prediction mechanism is proposed and adopted to improve the evolutionary efficiency. Compared to four other multi-objective algorithms on three benchmark suits of different types, the performance of the proposed algorithm is better than the peer algorithms, especially in large-infeasible-regions multi-objective optimization problems.

Following two decades of sustained studies, metaheuristic algorithms have made considerable achievements in the field of multi-objective optimization problems (MOPs). However, under most existing metaheuristic frameworks, an improved scheme introduced to address specific defects usually leads to additional problems that need to be solved further. Emerginging optimization mechanisms should be considered to break the bottleneck, and an adaptive multi-objective competitive swarm optimization (AMOCSO) algorithm, a promising option for solving MOPs, is proposed in this paper. Firstly, the competitive mechanism is modified so that it can perform well on MOPs, and an improved learning scheme is designed for the winners and the losers, which greatly enhances the optimization efficiency and balances the convergence and the diversity of the proposed algorithm. Then, an external archive and its maintenance schemes are introduced to prevent the population from degenerating and make the algorithm framework more comprehensive. Moreover, a practical adaptive strategy is proposed to fill the blank of parameter research, and no human factors exist in AMOCSO, which means that an amazing promotion can be achieved in generalization. Finally, abundant experimental studies are carried out, and the results of comparative experiments show that the proposed algorithm has significant advantages over several state-of-the-art algorithms.

In recent years, many efficient evolutionary multitasking (EMT) algorithms have been proposed to solve multi-objective multi-task optimization problems. However, EMT algorithms often face negative transfer problems. In this paper, a novel multi-objective evolution strategy, called CT-EMT-MOES, is proposed based on a cultural transmission theory for solving multi-objective multitask optimization problems. First, two evolutionary operators inspired by cultural transmission theory are proposed. The elite-guided variation strategy can transfer the information from the current Pareto front to all individuals and guide the population to quickly converge. The horizontal cultural transmission strategy can efficiently transfer information from the source task to the target task. Second, to solve the negative transfer problem, an adaptive information transfer strategy is proposed to adaptively adjust the probability of an information transfer. Third, the proposed algorithm can gain a Pareto front with good convergence and diversity by utilizing a smaller population size and fewer computing resources. As a result, the proposed algorithm can effectively utilize the implicit similarity and complementarity between simultaneous optimized tasks to improve the overall convergence efficiency and reduce a negative transfer. Finally, comprehensive experimental results show that the proposed algorithm can achieve a better performance compared with previous state-of-the-art multi-objective EMT algorithms.

Cancer survivability prediction has been of great interest to health professionals and researchers. The task refers to the procedure of estimating the potential survivability according to an individual’s medical history. The difficulty is that raw data is usually subject to some noise, such as missing values. To address this issue, we propose a novel low-rank and sparse representation-based learning algorithm, which consists of two main stages of data self expressiveness and classification. Firstly, in the data self expressiveness stage, raw inputs have been decomposed into one dictionary (which is enforced with a low-rank constraint) and one coefficient matrix (which is sparsely coded), respectively. Secondly, this sparse coefficient matrix is paired with sample labels for training during the classification stage. We further integrate these two stages and formulate them into an optimization problem, which is then solved using an iterative computational strategy. Theoretically, we analyze the convergence of the proposed algorithm. The relationship between the proposed algorithm and existing approaches are also discussed. The efficiency of the proposed algorithm is experimentally verified using several benchmarking classification problems and a public longitudinal dataset. Experimental results demonstrate that the proposed algorithm achieves superior performance in terms of affordable computational complexity and high prediction accuracy, compared to state-of-the-art approaches.

This paper proposes a particle swarm optimization with strategy dynamics (SDPSO) to solve single-objective optimization problems. SDPSO consists of four PSO search strategies. Evolutionary game theory is introduced to control the population state. In evolutionary game theory, through the interaction between players, better strategies will eventually dominate among the players. By extending this idea to PSO, a selection mechanism and a mutation mechanism are proposed. By using the selection mechanism, the adoption probability of the high payoff strategies will increase. The mutation mechanism can examine the stability of the incumbent strategy to evolutionary pressures. The performance of SDPSO is compared with 14 algorithms on the CEC 2014 test suite. The results show that SDPSO has the highest rank. SDPSO is applied to solve a real-world problem. SDPSO can find the best mean results comparing with 4 algorithms. The findings show that the proposed evolutionary game theory-based framework can adaptively control the population state. This study proposes a new application of evolutionary game theory to the design of swarm intelligence and contributes to a better understanding of the usefulness of the evolutionary game theory in the optimization method. The source codes of SDPSO are available at https://github.com/zi-ang-liu/SDPSO .

In this paper, we consider the predefined-time distributed optimization problems of homogeneous and heterogeneous linear multi-agent systems under undirected and connected communication topologies, in which, all agents share coupled equality constraint. In order to make all agents converge to the optimal output at predefined-time cooperatively, we propose two distributed algorithms for homogeneous and heterogeneous multi-agent systems according to time-base generator technology and output feedback, therein, the optimal output can make the global cost function reach minimum. In the design of the algorithms, the control gains are not required, which can avoid the requirement of some global information in advance. Furthermore, all agents converge to the optimal output with exponential speed, and we can set the convergence time arbitrarily. Finally, we provide examples to illustrate the effectiveness of the proposed distributed algorithms.

When solving multi-objective optimization problems with complex Pareto fronts characteristics, previous work generally ignores the information related to Pareto fronts provided by the population during the evolution, which is detrimental to efficiently tackle them. In order to taking full advantage of information associated with a population evolution, a multi-objective evolutionary optimization method based on online perceiving Pareto front characteristics is proposed in this study. To this end, the information associated with the Pareto front of an optimization problem is first extracted from the population. Following that, the characteristics of the Pareto front in concavity/convexity and continuity are perceived online. For the purpose of each sub-front containing only one characteristic, a Pareto front is divided based on the concavity/convexity and continuity. According to the characteristic of each sub-front, different reference points are selected to refine the distribution of reference vectors. Finally, a multi-objective evolutionary algorithm is designed targeting the characteristics of the Pareto front. The performance of the proposed method is evaluated by comparing it with 8 state-of-the-art optimizers on 31 test problems. Further, the experimental results demonstrate that the proposed method is competitive in handling multi-objective optimization problems with irregular Pareto fronts.

Multi-label learning algorithms have significant challenges due to high-dimensional feature space and noises in multi-label datasets. Feature selection methods are effective techniques to deal with these problems. ParetoCluster is an effective multi-label feature selection algorithm based on Pareto dominance and cluster analysis concepts which considers each label an objective function. This algorithm loses its effectiveness to differentiate features when dealing with high labeled datasets and makes most features incomparable (e.g., when most features fall into the first layer). Thus, a cluster analysis criterion in ParetoCluster will play a decisive role in determining the most relevant features. Bearing this in mind, in this paper, we have modeled the multi-label feature selection problem into a bi-objective optimization problem regarding the relevancy and redundancy degree of the features. We then handle it using Pareto dominance in this bi-objective domain. To illustrate the optimality and efficiency of the proposed method, we have compared our approach against some similar techniques.

Both problem characteristics in multimodality and multi-objective are involved in multimodal multi-objective optimization problems (MMOPs). How to locate diverse Pareto sets and approximate Pareto front simultaneously is a challenging research topic. To address this issue, a cluster-based immune-inspired algorithm using manifold learning is proposed in this paper for solving MMOPs. First of all, the population is partitioned into multiple subpopulations, and each of them is expected to find equivalent Pareto solutions in different regions. Subsequently, the immune-inspired algorithm with proportional cloning and hypermutation is developed for improving the diversity of the population and obtaining high-quality Pareto solutions in the decision space. Additionally, principal component analysis is adopted to learn the manifold of the Pareto set, further improve the convergence, and enhance interaction among subpopulations. The proposed algorithm is compared with six state-of-the-art algorithms. Experimental results demonstrate that the proposed algorithm is capable of locating equivalent Pareto optimal solutions in the decision space and maintaining the diversity and convergence of solutions in both decision space and objective space, simultaneously.

The prediction-error expansion (PEE) based reversible data hiding (RDH) methods mainly contain two steps: prediction-error histogram (PEH) generation and PEH modification . Multiple histograms modification (MHM) and pairwise PEE are two efficient methods used in PEH modification, which can greatly reduce the distortion to the cover image under given payloads. However, these two effective methods are not used together for their conflict on causality and large computational complexity. To address the above two problems, in this paper, a group of non-causal predictors with a specially designed complexity metric are proposed so that the conflict on causality is resolved. Then, the original optimization problem to determine the parameters of MHM and pairwise PEE is decomposed into two easier ones, which are addressed by a step-by-step optimization. Several efficient constraints are used to narrow the search range for the optimal parameters while minimizing the performance degradation . As a result, the proposed method can obtain high quality on the embedded images, that is, the average PSNR of the 8 standard 512 × 512 sized grayscale images from the USC-SIPI image database can reach 60.43 dB for the payload of 10,000 bits, while the running time for the proposed algorithm is acceptable. Experimental results show that the proposed method outperforms many kinds of RDH schemes , and achieves an obvious increment in fidelity over a series of state-of-the-art RDH schemes.

Particle swarm optimization (PSO) has been employed to solve numerous real-world problems because of its strong optimization ability and easy implementation. However, PSO still has some shortcomings in solving complicated optimization problems, such as premature convergence and poor balance between global exploration and local exploitation. A novel hybrid particle swarm optimization using adaptive strategy (ASPSO) is developed to address associated difficulties. The contribution of ASPSO is threefold: (1) a chaotic map and an adaptive position updating strategy to balance exploration behavior and exploitation nature in the search progress; (2) elite and dimensional learning strategies to enhance the diversity of the population effectively; (3) a competitive substitution mechanism to improve the accuracy of solutions. Based on various functions from CEC 2017, the numerical experiment results demonstrate that ASPSO is significantly better than the other 16 optimization algorithms. Furthermore, we apply ASPSO to a typical industrial problem, the optimization of melt spinning progress, where the results indicate that ASPSO performs better than other algorithms.

A gradient-based search method (GBSM) is developed to solve multi-objective optimization problems. It uses the multi-objective gradient information to construct descent directions, i.e., Pareto descent directions (PDDs), to accelerate the convergence. In addition, a multi-objective evolutionary algorithm based on decomposition is adopted to improve the diversity. The comparisons between GBSM with several selected multi-objective evolutionary algorithms and gradient based algorithms on benchmark functions indicate that the proposed method performs competitively and effectively.

Particle Swarm Optimization (PSO) is a simple yet efficient population based evolutionary algorithm, which has been widely applied to solve global optimization problems. Similar to other evolutionary algorithms (EAs), PSO algorithm still suffers from loss of diversity and slow convergence caused by the contradiction between exploration and exploitation abilities. In order to address the drawbacks, we propose Self-Adaptive two roles hybrid learning strategies-based particle swarm optimization (SAHLPSO) to improve optimization performance by using exploration-role and exploitation-role learning strategies with self-adaptively updating parameters manner. The exploration-role learning strategies with external archive adopt comprehensive learning to generate the learning exemplars, which can effectively maintain population diversity and thus avoid premature. On the other hand, the exploitation-role learning strategies utilize elitism learning to generate the learning exemplars so as to improve convergence performance. Considering that the performance improvement of PSO depends on not only the adaptation of different learning strategies but also the reasonable parameter settings, we also develop a self-adaptive parameter updating manner based on success history information. This manner can automatically update the learning parameters to appropriate values and avoid a user’s prior knowledge about the relationship between the parameter settings and the characteristics of optimization problems. In addition, the usage of the linear population size reduction is also helpful to well balance the exploration and exploitation along with the process of evolution. Simulation results on some classical and CEC2013 benchmark functions show that LSAHLPSO is better than other PSO variants or at least comparable to other state-of-the-art evolutionary algorithms.

Recently, multi and many-objective evolutionary algorithms (MOEAs) embedded with clustering techniques to enhance their environmental selection show promising performance for tackling multi and many-objective optimization problems (MOPs) with irregular Pareto fronts (PFs). However, the similarity metric used for clustering cannot reflect the diversity of solutions fairly when it is measured based on the direction distances of solutions to an ideal point. Consequently, it may mislead the environmental selection when solving MOPs with convex PFs. To alleviate this issue, this paper suggests an MOEA using clustering with a flexible similarity metric to run the environmental selection. In our approach, we first use a simple yet effective method to roughly predict the concavity or convexity of the target problem. Then, a flexible reference point is set to define the direction distances of solutions, which can properly measure the similarity between solutions and fairly show their diversity for solving MOPs with various PF shapes. After that, we use a hierarchical clustering method with this flexible similarity metric to run the environmental selection on all solutions, which will properly classify them into N clusters (N is the population size). Finally, in each cluster, one solution with the best convergence value will survive to compose the new population. When compared to twenty-seven competitive MOEAs in solving nineteen benchmark MOPs and sixteen real-world engineering MOPs with various PF shapes, the experimental results demonstrate that the proposed algorithm has significant advantages.

A key issue in evolutionary constrained optimization is how to achieve a balance between feasible and infeasible solutions. The quality of generated solutions in decomposition-based multi-objective evolutionary algorithms (MOEAs) depends strongly on the weights’ setting. To fully utilize both the promising feasible and infeasible solutions, this paper proposes two-type weight adjustments based on MOEA/D for solving highly constrained many-objective optimization problems (CMaOPs). During the course of the search, the number of infeasible weights is dynamically reduced, to guide infeasible solutions with better convergence to cross the infeasible barrier, and also to lead infeasible solutions with better diversity to locate multiple feasible subregions . Feasible weights are evenly distributed and keep unchanged throughout the evolution process, which aims to guide the population to search Pareto optimal solutions . The effectiveness of the proposed algorithm is verified by comparing it against six state-of-the-art CMaOEAs on three sets of benchmark problems. Experimental results show that the proposed algorithm outperforms compared algorithms on majority problems, especially on highly constrained optimization problems . Besides, the effectiveness of the proposed algorithm has also been verified on an antenna array synthesis problem .

In recent years, many graph compression methods have been introduced. One successful category of them is based on local decompression designed to answer neighborhood queries. These techniques mainly rely on local similarities of vertices. Besides, their performance is usually a function of graph sparsity. The proposed approach, in this paper, is a lossy compression technique used to answer neighborhood queries with a more general precondition, called transitivity. The output of this method is a sparse graph optimized to keep original adjacent vertices, in at most 2-distance from each other and vice versa. In other words, by traversing a compressed graph by depth of 2, from any desired vertex, its original adjacency list is reconstructed, with an acceptable error. This paper models an optimization problem to solve the inverse problem of finding the best compressed graph in order to minimize the reconstruction error. Then, this NP problem is approximated by a heuristic with a low degree polynomial time-complexity near to the complexity of the forward problem. The results of applying the proposed method on toy and real datasets are compared with the state of the art that improves compression ratio and performance with an acceptable query response time.

Multimodal multi-objective optimization problems (MMOPs) involve locating equivalent Pareto optimal solutions in decision space with the same objective values. The key to handling MMOPs is finding all equivalent Pareto optimal solutions and maintaining a promising balance between the convergence and diversity of solutions in both decision space and objective space. To tackle this issue, a self-organizing quantum-inspired particle swarm optimization algorithm (MMO_SO_QPSO) is proposed in this paper for handling MMOPs. In the proposed MMO_SO_QPSO, a self-organizing map is used to find the best neighbor leader of particles. With the aid of neighbor leader particles, a special zone searching method is adopted to update the position of particles and locate equivalent Pareto optimal solutions in decision space. To maintain diversity and convergence of Pareto optimal solutions, a special archive mechanism that relies on the maximum-minimum distance among solutions is introduced into MMO_SO_QPSO. And some outstanding Pareto optimal solutions are maintained in the special archive. In addition, a new performance indicator is developed to estimate properly the similarity between obtained Pareto optimal solutions and true Pareto optimal solutions. The performance of the proposed MMO_SO_QPSO is compared with six state-of-the-art multimodal multi-objective evolutionary algorithms on two well-known benchmark problems. Experimental results demonstrate the superior performance of MMO_SO_QPSO for solving MMOPs. The effectiveness of several strategies is also discussed.

Solving nonlinear equation systems using evolutionary algorithms involves solving two key problems. One problem is how to efficiently optimize the nonlinear equations derived from the physical features, and the other problems is how to locate more than one optimal solution in a single trial. To address these two problems, an improved dynamic tri-objective differential evolution method is proposed in this paper. First, we transform a given system with any type and number of nonlinear equations into a dynamic tri-objective optimization problem, which targets the first problem. Second, we develop a self-adaptive ranking multi-objective differential evolution, which targets the second problem. In addition, a probability distribution-based local search is introduced, which aims to identify the optimal solutions with a high level of accuracy. Based on previous studies of numerical optimizations with multiple solutions, each component is elaborately proposed and developed, so it is more suitable for a nonlinear equation system. Experiments were conducted on 30 numerical examples collected from real-world applications. The statistical results are encouraging, showing that the performance of the proposed approach is better than that of eight state-of-the-art evolutionary algorithms, with respect to root ratio and success rate metrics.

Differential evolution (DE) has been recognized as one of the most effective algorithms for solving numerical optimization problems. In this paper, we propose a tri-population differential evolution (TPDE) to further enhance the search capability of DE. More specifically, the parent population is partitioned into three sub-populations with different emphasises at each iteration based on a newly proposed zonal-constraint stepped division (ZSD) mechanism, which determines the size of each sub-population according to not only individual’s fitness value but also the evolutionary process. To make the best of information provided by elite individuals and play their leading role on other individuals, three elite-guided mutation strategies are presented for each sub-population. Moreover, three sets of adaptive control parameters including the scale factor F and crossover rate CR are configured for three mutations according to Gaussian distribution model, Cauchy distribution model and a triangular wave function respectively. The design of mutation strategies and control parameters for each sub-population is based on the principle of balancing the global exploration and local exploitation capabilities. To evaluate the performance of TPDE, comparative experiments are conducted based on 59 benchmark functions from CEC2014 and CEC2017 test suites. The results indicate that the proposed TPDE is significantly better than, or at least comparable to the recent nine state-of-the-art DE variants.

In this paper, we study the distributed cooperative optimization problem with globally equality and inequality constraints for a multi-agent system, where each agent is modeled by Euler-Lagrangian (EL) dynamics. The optimized function can be represented by the sum of all local cost functions corresponding to each individual agent. Two continuous-time algorithms are proposed to solve such the problem in a distributed manner. In virtue of geometric graph theory, convex analysis , and Lyapunov stability theory , it is proved that all agents achieve consensus on the Lagrangian multipliers associated with constraints while the proposed algorithms converge exponentially to the optimal solution of the problem given in the case that the parameters of EL agents are known, and converge asymptotically to the optimal solution of the problem in the case that the parameters of EL agents are unknown, respectively. Finally, an example is provided to demonstrate the effectiveness of the theoretical results.

Hybridization in context to Evolutionary Computation (EC) strives to combine operators, components, and the best merits of different EC paradigms, to form a new evolutionary algorithm that enjoys a statistically superior performance, compared to its ancestors, over a wide range of application-specific optimization problems. In this paper, we propose a simple yet powerful amalgam composed of a modified Cultural Algorithm (CA) that is supported with an Enhanced Levy Flight Search (ELFS) to guide the search and further promote the harmony between the explorative and exploitative capacities of the conventional techniques. The novel amalgam, denoted by q-aCA + m IS, utilizes a balanced search scheme where it employs an adapted Influence Function (IF) with a novel quality function that establishes a harmony between the Knowledge Sources (KSs) in the Belief Space (BS), and between the BS and other components in the hybrid to produce the most suitable knowledge needed for a certain search mode. The CA framework is reinforced with an updated Selection Function (SF) that employs a successful selection strategy that uses the extended situational knowledge for the future selection of individuals. The proposed algorithm is tested using more than 50 benchmark functions that are taken from the IEEE CEC’06, and the IEEE CEC’19 competitions on constrained real-parameter optimization. Moreover, three well-known engineering design problems are used to test the validity of the algorithm for the solution of complex real-life problems. The comparative study indicates that the q-aCA + mIS algorithm was able to obtain a statistically superior performance and scalability behavior over most of the considered functions in comparison with other state-of-the-art algorithms.

The recent proliferation of population-based meta-heuristics designed for solving optimization problems and their successes confirm that more promising techniques inspired by physical phenomena or biological systems are desired. Therefore, in this paper, a novel hybridization approach is proposed to improve the performance of optimization algorithms. The approach replaces a small number of worst solutions obtained by a meta-heuristic with predicted candidates without altering its search operators. Specifically, a target fitness value of the predicted candidate is determined based on the fitness of the population and a search strategy. Then, a calibration problem is solved to infer its decision variables. In this study, the proposed hybridization technique is applied to ten state-of-the-art population-based algorithms. The meta-heuristics and hybrids are evaluated on 82 functions, four engineering problems, and a new challenging problem of estimating a constant in Markov’s inequality using minimal polynomials of different degrees. The experimental results reveal the superiority of the hybrids over their counterparts and confirm the suitability of the proposed approach for improving the efficiency of meta-heuristics.

Motivated by applications in clustering and optimization, there has been significant interest in the variant for which the database models are high-dimensional data (such as imagesets or videos). Matrix factorization approaches, particularly Nonnegative Matrix Factorization (NMF) have achieved favorable results for data clustering tasks. However, these methods suffer from two major drawbacks. First, matrix factorization approaches only partially explore the inter- or intra- relationships between vector-valued data and are unable to handle high-dimensional data. Second, the clustering results largely rely on the predefined similarity matrix, which usually contain large amounts of noise. In this paper, we propose a novel Adaptive Graph Guided Concept Factorization model on Grassmann manifold (A G 3 CF) for imageset clustering in an unsupervised fashion, which imposes concept factorization over the space of linear subspaces. To further explore the structure of data and assign ideal neighbors, an adaptive graph regularization constraint is designed to automatically capture the local relationships of data samples and integrated well into concept factorization framework. An efficient algorithm is also derived to tackle the resulting optimization problem. Experimental results on several public datasets validate that the proposed approach achieves competitive performance in high-dimensional data clustering.

Differential Evolution (DE) is one of the most effective approaches in Evolutionary Computation to tackling complex black-box optimization, however, the overall performance of a DE variant is heavily dependent on its mutation strategy and the associated parameter control schemes. According to the “No free Lunch theorem”, every mutation strategy has its own fatal defect, therefore, DE variant with a single mutation strategy can not tackle all optimization problems. Furthermore, DE variant with ensemble of mutation strategies usually needs more parameter control schemes, which inevitably decreases the practicality of it. Here in this paper, a novel Cooperative Strategy based DE (CS-DE) with population diversity enhancement is proposed, and the CS-DE algorithm has the following contributions: First, a cooperative strategy pool containing two similar mutation strategies is advanced in the algorithm, and the two mutation strategies share the same parameter control. Second, a novel grouping strategy is proposed, then the two mutation strategies can be chosen in an adaptive manner by different individuals in generating trial vectors during the evolution. Third, novel adaptation schemes for the three control parameters F , CR and PS as well as a stagnation based re-initialization mechanism are also proposed in our CS-DE algorithm. 58 benchmarks from the CEC2013 and CEC2014 test suites are employed in algorithm validation, and experiment results show the competitiveness of the CS-DE algorithm with several state-of-the-art DE variants.

Teaching-learning-based optimization (TLBO) is a population-based metaheuristic algorithm which simulates the teaching and learning mechanisms in a classroom. The TLBO algorithm has emerged as one of the most efficient and attractive optimization techniques. Even though the TLBO algorithm has an acceptable exploration capability and fast convergence speed, there may be a possibility to converge into a local optimum during solving complex optimization problems and there is a need to keep a balance between exploration and exploitation capabilities. Hence, a Balanced Teaching-Learning-Based Optimization (BTLBO) algorithm is proposed in this paper. The proposed BTLBO algorithm is a modification of the TLBO algorithm and it consists of four phases: (1) Teacher Phase in which a weighted mean is used instead of a mean value for keeping the diversity, (2) Learner Phase, which is same as the learner phase of basic TLBO algorithm, (3) Tutoring Phase, which is a powerful local search for exploiting the regions around the best ever found solution, and (4) Restarting Phase, which improves exploration capability by replacing inactive learners with new randomly initialized learners. An acceptable balance between the exploration and exploitation capabilities is achieved by the proposed BTLBO algorithm. To evaluate the performance of BTLBO algorithm, several experimental studies are conducted on standard benchmark suits and the results are compared with several TLBO variants and state-of-the-art population-based optimization algorithms. The results are in excellent agreement and confirm the efficiency of BTLBO algorithm with accelerated exploitation and exploration capabilities with an appropriate balance between such criteria for solving complex optimization problems.

Broad Learning System (BLS) is an emerging network paradigm that has received considerable attention in the regression and classification fields. However, there are two deficiencies which seriously hinder its deployment in real applications. The first one is the internal correlations among samples are not fully considered in the modeling process. Second, the strict binary label matrix utilized in BLS provides little freedom for classification. In this paper, to address the above issues, we propose to impose group-sparsity constraints on the class-specific transformed features and label error terms, respectively. The effect is not only the more appropriate margins between data can be preserved, but also the learnt label space can be flexible for recognition. As a result, the obtained projection matrix can show more vital discriminative ability. Further, we employ the alternating direction method of multipliers to solve the resulting optimization problem. Extensive experiments and analysis on diverse benchmark databases are carried out to confirm our proposed model’s superiority in comparison with other competing classification methods.

This study aims to minimize the peak temperature for pipelined multi-core systems while providing hard real-time guarantees. Periodic Thermal Management (PTM) is adopted to control the temperature by periodically switching each pipelined stage into a lower power mode. The Pay-Burst-Only-Once principle from Real-Time Calculus theory is used in reverse to transform real-time guarantees into the constraints of the PTM schemes applied to all stages. We systematically study the peak temperature under PTM and present two algorithms to calculate it with different levels of accuracy and speed. A greedy principle-based heuristic method is proposed to solve the peak temperature optimization problem. Experiments are conducted on an Intel processor with physical temperature sensors . The results demonstrate that our approaches optimize the peak temperature more effectively than the sub-deadline partition approach. Simulations for scenarios involving more stages reveal that the proposed algorithms are scalable with respect to the number of stages.

Counterpoising convergence and distribution becomes more intractable in many-objective optimization where the number of objectives exceeds three, evolutionary algorithms (EAs) via decomposition are prominent in convergence promotion yet suffer from diversity loss. The setting of direction vectors (DVs), scalarizing function (SF) and selection strategy can significantly affect the performance of this sort of algorithms. To remedy the issue, we develop a hyperplane driven and projection assisted EA, referred here as HPEA, using three-stage search. At the very beginning, search is performed only along the extreme objective-wise points to capture the corners of Pareto front (PF). After that, the convergence and diversity are coordinated, a set of DVs, adapted by the evolving population itself, is utilized to extend the search wideness, and two novel SFs are exploited to collect elites in each subregion for approaching a more complete PF. At last, diversity is emphasized, a projection distance aided elimination mechanism is employed to prune poorly diversified solutions one by one. Note that hyperplane utilized at second stage aims at identifying well-converged solutions, the rationale behind using two novel SFs is to take complementary effects of different criteria. The resultant HPEA is compared with several state-of-the-art multiobjective EAs on handling various types of many-objective problems. Extensive empirical studies demonstrate the effectiveness and competitiveness of the proposal in obtaining high quality solution set.

Distance metric learning (DML) aims to learn distance metrics that reflect the interactions between features and labels. Due to the high computational complexity, existing DML models are unsuitable for large-scale datasets. This study proposes a DML approach for large-scale problems by reducing the number of variables, utilizing sparse structures of the optimization problems, and taking advantage of large-scale computation platforms. The proposed approach treats DML as a linear space transformation problem and suggests that a full DML matrix can be approximated by a diagonal matrix in many cases. We solve the diagonal DML problem along with its l 1 and l 2 regularizations via linear and quadratic programming. To facilitate large-scale learning problems, we design a MapReduce framework to build triplets, which are encapsulations of triple data points used for the optimization problem, and develop a weighting mechanism for triplets according to their contributions to the whole distance distortion. Experiments show that the proposed approach is fast in large-scale DML applications with comparable accuracy to much more time-consuming full matrix models. Since the approach is implemented with the Scala language based on the Spark platform, it can be used directly by productive Java applications, which makes it highly practical for large-scale datasets.

Semi-supervised learning is a new challenge that exploits the information of unlabeled instances. In this paper, we propose a novel Laplacian pair-weight vector projection (LapPVP) algorithm for semi-supervised classification . LapPVP consists of two stages: projection and classification. In the projection stage, LapPVP integrates a within-class scatter, a between-class scatter and a Laplacian regularization together and formulates a pair of optimization problems. The goal of LapPVP is to maximize the between-class scatter and minimize the within-class scatter on the basis of labeled data, and simultaneously maintain the geometric structure of labeled and unlabeled data in the projected subspace . The optimization problems of LapPVP are identical to generalized eigenvalue ones. Thus, it is easy to obtain the optimal solutions to LapPVP. In the classification stage, LapPVP adopts a minimal distance classifier to implement tasks in the projected subspace. The proposed LapPVP can find a better separability using the geometric information embedded in labeled and unlabeled instances and the discriminative information embedded in labeled instance. Experiments on artificial datasets and UCI datasets validate the feasibility and effectiveness of the proposed algorithm.

In this paper, a hybrid model with both clustering mechanism and regression mechanism is built with aid of an allocation of information granularity . Some numeric prototypes constructed by clustering methods are expanded into granular prototypes. The regression module accepts granular prototypes and outcomes information granules. Some advantages of this idea can be concluded: 1). The constructed granular prototypes are best able to represent original data with comparable smaller quantity and high quality. 2). The granular output could be used to predict a new sample’s real number output by giving a rational range. 3). It can be used to solve clustering and regression problems simultaneously. Two different granularity allocation strategies are proposed and experimented while constructing granular prototypes: non-uniformly allocation of information granularity to each cluster and non-uniformly allocation to each feature. A comprehensive objective function is defined considering specificity and generality of the output. The allocation of granularity itself is in fact a multiple-parameters optimization problem which invokes the usage of an evolutionary method. Two popular methods (GA and PSO) are tried and compared with a real data set’s experiment. Three data sets are collected from UCI website to testify the effectiveness of our approach in the experimental part.

A novel alternate evolution of sub-populations based on tensor decomposition called, TASE is proposed, for solving multi- and many-objective optimization problems with large-scale decision variables in this work. Tensor canonical polyadic (CP) decomposition is introduced for the first time to divide the heterogeneous variables of higher-dimensional decision space into several lower-dimensional sub-components. Furthermore, these sub-populations are optimized alternatively to search for improved solutions in their respective lower-dimensional decision subspace . Finally, a cross-population matching scheme is designed to reconstruct the whole population accurately. The experiments use some largescale multi- and many-objective problems with 2–6 objectives and 1000–5000 variables. The proposed algorithm is compared with other state-of-the-art algorithms, and the experimental results indicate that it can solve some problems that other well-known large-scale optimization algorithms cannot, as well as outperforming other algorithms in terms of solution quality and convergence rate.

Convergence is always a major concern for many-objective optimization problems. Over the past few decades, various methods have been designed for measuring the convergence. However, according to our mathematical and empirical analyses, most of these methods are more focused on the convergence, and may neglect the exploration of boundary solutions, resulting in the incomplete Pareto fronts and the poor extent of spread achieved among the obtained non-dominated solutions. Regarding this issue, this paper proposes a Many-Objective Evolutionary Algorithm with Adaptive Reference Vector (MaOEA-ARV). In MaOEA-ARV, an adaptive reference vector strategy is designed to dynamically adjust the reference vectors according to the current distribution of candidate solutions for ensuring the spread and convergence simultaneously. Additionally, a hierarchical clustering strategy is employed to adaptively partition candidate solutions into multiple clusters for the diversity of candidate solutions. Experimental results on DTLZ, BT, ZDT and WFG test suites with up to 12 objectives demonstrate the effectiveness of MaOEA-ARV.

Surrogate-assisted evolutionary algorithms (SAEAs), which use efficient surrogate models or meta-models to approximate the fitness function in evolutionary algorithms (EAs), are effective and popular methods for solving computationally expensive optimization problems. During the past decades, a number of SAEAs have been proposed by combining different surrogate models and EAs. This paper dedicates to providing a more systematical review and comprehensive empirical study of surrogate models used in single-objective SAEAs. A new taxonomy of surrogate models in SAEAs for single-objective optimization is introduced in this paper. Surrogate models are classified into two major categories: absolute fitness models, which directly approximate the fitness function values of candidate solutions, and relative fitness models, which estimates the relative rank or preference of candidates rather than their fitness values. Then, the characteristics of different models are analyzed and compared by conducting a series of experiments in terms of time complexity (execution time), model accuracy, parameter influence, and the overall performance when used in EAs. The empirical results are helpful for researchers to select suitable surrogate models when designing SAEAs. Open research questions and future work are discussed at the end of the paper.

Evolutionary algorithms have been proven to be effective in solving multi-objective optimization problems. However, their performance deteriorates progressively in handling many-objective optimization problems due to the sensitivity upon the curvature of Pareto front, as well as the implicit evaluation on similarity in high dimensionality. This paper proposes an on-line Pareto front curvature estimator for an adaptive selection, in which the achievement scalarizing function is used to identify the pivotal solution to extrapolate the geometric information. Then an adaptive scalarizing function based fitness assessment, which guarantees the Pareto optimality, is presented. The diversity of the Pareto optimal solutions is also ensured by introducing a novel similarity metric. Finally, an extensive experimental analysis is presented to corroborate the analytical result by evaluating problems with various types of Pareto fronts. The experimental results substantiate the efficacy of the results with competitive performance.

This paper is mainly devoted to the distributed second-order multi-agent optimization problem with unbalanced and directed networks, where the gradient of the private objective functions might be unbounded. To deal with this problem, a new distributed algorithm is proposed based on the local neighbor information and the private objective functions. By a coordination transformation, the closed-loop system is divided into two simple first-order subsystems. Under the assumption of the strong connectivity of networks, it is proved that all agents can collaboratively converge to some optimal solution of the team objective function. At last, we give two numerical examples to show the effectiveness of our proposed algorithm.

Velocity planning serves as an important issue in motion planning for autonomous vehicles. The presented paper proposes a novel velocity planning method with minimum moving time on the basis of path curvature which is accomplished in three steps. First, the assigned path is divided into some elementary parts based on the path curvature. Second, the velocity planning is transformed into an unconstrained optimization problem by assuming the velocity of vehicle to be a specific cubic polynomial on every elementary part to avoid a sudden acceleration in path switching. Finally, we use a modified projection particle swarm optimization (PPSO) algorithm to obtain the time-optimal velocity profile . The proposed method can generate a smooth time-optimal velocity profile while considering all possible relevant constraints. Three examples are provided on different types of path to demonstrate that the final velocity profile is efficient to avoid the sudden acceleration change. Furthermore, the modified PPSO algorithm in this paper is used to solve the optimization problem with high dimensional variables when its upper bound is known, which can not be achieved by the general PPSO algorithm.

Surrogate-assisted evolutionary algorithms (SAEAs) are potential approaches to solve computationally expensive optimization problems. The critical idea of SAEAs is to combine the powerful searching capabilities of evolutionary algorithms with the predictive capabilities of surrogate models . In this study, an efficient surrogate-assisted hybrid optimization (SAHO) algorithm is proposed via combining two famous algorithms, namely, teaching–learning-based optimization (TLBO) and differential evolution (DE). The TLBO is focused on global exploration and the DE is concentrated on local exploitation. These two algorithms are carried out alternately when no better candidate solution can be found. Meanwhile, a new prescreening criterion based on the best and top collection information is introduced to choose promising candidates for real function evaluations. Besides, two evolution control (i.e., the generation-based and individual-based) strategies and a top-ranked restart strategy are integrated in the SAHO. Moreover, a local RBF surrogate which does not need too many training samples is employed to model the landscapes of the target function. Sixteen benchmark functions and the tension/compression spring design problem are adopted to compare the proposed SAHO with other state-of-the-art approaches. Extensive comparison results demonstrate that the proposed SAHO has superior performance for solving expensive optimization problems.

Constrained multi-objective optimization problems (CMOPs) are difficult to handle because objectives and constraints need to be considered simultaneously, especially when the constraints are extremely complex. Some recent algorithms work well when dealing with CMOPs with a simple feasible region; however, the effectiveness of most algorithms degrades considerably for CMOPs with complex feasible regions. To address this issue, this paper proposes a multi-stage evolutionary algorithm, where constraints are added one after the other and handled in different stages of evolution. Specifically, in the early stages, the algorithm only considers a small number of constraints, which can make the population efficiently converge to the potential feasible region with good diversity. As the algorithm moves to the later stages, more constraints are considered to search the optimal solutions based on the solutions obtained in the previous stages. Furthermore, a strategy for sorting the constraint-handling priority according to the impact on the unconstrained Pareto front is proposed, which can accelerate the convergence of the algorithm. Experimental results on five benchmark suites and three real-world applications showed that the proposed algorithm outperforms several state-of-the-art constraint multi-objective evolutionary algorithms when dealing with CMOPs, especially for problems with complex constraints.

As a variant of the support vector machine (SVM), the twin support vector machine (TSVM) has attracted substantial attention; however, TSVM is sensitive to outliers. To remedy it,this study introduces the correntropy-induced loss (C-loss) function, which is a non-convex, bounded, smooth loss function, and proposes the C-loss TSVM (CTSVM). CTSVM is designed to perform the following optimizations: minimization of the C-loss function, the 2-norm regularization of model coefficients, and the distance between the positive (negative) samples and the positive-class (negative-class) hyperplane, which ensures that negative (positive) samples are far away from the positive (negative) hyperplane. CTSVM is cast into two half-quadratic optimization problems that can be solved by using an alternating iterative method. This approach enables the computational time of CTSVM to remain comparable to that of related methods, especially when solving linear problems. Experimental results were processed statistically to compare the performance of multiple algorithms on multiple datasets. These results confirmed that CTSVM outperforms similar classifiers in terms of its robustness to outliers and classification accuracy for binary tasks. In the future, we aim to investigate the performance of CTSVM with respect to multi-class classification tasks.

This paper presents a new fractal search space decomposition-based algorithm to address the issue of scaling up the divide and conquer approach to deal with large scale problems (up to 50 continuous decision variables). The proposed algorithm, called polyFrac, fractally decomposes the search space using hyper-polytopes. It allows moving throughout different granularity levels by only computing the average of vertices of a hyper-polytope to obtain the coordinates of the centroids. Only the most promising hyper-polytopes are decomposed into child-polytopes. Then, a simple deterministic local search (single solution-based metaheuristic) is used to perform the intensification process to find the best solution within the selected lowest hyper-polytope. The proposed algorithm performance is evaluated on the well-known SOCO 2011, CEC 2013, and CEC 2017 benchmarks and compared with 26 states of the art algorithms. A real-world optimization problem is also used to calibrate its performance. The obtained results show that polyFrac outperforms all the algorithms. Moreover, experimental results and analysis suggest that polyFrac is a highly competitive optimization algorithm for solving large-scale and complex optimization problems.

Optimum-path forest (OPF) is one of the efficient graph-based frameworks that can determine the patterns of input dataset by extracting the optimal partitions of graph obtained through encoding data into a graph. Since OPF was introduced based on simple assumptions without considering the requirements of large-scale problems, this machine learning is an effective algorithm only for a reasonable size of input datasets. To provide a scalable OPF, this study introduces a strong coreset for accelerating OPF algorithm. Applying this approach can expedite OPF procedure, especially when it is working on massive datasets. Accordingly, a novel algebra is developed to represent the problem of OPF as an optimization problem for the proposed coreset definition. A novel coreset construction algorithm that can approximate the OPF solutions is subsequently proposed in order to improve the OPF construction speed. The simulation results of diverse experiments on various benchmark datasets illustrate computation gain and superiority of the proposed algorithm in terms of the construction and classification speeds as compared to the original algorithm while displaying reliably accurate performance. The presented coreset construction algorithm performs the training and testing phases of OPF up to 6.1 and 4.9 times faster than before, respectively.

Multiview clustering partitions a set of data into groups by exploring complementary information of multiple views. The mainstream tries to project the multiview data into a commonly shared subspace and further discover the true data structure. Ideally, clusters in the subspace should share less semantics with each other so that distinct groups can be obtained while this exclusivity is not guaranteed in previous works. To this end, this paper proposes a non-negative matrix factorization based subspace learning method for exclusive multiview clustering, where the double-orthogonal constraints are imposed for the cluster exclusivity. Moreover, to boost the clustering performance, the proposed method also exploits the available labeled data and is extended into a semi-supervised manner. Particularly, by incorporating the propagated semi-supervised manifold regularizations, the limited supervised information is enriched and encoded in our method to guide the learning process. The formulated optimization problem can be solved by the derived iterative updating rules. Experimental results on seven public datasets demonstrate its promising performance against other state-of-the-art approaches.

Although there exist various adversarial attacking methods, most of them are performed by generating adversarial noises. Inspired by the fact that people usually set different camera parameters to obtain diverse visual styles when taking a picture, we propose the adversarial attributes, which generate adversarial examples by manipulating the image attributes like brightness, contrast, sharpness, chroma to simulate the imaging process. This task is accomplished under the black-box setting, where only the predicted probabilities are known. We formulate this process into an optimization problem. After efficiently solving this problem, the optimal adversarial attributes are obtained with limited queries. To guarantee the realistic effect of adversarial examples, we bound the attribute changes using L p norm versus different p values. Besides, we also give a formal explanation for the adversarial attributes based on the linear nature of Deep Neural Networks (DNNs). Extensive experiments are conducted on two public datasets, including CIFAR-10 and ImageNet with respective to four representative DNNs like VGG16, AlexNet, Inception v3 and Resnet50. The results show that at most 97.79 % of images in CIFAR-10 test dataset and 98.01 % of the ImageNet images can be successfully perturbed to at least one wrong class with only ⩽ 300 queries per image on average.

In recent years, many different differential evolution (DE) variants have been proposed to solve real-world optimization problems. However, the performance of them is largely determined by the selection of the mutation strategy, an approach to choose favorable mutation strategy when solving various optimization problems has attracted increasing attention recently. In this paper, we propose a DE with an adaptive mutation operator based on fitness landscape (FLDE). The application of fitness landscape to DE requires three stages. First, we analyzed the fitness landscape features of each benchmark training function, a total of 45 benchmark functions are taken from CEC2014 and 2015. Then, the relationship between three mutation strategies and fitness landscape features is trained by random forest (RF) offline. Finally, the trained RF is used to predict which mutation strategy should be utilized to perform mutation operator for each problem during the evolutionary process. Besides, a historical memory parameter adaption mechanism and population size linear reduction are applied to the FLDE. The CEC2017 benchmark set is utilized to perform the experiments, and five well-known DE variant algorithms are compared with the FLDE algorithm. The experimental results attest that the proposed FLDE algorithm is highly competitive with the other five DE algorithms.

Various evolutionary algorithms have been proposed for tackling many-objective optimization problems over the past three decades. However, these algorithms still suffer from the loss of selection pressures due to the existence of dominance resistance. To tackle this issue, this paper proposes a relative non-dominance matrix, based on which a fitness formula is defined. Empirical analyses show that solutions with smaller fitness values are likely to dominate more other solutions in the future evolutionary process, and play a vital role in enhancing the convergence toward to the true Pareto fronts. Additionally, to further ensure the diversity, k-means clustering strategy is combined with the relative non-dominance matrix for a new design of the environmental selection, where parameter k in the clustering strategy is adjusted adaptively. The proposed algorithm is extensively tested with four state-of-art algorithms on WFG, MaF and DTLZ test suites. Empirical comparisons demonstrate the competitiveness of the proposed algorithm regarding to the convergence, diversity and spread.

Multi-objective multi-modal optimization problems have recently received increasing attention in the field of evolutionary computation. Addressing such problems is not easy for existing evolutionary multi-objective algorithms (EMOAs) since they require finding solutions with good convergence and diversity in both objective and decision spaces. This study therefore proposes a new algorithm, namely, the preference-inspired coevolutionary algorithm (PICEAg) with an active diversity strategy, to deal with multi-objective multi-modal optimization problems. The proposed algorithm, denoted as MMPICEAg, adopts the popular coevolutionary framework of PICEAg and introduces a diversity-aware fitness assignment and a double-diversity archive update strategy to promote diversity in objective and decision spaces simultaneously. The performance of MMPICEAg is compared with that of three general EMOAs as well as four state-of-the-art multi-modal EMOAs. The comparison results on three sets of widely used benchmarks clearly demonstrate the effectiveness of MMPICEAg for multi-objective multi-modal optimization.