Search

Machine learning (ML) has been recently introduced to solving optimization problems, especially for combinatorial optimization (CO) tasks. In this paper, we survey the trend of leveraging ML to solve the mixed-integer programming problem (MIP). Theoretically, MIP is an NP-hard problem, and most CO problems can be formulated as MIP. Like other CO problems, the human-designed heuristic algorithms for MIP rely on good initial solutions and cost a lot of computational resources. Therefore, researchers consider applying machine learning methods to solve MIP since ML-enhanced approaches can provide the solution based on the typical patterns from the training data. Specifically, we first introduce the formulation and preliminaries of MIP and representative traditional solvers. Then, we show the integration of machine learning and MIP with detailed discussions on related learning-based methods, which can be further classified into exact and heuristic algorithms. Finally, we propose the outlook for learning-based MIP solvers, the direction toward more combinatorial optimization problems beyond MIP, and the mutual embrace of traditional solvers and ML components. We maintain a list of papers that utilize machine learning technologies to solve combinatorial optimization problems, which is available at https://github.com/Thinklab-SJTU/awesome-ml4co. [ABSTRACT FROM AUTHOR]

Mutual information (MI) is a widely used similarity measure in pattern recognition. MI uses entropy as a measure of uncertainty to quantify the structural similarity of two vectors. Replacing entropy with variance as a measure of uncertainty, an analogous class of similarity measures can be derived and estimated by regression techniques. Recently, the non-linear piecewise constant regression (PWCR) has been proposed to drive similarity measures of this scheme, leading to competitive alternatives of MI. Although PWCR is based on binning, the optimal binning technique for certain problems remained an open question. In this paper, we show mathematically that the optimal binning needs to be aligned with the expected relationship between the vectors being compared. In general, approximately optimal binnings can be found by combinatorial optimization, and in certain cases the optimal binning can be determined by k-means clustering. The theoretical findings are supported by numerical experiments that show a 2–5% increase in the AUC scores in simulated pattern recognition scenarios and improved feature rankings in feature selection problems. The results suggest that the proposed binning techniques could improve the performance of PWCR-driven similarity measures in real-world applications. [ABSTRACT FROM AUTHOR]

For NP-hard combinatorial optimization problems, it is usually challenging to find high-quality solutions in polynomial time. Designing either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods have provided new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input, masks, and decoder context vectors. The proposed framework aims to improve the models in literacy in terms of the neural network model and the training algorithm. The solution quality of TSP and the CVRP up to 100 nodes are significantly improved via our framework. Compared with the best results of the state-of-the-art methods, the average optimality gap is reduced from 4.53% to 3.67% for TSP with 100 nodes and from 7.34% to 6.68% for CVRP with 100 nodes when using the greedy decoding strategy. Besides, the proposed framework can be used to solve a multi-depot CVRP case without any structural modification. Furthermore, our framework uses about 1/3 ∼ 3/4 training samples compared with other existing learning methods while achieving better results. The results performed on randomly generated instances, and the benchmark instances from TSPLIB and CVRPLIB confirm that our framework has a linear running time on the problem size (number of nodes) during training and testing phases and has a good generalization performance from random instance training to real-world instance testing. [ABSTRACT FROM AUTHOR]

• An advanced shuffled frog leaping algorithm (DSSRLFLA) is developed for model evaluation and feature selection. • A dynamic step size adjustment strategy based on historical information is designed to balance exploration and exploitation. • Specular reflection learning mechanism increases the possibility of searching for valid solutions and enhances the search ability. • An improved simulated annealing strategy is executed for each memetic to improve the efficiency of local exploitation. • DSSRLFLA can obtain higher quality optimal solutions compared to some competitive algorithms. The shuffled frog leaping algorithm is a new optimization algorithm proposed to solve the combinatorial optimization problem, which effectively combines the memetic algorithm based on a memetic algorithm and the particle swarm algorithm based on population behavior. The algorithm is widely used because it is easy to implement and requires few parameters to be adjusted. However, there are still some characteristics of this method that need to be improved because it is easy to fall into local optimization or poor search ability. To alleviate this limitation, a new version of the improved SFLA is proposed in this paper, which incorporates a dynamic step size adjustment strategy based on historical information, a specular reflection learning mechanism, and a simulated annealing mechanism based on chaotic mapping and levy flight. Firstly, the dynamic step size adjustment strategy based on historical information effectively helps to balance local exploration and global exploitation and alleviates the problem of falling into local optimum. Second, the specular reflection learning mechanism increases the possibility of searching for valid solutions in feasible domains and enhances the search ability of individuals in the population. Finally, an improved simulated annealing strategy is executed for each memetic, which improves the efficiency of local exploitation. In order to test the performance of the proposed algorithm, 31 test functions were selected from IEEE CEC2014 and 23 essential benchmark functions, and comparative experiments were carried out from the two dimensions of 30 and 100. Furthermore, a series of competing algorithms are selected, which involve nine classical standard algorithms, including PSO, BA, SSA, FA, SCA, WOA, GWO, MFO, and SFLA, as well as, six well-known improved algorithms, including LSFLA, DDSFLA, GOTLBO, ALCPSO, BLPSO, CLPSO. Furthermore, Wilcoxon signed-rank test and Friedman test are used as testing tools to illustrate the scalability of the proposed algorithm. From the analysis of the results, it can be seen that this proposed method has been effectively improved concerning stability and the quality of the optimal solution obtained from the search, both in low and high dimensions, and the ability to jump out of the local optimum has also been enhanced. In addition, to prove that this method has a reliable performance in discrete problems and continuous problems, DSSRLFLA is mapped into a discrete space, and 24 UCI data sets are also selected to evaluate the performance of the new feature selection method. The experimental results illustrate that this improved method can obtain fewer features and higher classification accuracy than some well-known feature selection methods. [ABSTRACT FROM AUTHOR]

Combinatorial optimization problems have been widely used in various fields. And many types of combinatorial optimization problems can be generalized into the model of unconstrained binary quadratic programming (UBQP). Therefore, designing an effective and efficient algorithm for UBQP problems will also contribute to solving other combinatorial optimization problems. Pointer network is an end-to-end sequential decision structure and combines with deep learning technology. With the utilization of the structural characteristics of combinatorial optimization problems and the ability to extract the rule behind the data by deep learning, pointer network has been successfully applied to solve several classical combinatorial optimization problems. In this paper, a pointer network based algorithm is designed to solve UBQP problems. The network model is trained by supervised learning (SL) and deep reinforcement learning (DRL) respectively. Trained pointer network models are evaluated by self-generated benchmark dataset and ORLIB dataset respectively. Experimental results show that pointer network model trained by SL has strong learning ability to specific distributed dataset. Pointer network model trained by DRL can learn more general distribution data characteristics. In other words, it can quickly solve problems with great generalization ability. As a result, the framework proposed in this paper for UBQP has great potential to solve large scale combinatorial optimization problems. [ABSTRACT FROM AUTHOR]

Combinatorial optimization problems (COPs) play an important role in both industrial production and theoretical research. As a classical class of optimization algorithms, evolutionary algorithms have shown good performance in COPs. However, as the size of the problem increases, it becomes difficult for evolutionary algorithms to produce satisfactory solutions within the specified time. In recent years, neural network algorithms have been applied to solve COPs and have shown superior effects. Inspired by this, this paper proposes a new algorithm based on graph structure for solving COPs. Through relaxation, COPs are transformed into their corresponding continuous optimization problems. The output state of our algorithm is converted to a discrete solution by approximation method. Compared with previous algorithms, our algorithm can not only solve COPs with general form such as binary quadratic programming (BQP), but also solve COPs with special form, which greatly expands the application scope of neural network algorithms. In addition, our algorithm can be embedded into other problem frameworks such as solving multi-objective combinatorial optimization problem (MOCOP) through scalarization method. A new penalty function is added to the objective function, which enables our algorithm to overcome the inconsistency between the optimal solution of the original problem and the corresponding continuous problem, so that our algorithm can perform well on various problems. Several numerical simulations are used to illustrate the effectiveness of our algorithm in solving COPs. Experimental results show that our algorithm can effectively solve COPs with general form and has a shorter solving time compared to traditional evolutionary algorithms. [ABSTRACT FROM AUTHOR]

Quantization methods for convolutional neural network models can be broadly categorized into post-training quantization (PTQ) and quantization aware training (QAT). While PTQ offers the advantage of requiring only a small portion of the data for quantization, the resulting quantized model may not be as effective as QAT. To address this limitation, this paper proposes a novel quantization function named Attention Round. Unlike traditional quantization function that map 32 bit floating-point value w to nearby quantization levels, Attention Round allows w to be mapped to all possible quantization levels in the entire quantization space, expanding the quantization optimization space. The possibilities of mapping w to different quantization levels are inversely correlated with the distance between w and the quantization levels, regulated by a Gaussian decay function. Furthermore, to tackle the challenge of mixed precision quantization, this paper introduces a lossy coding length measure to assign quantization precision to different layers of the model, eliminating the need for solving a combinatorial optimization problem. Experimental evaluations on various models demonstrate the effectiveness of the proposed method. Notably, for ResNet18 and MobileNetV2, the PTQ approach achieves comparable quantization performance to QAT while utilizing only 1024 training data and 10 min for the quantization process. • Attention Round quantization function expands the quantization optimization space. • Mixed precision allocation method improves mixed precision quantization efficiency. • Enriched lightweight CNNs contribute to applications in resource-limited scenarios. [ABSTRACT FROM AUTHOR]

• The 0-1 knapsack problem (KP01) is one of the classic variants of knapsack problems in which the aim is to select the items with the total profit to be in the knapsack. • In this paper, a comprehensive survey on the recent advances in 0-1 knapsack problem is presented. • Literature survey reveals some interesting challenges and future research directions. 0-1 knapsack problem (KP01) is one of the classic variants of knapsack problems in which the aim is to select the items with the total profit to be in the knapsack. In contrast, the constraint of the maximum capacity of the knapsack is satisfied. KP01 has many applications in real-world complex problems such as resource distribution, portfolio optimization, etc. This study reviews the basic theory and main algorithms of 0-1 knapsack, proposes many different types of nature-inspired metaheuristic algorithms, and divides existing 0-1 knapsack problems into 6 types according to the different coding methods of the algorithms, for a comprehensive overview of it. In this paper, a comprehensive survey on the recent advances in 0-1 knapsack problem is presented. Literature survey reveals some interesting challenges and future research directions. [ABSTRACT FROM AUTHOR]

How to identify a small fraction of users with the best capacity to influence other users is central to the research of social network analysis. This problem is termed influence maximization (IM) and is known for its extensive applications. IM can be formulated as a combinatorial optimization problem, which has been shown to be NP -hard under some diffusion models. Therefore, seeking for high-accuracy IM algorithms with acceptable running time has attracted much attention in literature. However, most of existing IM algorithms only adopt a uniform mechanism in the whole solution search process, which lacks of flexible response when the algorithms trap in local optimum. This paper proposes a phased evaluation-enhanced (PHEE) approach for IM, which utilizes two distinct strategies to search the optimal solutions: The first one is a random range division-based evolutionary algorithm for improving solution quality; the second is a fast convergence strategy for searching an optimal solution. Two PHEE-based algorithms, MDD-PHEE and GCI-PHEE, are generated and are evaluated on 10 real-world social networks of different sizes and types. Experimental results demonstrate the effectiveness of PHEE; in particular, MDD-PHEE obtains the best influence spread on all networks compared with the state-of-the-art algorithms, and has a better performance than the time-consuming algorithm CELF on four datasets. [ABSTRACT FROM AUTHOR]

Short term electric load forecasting, as an important tool in the electricity market, plays a critical role in the management of electric systems. Proposing an accuracy and optimization method is not only a challenging task but also an indispensable part of the energy system. More and more accurate forecasting methods are needed by different people in different areas. This paper proposes a novel short-term electric load forecasting method EMD-Mixed-ELM which based on empirical mode decomposition (EMD) and extreme learning machine (ELM). EMD-Mixed-ELM first uses the empirical mode decomposition to decompose the load series for capturing the complicated features of the electric load and de-noising the data. Considering that the performance of extreme learning machine (ELM) is greatly influenced by the choice of kernel, the mixed kernel method is proposed for ELM. The mixed kernel combines the RBF kernel and the UKF kernel. The forecasting results of the EMD-Mixed-ELM are proved to be better than all the other three methods (RBF-ELM, UKF-ELM and Mixed-ELM) and other existing methods (MFES, ESPLSSVM and Combined method). To verify the forecasting ability of the EMD-Mixed-ELM, half-hourly electric load data from the state of New South Wales, Victoria and Queensland in Australia are used in this paper as a case study. The experimental results clearly indicate that for this three datasets, the forecasting accuracy of the proposed method is superior to other methods. [ABSTRACT FROM AUTHOR]

Entity alignment (EA) aims to discover unique equivalent entity pairs with the same meaning across different knowledge graphs (KG), which is a crucial step for expanding the scale of KGs. Existing EA methods commonly leverage graph neural networks (GNNs) to align entities. However, these methods inherit the complex structure of GNNs, which results in lower efficiency. Meanwhile, most EA methods are either limited in their performance due to insufficient utilization of available information or require extensive manual preprocessing to obtain additional information. Furthermore, seed alignment acquisition is challenging for most EA methods that rely on supervised learning. To address these challenges, this paper proposes a simple and effective unsupervised EA model named COEA. COEA leverages the entity name information to obtain reliable supplementary information for EA and enhances performance by combining text features captured by entity names with structural features of the KG. Importantly, COEA inherits the advantages of GNN while reducing redundancy. It only uses the way of aggregating neighbor features in graph convolutional network (GCN), and transforms the EA problems into combination optimization problems. Sufficient experimental of COEA on five datasets have validated the exceptional performance and generalization capabilities of the framework. COEA achieved the best performance in all performance indicators. Notably, the framework enables the rapid implementation of entity alignment with minimal computational delays. [ABSTRACT FROM AUTHOR]

Abstract Random single-hidden layer feedforward neural network (RSLFN) is currently a popular learning algorithm proposed for improving traditional gradient-based model due to its fast learning speed and acceptable performance. For RSLFN, the input weights and/or other parameters are randomly initialized, and the other ones are iteratively or non-iteratively trained. However, the performance of RSLFN is sensitive to the number of hidden neurons and randomly initialized parameters. Numerous methods have been successfully employed to improve the RSLFN from various perspectives. Because of their favourable search ability, metaheuristic optimization approaches gradually attract more and more attentions. Metaheuristic algorithms usually formulate the random parameters of RSLFN into an optimization model, and then provide a near-optimum solution which could be converted into RSLFN with better generalization performance. The hybrid method for optimizing RSLFN therefore shows considerable potential in intelligent computing and artificial intelligence. However, there is no comprehensive survey on RSLFN with metaheuristic in the research area, which ultimately leads to lost opportunities for an advancement. This paper firstly introduces the basic principles of RSLFN along with several metaheuristic algorithms. Secondly, it provides a comprehensive survey of the state-of-the-art contributions in the area. Finally, current challenges are highlighted and promising research directions are also presented. [ABSTRACT FROM AUTHOR]

Abstract Over past few decades, swarm intelligent algorithms based on the intelligent behaviors of social creatures have been extensively studied and applied for all kinds of optimization areas. Teaching–learning-based optimization (TLBO) algorithm which imitates the teaching–learning process in a classroom, is one of population-based heuristic stochastic swarm intelligent algorithms. TLBO executes through similar iterative evolution processes as utilized by a standard evolutionary algorithm. Unlike traditional evolutionary algorithms and swarm intelligent algorithms, the iterative computation process of TLBO is divided into two phases and each phase executes iterative learning operation. Since its introduction by Rao and his team in 2010, TLBO has attracted more and more researchers' attention because of some of its strengths such as simple concept, without algorithm-specific parameters, rapid convergence and easy implementation yet effectiveness. In this paper we attempt to provide a brief review of the basic concepts of TLBO and a comprehensive survey of its prominent variants and its typical application, and the theoretical analysis conducted on TLBO so far. We hope that this survey can be very beneficial for the researchers engaged in the study of TLBO. [ABSTRACT FROM AUTHOR]

Abstract High-order graph matching utilizes the high-order relations to establish the correspondences between two sets of features. It has been formulated as an optimization problem, and this problem is an NP-hard combinatorial optimization problem. This paper presents a high-order graph matching method based on ant colony optimization. A problem-specific pheromone initialization method is introduced, meanwhile local and global pheromone updating rules are redefined. The proposed algorithm is evaluated on synthetic datasets, CMU House dataset, and real-world datasets. Experimental results demonstrate that our algorithm achieves the promising performance compared with three state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Highlights • Genetic algorithms (GAs) are used to find optimal assignment of tasks to teams of collaborating agents, where performance of a team is based on the collaboration of its members. • We show the shortcoming of standard crossover operators to efficiently solve large instances of the problem. • We suggest extension (shuffled list) to standard crossover operators and introduce two more operators (team-based and team-based shuffled list), which all are capable to efficiently solve the problem. • The position-based crossover operator with a shuffled list is singled out to have the best performance. • Performance of the GAs is further enhanced by using chaotic sequences and the results are significant as compared with PSO and DE. Abstract The problem of optimally assigning agents (resources) to a given set of tasks is known as the assignment problem (AP). The classical AP and many of its variations have been extensively discussed in the literature. In this paper, we examine a specific class of the problem, in which each task is assigned to a group of collaborating agents. APs in this class cannot be solved using the Hungarian or other known polynomial time algorithms. We employ the genetic algorithm (GA) to solve the problem. However, we show that if the size of the problem is large, then standard crossover operators cannot efficiently find near-optimal solutions within a reasonable time. In general, the efficiency of the GA depends on the choice of genetic operators (selection, crossover, and mutation) and the associated parameters. In order to design an efficient GA for determining the near-optimal assignment of tasks to collaborative agents, we focus on the construction of crossover operators. We analyze why a naive implementation with standard crossover operators is not capable of sufficiently solving the problem. Furthermore, we suggest modifications to these operators by adding a shuffled list and introduce two new operators (team-based and team-based shuffled list). We demonstrate that the modified and new operators with shuffled lists perform significantly better than all operators without shuffled lists and solve the presented AP more efficiently. The performance of the GA can be further enhanced by using chaotic sequences. Moreover, the GA is also compared with the particle swarm optimization (PSO) and differential evolution (DE) algorithms, demonstrating the superiority of the GA over these search algorithms. [ABSTRACT FROM AUTHOR]

Abstract Feed-forward neural networks are efficient at solving various types of problems. However, finding efficient training algorithms for feed-forward neural networks is challenging. The dynamic group optimisation (DGO) algorithm is a recently proposed half-swarm half-evolutionary algorithm, which exhibits a rapid convergence rate and good performance in searching and avoiding local optima. In this paper, we propose a new hybrid algorithm, FNNDGO that integrates the DGO algorithm into a feed-forward neural network. DGO plays an optimisation role in training the neural network, by tuning parameters to their optimal values and configuring the structure of feed-forward neural networks. The performance of the proposed algorithm was determined by comparing its performance with those of other training methods in solving two types of problems. The experimental results show that our proposed algorithm exhibits promising performance for solving real-world problems. [ABSTRACT FROM AUTHOR]

Heuristic and deterministic optimization methods are extensively applied for the training of artificial neural networks. Both of these methods have their own advantages and disadvantages. Heuristic stochastic optimization methods like genetic algorithm perform global search, but they suffer from the problem of slow convergence rate near global optimum. On the other hand deterministic methods like gradient descent exhibit a fast convergence rate around global optimum but may get stuck in a local optimum. Motivated by these problems, a hybrid learning algorithm combining genetic algorithm (GA) with gradient descent (GD), called HGAGD, is proposed in this paper. The new algorithm combines the global exploration ability of GA with the accurate local exploitation ability of GD to achieve a faster convergence and also a better accuracy of final solution. The HGAGD is then employed as a new training method to optimize the parameters of a quantum-inspired neural network (QINN) for two different applications. Firstly, two benchmark functions are chosen to demonstrate the potential of the proposed QINN with the HGAGD algorithm in dealing with function approximation problems. Next, the performance of the proposed method in forecasting Mackey–Glass time series and Lorenz attractor is studied. The results of these studies show the superiority of the introduced approach over other published approaches. [ABSTRACT FROM AUTHOR]

Researchers who investigate in any area related to computational algorithms (both defining new algorithms or improving existing ones) usually find large difficulties to test their work. Comparisons among different researches in this field are often a hard task, due to the ambiguity or lack of detail in the presentation of the work and its results. On many occasions, the replication of the work conducted by other researchers is required, which leads to a waste of time and a delay in the research advances. The authors of this study propose a procedure to introduce new techniques and their results in the field of routing problems. In this paper, this procedure is detailed, and a set of good practices to follow are deeply described. It is noteworthy that this procedure can be applied to any combinatorial optimization problem. Anyway, the literature of this study is focused on routing problems. This field has been chosen because of its importance in real world, and its relevance in the actual literature. [ABSTRACT FROM AUTHOR]

Cosegmentation is the task of simultaneously segmenting multiple images that contain common or similar foreground objects. The assumption that common objects appear in multiple images provides a weak form of supervised prior information, thus cosegmentation usually performs better than unsupervised segmentation methods. Recently, a multi-task learning based cosegmentation method was proposed, and it can simultaneously segment more than two images and easily add different types of prior. However, it has the shortcoming of information loss in the initialization of the multi-task classification model. To ameliorate this problem, in this paper, we propose a novel multi-task ranking SVM model which incorporates multi-task learning and learning to rank into a unified framework. The proposed model is trained using the relative order information between the cosaliency score of pixel pairs. In addition, an optimization algorithm is proposed to optimize the multi-task ranking SVM model based on the alternative direction method of multipliers (ADMM), which ensures that the proposed method is faster than most of state-of-the-art cosegmentation approaches. Finally, the proposed method is evaluated on two widely used benchmark datasets, i.e. CMU iCoseg and MSRC. The experiment results show that the proposed approach is effective and performs better than most of the state-of-the-art works. [ABSTRACT FROM AUTHOR]

The multiobjective unconstrained binary quadratic programming (mUBQP) is a combinatorial optimization problem which is able to represent several multiobjective optimization problems (MOPs). The problem can be characterized by the number of variables, the number of objectives and the objective correlation strength. Multiobjective evolutionary algorithms (MOEAs) are known as an efficient technique for solving MOPs. Moreover, several recent studies have shown the effectiveness of the MOEA/D framework applied to different MOPs. Previously, we have presented a preliminary study on an algorithm based on MOEA/D framework and the bio-inspired metaheuristic called binary ant colony optimization (BACO). The metaheuristic uses a positive feedback mechanism according to the best solutions found so far to update a probabilistic model which maintains the learned information. This paper presents the improved MOEA/D-BACO framework for solving the mUBQP. The components (i) mutation-like effect, and (ii) diversity preserving method are incorporated into the framework to enhance its search ability avoiding the premature convergence of the model and consequently maintaining a more diverse population of solutions. Experimental studies were conducted on a set of mUBQP instances. The results have shown that the proposed MOEA/D-BACO has outperformed MOEA/D, which uses genetic operators, in most of the test instances. Moreover, the algorithm has produced competitive results in comparison to the best approximated Pareto fronts from the literature. [ABSTRACT FROM AUTHOR]

In this paper, quasi-synchronization is investigated for heterogeneous complex networks. All nodes of complex networks are nonlinear and have different dynamics. When topology is switching and disconnected, some sufficient criteria that guaranteeing the quasi-synchronization of heterogeneous complex networks are obtained based on the theory of sequential connectivity and joint connectivity. Even if the complex networks are heterogeneous, without any external control, and the topology is not connected at any moment, the errors between any pair of nodes will not exceed the upper bound, which is derived by calculating the maximal distance between any pair of nodes. Finally, some numerical simulations are provided to demonstrate the efficiency of the theoretical results. [ABSTRACT FROM AUTHOR]

Clustering ensemble has emerged as an important tool for data analysis, by which a more robust and accurate consensus clustering can be generated. On forming the ensembles, empirical studies have suggested that better ensembles can be obtained by simultaneously considering the quality of the ensembles and the diversity among ensemble members. However, little research efforts have been paid to incorporate prior background knowledge. In this paper, we first provide a theoretical analysis on the effect of the diversity and quality of the ensemble members. We then propose a unified framework to solve constraint-based clustering ensemble selection problem, where some instance level must-link and cannot-link constraints are given as prior knowledge or background information. We formalize this problem as a combinatorial optimization problem in terms of the consistency under the constraints, the diversity among ensemble members, and the overall quality of ensembles. Our proposed framework brings together two distinct yet interrelated themes from clustering: ensemble clustering and semi-supervised clustering. We study different techniques for searching high-quality solutions. Experiments on benchmark datasets demonstrate the effectiveness of our framework. [ABSTRACT FROM AUTHOR]

Image reconstruction by l 0 minimization is an NP-hard problem that requires exhaustively listing all possibilities of the original signal with a very high computational complexity, which is difficult to be achieved by traditional algorithms. Although greedy algorithm aims at solving l 0 minimization, it is more likely to fall into a suboptimal solution. In this paper, we propose a multi-variable intelligent matching pursuit algorithm (MIMP), which can solve l 0 minimization problem essentially by taking the advantage of the intelligent optimization algorithm in solving combinatorial optimization problems and searching for the global optimal solution to improve the performance of image reconstruction. The updating mechanism of MIMP is designed by introducing the matching strategies of greedy algorithm to accelerate the reconstruction speed. Also, the multi-variable scheme is utilized to sample images and then the joint reconstruction is implemented to the measurements, which can not only improve the reconstruction accuracy but also reduce the computational complexity. Moreover, the edge saliency can be obtained as the prior knowledge to guide the compressive sensing reconstruction, which contributes a lot to reduce the computational complexity and accelerate the reconstruction speed. As the sparsity level of image is hard to be estimated, a new optimization function is proposed to solve this problem without knowing the sparsity level as a prior. Compared with other state-of-the-art algorithms, the proposed method MIMP can achieve a better reconstruction accuracy by solving l 0 minimization essentially with intelligent optimization algorithms. Also, MIMP has a reasonable relatively faster reconstruction speed by introducing the matching strategies of greedy algorithm and using the edge saliency as a prior knowledge. Numerical experiments on several images demonstrate that the proposed method MIMP significantly outperforms the state-of-the-art algorithms and the structure based reconstruction algorithms in PSNR, SSIM and visual quality. [ABSTRACT FROM AUTHOR]

As a recently developed optimization method inspired by far-from-equilibrium dynamics of self-organized criticality, extremal optimization (EO) has been successfully applied to a variety of combinatorial optimization problems while its applications in continuous optimization problems are relatively rare. Additionally, there are only few studies concerning the effects of mutation operation on EO algorithms although mutation operation plays a crucial role in controlling the optimization dynamics and consequently affecting the performance of EO-based algorithms. This paper proposes a novel real-coded population-based EO algorithm with polynomial mutation (RPEO-PLM) for continuous optimization problems. The basic idea behind RPEO-PLM is the population-based iterated optimization consisting of generation of a real-coded random initial population, evaluation of individual and population fitness, generation of a new population based on polynomial mutation, and updating the population by accepting the new population unconditionally. One of the most attractive advantages is its relative simplicity compared with other popular evolutionary algorithms due to its fewer adjustable parameters needing to be tuned and only selection and mutation operations. Furthermore, the experimental results on a large number of benchmark functions with the different dimensions by using non-parametric statistical tests including Friedman and Quade tests have shown that the proposed RPEO-PLM algorithm outperforms other popular population-based evolutionary algorithms, e.g., real-coded genetic algorithm (RCGA) with adaptive directed mutation (RCGA-ADM), RCGA with polynomial mutation (RCGA-PLM), intelligent evolutionary algorithm (IEA), a hybrid particle swarm optimization and EO algorithm (PSO–EO), the original population-based EO (PEO), and an improved RPEO algorithm with random mutation (IRPEO-RM) in terms of accuracy. [ABSTRACT FROM AUTHOR]

In this paper an efficient active set algorithm is presented for fast training of Optimization Extreme Learning Machines (OELMs). This algorithm suggests the use of an efficient identification technique of active set and the value reassignment technique for quadratic programming problem. With these strategies, this algorithm is able to drop many constraints from the active set at each iteration, and it can converge to the optimal solution with less iterations. The global convergence properties of the algorithm as well as its theoretical properties are analyzed. The effectiveness of the algorithm is demonstrated via benchmark datasets from many sources. Experiment results indicate that the quadratic programming problem which keeps the number of constraints in the active set as small as possible is computationally most efficient. [ABSTRACT FROM AUTHOR]

Web-scale image understanding is drawing more and more attention from the computer vision and multimedia domain. To solve the key problem of visual polysemia and concept polymorphism in the image understanding, this paper proposes a semantic dictionary to describe the images on the level of semantic. The semantic dictionary characterizes the probability distribution between visual appearances and semantic concepts, and the learning procedure of semantic dictionary is formulated into a minimization optimization problem. Mixed-norm regularization is adopted to solve the above optimization for learning the concept membership distribution of visual appearance. Furthermore, to improve the generalization ability of the semantic description, we propose the semantic expansion technology, where a concept transferring matrix is learnt to quantize the implicit relevancy among the concepts. Finally, the distributed framework on the basis of the semantic dictionary is constructed to speed up the large scale image understanding. The semantic dictionary is validated in the tasks of large scale semantic image search and image annotation. [ABSTRACT FROM AUTHOR]

Currently, most of the multiobjective evolutionary algorithms (MOEAs) directly adopt the reproduction operators designed for the single-objective optimization. Since these operators do not consider the characteristics of multiobjective optimization problems (MOPs), they cannot always perform well in the MOEAs. Inspired by this case, this paper presents a self-organizing reproduction mechanism based on the regularity property of MOPs, and proposes a self-organizing multiobjective evolutionary algorithm based on decomposition with neighborhood ensemble. In the new reproduction, a self-organizing map approach is firstly employed to discover the population distribution structure, and to build a mating pool for each solution. Thereafter, reproductions are only allowed among the solutions within the same mating pools. In order to establish the mating pools, an ensemble of multiple neuron neighborhood sizes is also introduced. The probability of choosing different neighborhood sizes is updated based on their performance on producing new solutions over the last certain generations. Comprehensive experiments denote that the proposed algorithm is efficient and competitive. The contributions of the new reproduction mechanism and neighborhood ensemble are also experimentally validated. [ABSTRACT FROM AUTHOR]

The studies on the controllability of complex networks, arising from natural, social, and man-made-engineered systems, have attracted great attention from both network community and control community. With the fixed number of control nodes, it is of great significance in both academic research and industrial applications to design the optimal control configurations of a directed network to make its controllability index (i.e., the maximum dimension of the controllable subnetwork) maximum. In this paper, a design strategy for the optimal control configurations is proposed, and the results of the experiments conducted on multiple real and model networks show the effectiveness of this design strategy compared to other commonly-used design strategies. Moreover, we have two interesting findings in the macroscopic level and the microscopic level, respectively: (1) the dense and homogeneous networks have larger controllability indexes than the sparse and heterogeneous ones; (2) the average in-degree of the controlled state nodes in the optimal control configurations is far less than that of the network, which provides us a heuristic way to design a sub-optimal control configuration. These findings are helpful to further our understanding on the interplay between the network structure and its control. [ABSTRACT FROM AUTHOR]

Single-hidden layer feedforward neural networks with randomly fixed hidden neurons (RHN-SLFNs) have been shown, both theoretically and experimentally, to be fast and accurate. Besides, it is well known that deep architectures can find higher-level representations, thus can potentially capture relevant higher-level abstractions. But most of current deep learning methods require a long time to solve a non-convex optimization problem. In this paper, we propose a stacked deep neural network, St-URHN-SLFNs, via unsupervised RHN-SLFNs according to stacked generalization philosophy to deal with unsupervised problems. Empirical study on a wide range of data sets demonstrates that the proposed algorithm outperforms the state-of-the-art unsupervised algorithms in terms of accuracy. On the computational effectiveness, the proposed algorithm runs much faster than other deep learning methods, i.e. deep autoencoder (DA) and stacked autoencoder (SAE), and little slower than other methods. [ABSTRACT FROM AUTHOR]

This paper presents a framework for modeling and optimizing the glutamic acid fermentation process using computational intelligence techniques. Considering the special characteristics of such an industrial process, we propose a two-phase optimization strategy to maximize the conversion rate and product concentration of the glutamic acid. Neural network ensembles and an improved Differential Evolutionary Algorithm (DEA) with a non-inferior sorting scheme and niche technology are employed for problem solving. This work provides an approach for design of a model-free optimal control system for the fed-batch fermentation process. Experimental results are promising and demonstrate the applicability of the proposed modeling and optimization techniques for real world applications. [ABSTRACT FROM AUTHOR]

To estimate the dissolved oxygen (DO) concentration in the wastewater treatment process more accurately, a group genetic case-based reasoning (GGCBR) soft-sensing method of DO concentration is proposed by the integrated use of genetic algorithms (GA) and group decision making (GDM) in this paper. First, an attribute weight iterative optimization using GA is discussed in the case retrieval phase. Then, a case reuse method is carried out by using the GDM theory. Finally, the historical data of DO concentration in a wastewater treatment process are used to carry out a comparison experiment by 10-fold cross validation. The results show that this proposed GGCBR is superior to other methods and significantly reduces the fitting error of DO concentration. [ABSTRACT FROM AUTHOR]

Dynamic (also known as instance-based) ensemble pruning selects a (potentially) different subset of models from an ensemble during prediction based on the given unknown instance with the goal of maximizing prediction accuracy. This paper models dynamic ensemble pruning as a multi-label classification task, by considering the members of the ensemble as labels. Multi-label training examples are constructed by evaluating whether ensemble members are accurate or not on the original training set via cross-validation. We show that classification accuracy is maximized when learning algorithms that optimize example-based precision are used in the multi-label classification task. Results comparing the proposed framework against state-of-the-art dynamic ensemble pruning approaches in a variety of datasets using a heterogeneous ensemble of 200 classifiers show that it leads to significantly improved accuracy. [ABSTRACT FROM AUTHOR]

Clustering ensemble has become a very popular technique in the past few years due to its potentialities for improving the clustering results. Roughly speaking it consists in the combination of different partitions of the same set of objects in order to obtain a consensus one. A common way of defining the consensus partition is as the solution of the median partition problem. In this way, the consensus partition is defined as the solution of a complex optimization problem. In this paper, we study possible prunes of the search space for this optimization problem. Particularly, we introduce a new prune that allows a dramatic reduction of the search space. We also provide a characterization of the family of dissimilarity measures that can be used to take advantage of this prune and we present two measures that fit into this family. We carry out an experimental study on synthetic data by comparing, under different circumstances, the size of the original search space and the size after the proposed prunes. Outstanding reductions are obtained, which can be beneficial for the development of clustering ensemble algorithms. We also compare, on real data, the behavior of a simulated annealing-based ensemble algorithm in the original partition space and in the two proposed pruned spaces. In all cases, the proposed prunes allow the algorithm to find solutions closer to the theoretical optimum. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) has been an important research topic over the last decade due to its high efficiency, easy-implementation, unification of classification and regression, and unification of binary and multi-class learning tasks. Though integrating these advantages, existing ELM algorithms pay little attention to optimizing the choice of kernels, which is indeed crucial to the performance of ELM in applications. More importantly, there is the lack of a general framework for ELM to integrate multiple heterogeneous data sources for classification. In this paper, we propose a general learning framework, termed multiple kernel extreme learning machines (MK-ELM), to address the above two issues. In the proposed MK-ELM, the optimal kernel combination weights and the structural parameters of ELM are jointly optimized. Following recent research on support vector machine (SVM) based MKL algorithms, we first design a sparse MK-ELM algorithm by imposing an ℓ 1 -norm constraint on the kernel combination weights, and then extend it to a non-sparse scenario by substituting the ℓ 1 -norm constraint with an ℓ p -norm ( p > 1 ) constraint. After that, a radius-incorporated MK-ELM algorithm which incorporates the radius of the minimum enclosing ball (MEB) is introduced. Three efficient optimization algorithms are proposed to solve the corresponding kernel learning problems. Comprehensive experiments have been conducted on Protein, Oxford Flower17, Caltech101 and Alzheimer׳s disease data sets to evaluate the performance of the proposed algorithms in terms of classification accuracy and computational efficiency. As the experimental results indicate, our proposed algorithms can achieve comparable or even better classification performance than state-of-the-art MKL algorithms, while incurring much less computational cost. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) as an emerging technology has achieved exceptional performance in large-scale settings, and is well suited to binary and multi-class classification, as well as regression tasks. However, existing ELM and its variants predominantly employ single hidden layer feedforward networks, leaving the popular and potentially powerful stacked generalization principle unexploited for seeking predictive deep representations of input data. Deep architectures can find higher-level representations, thus can potentially capture relevant higher-level abstractions. But most of current deep learning methods require solving a difficult and non-convex optimization problem. In this paper, we propose a stacked model, DrELM, to learn deep representations via extreme learning machine according to stacked generalization philosophy. The proposed model utilizes ELM as a base building block and incorporates random shift and kernelization as stacking elements. Specifically, in each layer, DrELM integrates a random projection of the predictions obtained by ELM into the original feature, and then applies kernel functions to generate the resultant feature. To verify the classification and regression performance of DrELM, we conduct the experiments on both synthetic and real-world data sets. The experimental results show that DrELM outperforms ELM and kernel ELMs, which appear to demonstrate that DrELM could yield predictive features that are suitable for prediction tasks. The performances of the deep models (i.e. Stacked Auto-encoder) are comparable. However, due to the utilization of ELM, DrELM is easier to learn and faster in testing. [ABSTRACT FROM AUTHOR]

In the steady-state visual evoked potentials (SSVEP)-based brain–computer interfaces (BCIs), the time-window length plays an important role as it controls how much data is used each time in signal processing and classification for target detection. Normally, the larger the time-window length, the higher the detection accuracy and the longer the detection time, while the overall performance of a BCI system involves a trade-off between the detection accuracy and the detection time. An optimal time-window length is thus preferred but unfortunately such a value varies considerably among different subjects. This paper proposes an adaptive method to optimize the time-window length based on the subject׳s online performance. More specifically, a feedback from the subject using two commands, “Undo” and “Delete”, is designed to assess the performance in real time. Based on the assessment, the adaptive mechanism decides whether to change or maintain the time-window length. The proposed system was tested on 7 subjects, with on average an accuracy of 98.42% and an information transfer rate (ITR) of 70.71 bits/min, representing an ITR improvement of 19.36% compared to its non-adaptive counterpart. [ABSTRACT FROM AUTHOR]

Compared with conventional weighted voting methods, class-specific soft voting (CSSV) system has several advantages. On one hand, it not only deals with the soft class probability outputs but also refines the weights from classifiers to classes. On the other hand, the class-specific weights can be used to improve the combinative performance without increasing much computational load. This paper proposes two weight optimization based ensemble methods (CSSV-ELM and SpaCSSV-ELM) under the framework of CSSV scheme for multiple extreme learning machines (ELMs). The designed two models are in terms of accuracy and sparsity aspects, respectively. Firstly, CSSV-ELM takes advantage of the condition number of matrix, which reveals the stability of linear equation, to determine the weights of base ELM classifiers. This model can reduce the unreliability induced by randomly input parameters of a single ELM, and solve the ill-conditioned problem caused by linear system structure of ELM simultaneously. Secondly, sparse ensemble methods can lower memory requirement and speed up the classification process, but only for classifier-specific weight level. Therefore, a SpaCSSV-ELM method is proposed by transforming the weight optimization problem to a sparse coding problem, which uses the sparse representation technique for maintaining classification performance with less nonzero weight coefficients. Experiments are carried out on twenty UCI data sets and Finance event series data and the experimental results show the superior performance of the CSSV based ELM algorithms by comparing with the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Model-based Genetic Algorithms (GAs), as the Linkage Tree Genetic Algorithm (LTGA) and most Estimation of Distribution Algorithms (EDAs), assume a reductionist perspective when solving optimization problems. They use machine-learning techniques to discover problem׳s substructures that might be useful to generate new solutions. This idea was grounded on Simon׳s near-decomposability principle and Holland׳s Building Block (BB)-hypothesis, and have enabled the development of effective algorithms in some contexts. Although near-decomposability is commonly seen in nature, we cannot assume the same occurs for optimization problems. Therefore, the existence of problems where these algorithms are not effective is also focus of research. Recent studies have argued that Multidimensional Knapsack Problems (MKPs) are examples of such cases. This paper extends these studies with an extensive comparison of various LTGA variants for the MKP. Using a well-known GA as reference, we analyzed the difficulties faced by the LTGA and explained why its linkage-tree model is not of much help to solve the problem. The results have shown that the LTGA was not able to outperform the GA and performed very similarly to a LTGA using random linkage-models. Further analysis of the linkage-trees, grounded on the knapsack-core concept, enabled interesting conclusions about the reason that linkage-learning did not provide useful information to solve MKPs in the settings used for the experiments. [ABSTRACT FROM AUTHOR]