Search

For NP-hard combinatorial optimization problems, it is usually challenging to find high-quality solutions in polynomial time. Designing either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods have provided new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input, masks, and decoder context vectors. The proposed framework aims to improve the models in literacy in terms of the neural network model and the training algorithm. The solution quality of TSP and the CVRP up to 100 nodes are significantly improved via our framework. Compared with the best results of the state-of-the-art methods, the average optimality gap is reduced from 4.53% to 3.67% for TSP with 100 nodes and from 7.34% to 6.68% for CVRP with 100 nodes when using the greedy decoding strategy. Besides, the proposed framework can be used to solve a multi-depot CVRP case without any structural modification. Furthermore, our framework uses about 1/3 ∼ 3/4 training samples compared with other existing learning methods while achieving better results. The results performed on randomly generated instances, and the benchmark instances from TSPLIB and CVRPLIB confirm that our framework has a linear running time on the problem size (number of nodes) during training and testing phases and has a good generalization performance from random instance training to real-world instance testing. [ABSTRACT FROM AUTHOR]

Quantization methods for convolutional neural network models can be broadly categorized into post-training quantization (PTQ) and quantization aware training (QAT). While PTQ offers the advantage of requiring only a small portion of the data for quantization, the resulting quantized model may not be as effective as QAT. To address this limitation, this paper proposes a novel quantization function named Attention Round. Unlike traditional quantization function that map 32 bit floating-point value w to nearby quantization levels, Attention Round allows w to be mapped to all possible quantization levels in the entire quantization space, expanding the quantization optimization space. The possibilities of mapping w to different quantization levels are inversely correlated with the distance between w and the quantization levels, regulated by a Gaussian decay function. Furthermore, to tackle the challenge of mixed precision quantization, this paper introduces a lossy coding length measure to assign quantization precision to different layers of the model, eliminating the need for solving a combinatorial optimization problem. Experimental evaluations on various models demonstrate the effectiveness of the proposed method. Notably, for ResNet18 and MobileNetV2, the PTQ approach achieves comparable quantization performance to QAT while utilizing only 1024 training data and 10 min for the quantization process. • Attention Round quantization function expands the quantization optimization space. • Mixed precision allocation method improves mixed precision quantization efficiency. • Enriched lightweight CNNs contribute to applications in resource-limited scenarios. [ABSTRACT FROM AUTHOR]

Abstract Random single-hidden layer feedforward neural network (RSLFN) is currently a popular learning algorithm proposed for improving traditional gradient-based model due to its fast learning speed and acceptable performance. For RSLFN, the input weights and/or other parameters are randomly initialized, and the other ones are iteratively or non-iteratively trained. However, the performance of RSLFN is sensitive to the number of hidden neurons and randomly initialized parameters. Numerous methods have been successfully employed to improve the RSLFN from various perspectives. Because of their favourable search ability, metaheuristic optimization approaches gradually attract more and more attentions. Metaheuristic algorithms usually formulate the random parameters of RSLFN into an optimization model, and then provide a near-optimum solution which could be converted into RSLFN with better generalization performance. The hybrid method for optimizing RSLFN therefore shows considerable potential in intelligent computing and artificial intelligence. However, there is no comprehensive survey on RSLFN with metaheuristic in the research area, which ultimately leads to lost opportunities for an advancement. This paper firstly introduces the basic principles of RSLFN along with several metaheuristic algorithms. Secondly, it provides a comprehensive survey of the state-of-the-art contributions in the area. Finally, current challenges are highlighted and promising research directions are also presented. [ABSTRACT FROM AUTHOR]

Highlights • Genetic algorithms (GAs) are used to find optimal assignment of tasks to teams of collaborating agents, where performance of a team is based on the collaboration of its members. • We show the shortcoming of standard crossover operators to efficiently solve large instances of the problem. • We suggest extension (shuffled list) to standard crossover operators and introduce two more operators (team-based and team-based shuffled list), which all are capable to efficiently solve the problem. • The position-based crossover operator with a shuffled list is singled out to have the best performance. • Performance of the GAs is further enhanced by using chaotic sequences and the results are significant as compared with PSO and DE. Abstract The problem of optimally assigning agents (resources) to a given set of tasks is known as the assignment problem (AP). The classical AP and many of its variations have been extensively discussed in the literature. In this paper, we examine a specific class of the problem, in which each task is assigned to a group of collaborating agents. APs in this class cannot be solved using the Hungarian or other known polynomial time algorithms. We employ the genetic algorithm (GA) to solve the problem. However, we show that if the size of the problem is large, then standard crossover operators cannot efficiently find near-optimal solutions within a reasonable time. In general, the efficiency of the GA depends on the choice of genetic operators (selection, crossover, and mutation) and the associated parameters. In order to design an efficient GA for determining the near-optimal assignment of tasks to collaborative agents, we focus on the construction of crossover operators. We analyze why a naive implementation with standard crossover operators is not capable of sufficiently solving the problem. Furthermore, we suggest modifications to these operators by adding a shuffled list and introduce two new operators (team-based and team-based shuffled list). We demonstrate that the modified and new operators with shuffled lists perform significantly better than all operators without shuffled lists and solve the presented AP more efficiently. The performance of the GA can be further enhanced by using chaotic sequences. Moreover, the GA is also compared with the particle swarm optimization (PSO) and differential evolution (DE) algorithms, demonstrating the superiority of the GA over these search algorithms. [ABSTRACT FROM AUTHOR]

Abstract Feed-forward neural networks are efficient at solving various types of problems. However, finding efficient training algorithms for feed-forward neural networks is challenging. The dynamic group optimisation (DGO) algorithm is a recently proposed half-swarm half-evolutionary algorithm, which exhibits a rapid convergence rate and good performance in searching and avoiding local optima. In this paper, we propose a new hybrid algorithm, FNNDGO that integrates the DGO algorithm into a feed-forward neural network. DGO plays an optimisation role in training the neural network, by tuning parameters to their optimal values and configuring the structure of feed-forward neural networks. The performance of the proposed algorithm was determined by comparing its performance with those of other training methods in solving two types of problems. The experimental results show that our proposed algorithm exhibits promising performance for solving real-world problems. [ABSTRACT FROM AUTHOR]

The studies on the controllability of complex networks, arising from natural, social, and man-made-engineered systems, have attracted great attention from both network community and control community. With the fixed number of control nodes, it is of great significance in both academic research and industrial applications to design the optimal control configurations of a directed network to make its controllability index (i.e., the maximum dimension of the controllable subnetwork) maximum. In this paper, a design strategy for the optimal control configurations is proposed, and the results of the experiments conducted on multiple real and model networks show the effectiveness of this design strategy compared to other commonly-used design strategies. Moreover, we have two interesting findings in the macroscopic level and the microscopic level, respectively: (1) the dense and homogeneous networks have larger controllability indexes than the sparse and heterogeneous ones; (2) the average in-degree of the controlled state nodes in the optimal control configurations is far less than that of the network, which provides us a heuristic way to design a sub-optimal control configuration. These findings are helpful to further our understanding on the interplay between the network structure and its control. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes a one-layer recurrent neural network for solving nonlinear nonsmooth pseudoconvex optimization problem subject to linear equality constraints. We first prove that the equilibrium point set of the proposed neural network is equivalent to the optimal solution of the original optimization problem, even though the objective function is pseudoconvex. Then, it is proved that the state of the proposed neural network is stable in the sense of Lyapunov, and globally convergent to an exact optimal solution of the original optimization. In the end, some illustrative examples are given to demonstrate the effectiveness of the proposed neural network. [Copyright &y& Elsevier]

Abstract: Recently, some continuous-time recurrent neural networks have been proposed for associative memories based on optimizing linear or quadratic programming problems. In this paper, a simple and efficient neural network with a single recurrent unit is proposed for realizing associative memories. Compared with the existing neural networks for associative memories, the main advantage of the proposed model is that it has only one recurrent unit, which lowers the model complexity by the greatest extent. In the proposed neural network, each prototype pattern is stored in the connection weights between the input and hidden layers. In addition, the advanced performance of the proposed network is demonstrated by means of simulations of three numerical examples. [Copyright &y& Elsevier]

Abstract: This paper is concerned with the stability analysis of neutral type neural networks with discrete and distributed delays. Some improved delay-dependent stability results are established by using a delay partitioning approach for the networks. By employing a new type of Lyapunov–Krasovskii functionals, new delay-dependent stability criteria are derived. All the criteria are expressed in terms of linear matrix inequalities (LMIs), which can be solved efficiently by using standard convex optimization algorithms. Finally, numerical examples are given to illustrate the less conservatism of the proposed method. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a novel weight initialization method for the random neural network. The method relies on approximating the signal-flow equations of the network to obtain a linear system of equations with nonnegativity constraints. For the solution of the formulated linear nonnegative least squares problem we have developed a projected gradient algorithm. It is shown that supervised learning with the developed initialization method has better performance in terms of both solution quality and execution time than learning with random initialization when applied to a combinatorial optimization emergency response problem. [Copyright &y& Elsevier]

Abstract: This paper presents a method to solve disassembly sequence planning problem. Firstly, disassembly feasibility information graph (DFIG) is presented to describe the product disassembly sequence and operation information. Then, disassembly sequence planning problem is mapped onto the DFIG as an optimal path-searching problem. Based on this idea, a genetic algorithm is provided to find out feasible and optimal disassembly solutions efficiently. Finally, a case study, which is used to illustrate the performance of proposed method, is presented. [Copyright &y& Elsevier]

Abstract: Gene prediction is still an important step to annotate genomes. In this paper, we proposed a novel method for recognizing gene in genomes. The method combines three famous gene-finding programs. After calculating the accuracy parameters, the equitable weight for each parameter is calculated using genetic algorithm. Then the integrative evaluation is performed. The integrative evaluation is employed to instruct the training of an artificial neural network. The simulation results show that the proposed method integrates advantages of three programs and the accuracy has an obvious improvement, which indicate that the proposed method has a powerful capability for gene prediction. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a massively parallel architecture for hardware implementation of genetic algorithms. This design is quite innovative as it provides a viable solution to the fitness computation problem, which depends heavily on the problem-specific knowledge. The proposed architecture is completely independent of such specifics. It implements the fitness computation using a neural network. The hardware implementation of the used neural network is stochastic and thus minimise the required hardware area without much increase in response time. Last but not least, we demonstrate the characteristics of the proposed hardware and compare it to existing ones. [Copyright &y& Elsevier]

Abstract: The classification ability of a single-layer perceptron could be improved by considering some enhanced features. In particular, this form of neural networks is called a functional-link net. In the output neuron''s activation function, such as the sigmoid function, an inner product of a connection weight vector with an input vector is computed. However, since the input features are not independent of each other for the enhanced pattern, an assumption of the additivity is not reasonable. This paper employs a non-additive technique, namely the fuzzy integral, to aggregate performance values for an input pattern by interpreting each of the connection weights as a fuzzy measure of the corresponding feature. A learning algorithm with the genetic algorithm is then designed to automatically find connection weights. The sample data for bankruptcy analysis obtained from Moody''s Industrial Manuals is considered to examine the classification ability of the proposed method. The results demonstrate that the proposed method performs well in comparison with traditional functional-link net and multivariate techniques. [Copyright &y& Elsevier]

Abstract: Recently, a new learning algorithm for the feedforward neural network named the extreme learning machine (ELM) which can give better performance than traditional tuning-based learning methods for feedforward neural networks in terms of generalization and learning speed has been proposed by Huang et al. In this paper, we first extend the ELM algorithm from the real domain to the complex domain, and then apply the fully complex extreme learning machine (C-ELM) for nonlinear channel equalization applications. The simulation results show that the ELM equalizer significantly outperforms other neural network equalizers such as the complex minimal resource allocation network (CMRAN), complex radial basis function (CRBF) network and complex backpropagation (CBP) equalizers. C-ELM achieves much lower symbol error rate (SER) and has faster learning speed. [Copyright &y& Elsevier]

Abstract: This paper presents a modified particle swarm optimization (PSO) with simulated annealing (SA) technique. An improved PSO-based artificial neural network (ANN) is developed. The results show that the proposed SAPSO-based ANN has a better ability to escape from a local optimum and is more effective than the conventional PSO-based ANN. [Copyright &y& Elsevier]

There have been several neural network approaches to the tasks of solving combinatorial optimization problems. In this paper, a new type of neural networks—the WTA-type networks, which incorporates the Winner-Take-All mechanism into the automata networks, is proposed. Five specifications of the WTA-type networks (N1–N5) are presented. The theoretical foundations of the networks are developed from the standpoint of taking them as combinatorial optimization solvers. We also investigate the two key issues, reliability and efficiency, related to the application of the networks. The proposed networks and the established theories are applied to a set of combinatorial optimization benchmark problems—traveling salesman problems. The simulation results demonstrate that the proposed WTA-type networks are more effective or comparable with the Hopfield networks, the Boltzmann machine and the self-organizing feature map network. [Copyright &y& Elsevier]