Search

Multi-objective particle swarm optimization (MOPSO) algorithms based on angle preference provide a set of preferred solutions by incorporating a user's preference. However, since the search mechanism is stochastic and asymmetric, traditional MOPSO based on angle preference are still easy to fall into local optima and lack enough selection pressure on excellent individuals. In this paper, an improved MOPSO algorithm based on angle preference called IAPMOPSO is proposed to alleviate those problems. First, to create a stricter partial order among the non-dominated solutions, reference vectors are established in the preference region, and the adaptive penalty-based boundary intersection (PBI) value is used to update the external archive. Second, to effectively alleviate the swarm to fall into local optima, an adaptive preference angle is designed to increase the diversity of the population. Third, neighborhood individuals are selected for each particle to update the individual optimum to increase the information exchange among the particles. With the proposed angle preference-based external archive update strategy, solutions with a smaller PBI are given higher priority to be selected, and thus the selection pressure on excellent individuals is enhanced. In terms of an increase in the diversity of the population, the adaptive preference angle adjustment strategy that gradually narrows the preferred area, and the individual optimum update strategy which updates the individual optimum according to the information of neighborhood individuals, are presented. The experimental results on the benchmark test functions and GEM data verify the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Background: The main goal of successful gene selection for microarray data is to find compact and predictive gene subsets which could improve the accuracy. Though a large pool of available methods exists, selecting the optimal gene subset for accurate classification is still very challenging for the diagnosis and treatment of cancer. Results: To obtain the most predictive genes subsets without filtering out critical genes, a gene selection method based on least absolute shrinkage and selection operator (LASSO) and an improved binary particle swarm optimization (BPSO) is proposed in this paper. To avoid overfitting of LASSO, the initial gene pool is divided into clusters based on their structure. LASSO is then employed to select high predictive genes and further calculate the contribution value which indicates the genes' sensitivity to samples' classes. With the second-level gene pool established by double filter strategy, the BPSO encoding the contribution information obtained from LASSO is improved to perform gene selection. Moreover, from the perspective of the bit change probability, a new mapping function is defined to guide the updating of the particle to select the more predictive genes in the improved BPSO. Conclusions: With the compact gene pool obtained by double filter strategies, the improved BPSO could select the optimal gene subsets with high probability. The experimental results on several public microarray data with extreme learning machine verify the effectiveness of the proposed method compared to the relevant methods. [ABSTRACT FROM AUTHOR]

Abstract Extreme learning machine (ELM) for random single-hidden-layer feedforward neural networks (RSLFN) has been widely applied in many fields in the past ten years because of its fast learning speed and good generalization performance. But because traditional ELM randomly selects the input weights and hidden biases, it typically requires high number of hidden neurons and thus decreases its convergence performance. It is necessary to select optimal input weights and hidden biases to improve the convergence performance of the traditional ELM. Generally, the single-hidden-layer feedforward neural networks (SLFN) with low input-to-output sensitivity will cause good robustness of the network, which may further lead into good generalization performance. Moreover, particle swarm optimization (PSO) has no complicated evolutionary operators and fewer parameters need to adjust, and is easy to implement. In this study, an improved ELM based on PSO and input-to-output sensitivity information is proposed to improve RSLFN's convergence performance. In the improved ELM, PSO encoding the input to output sensitivity information of the SLFN is used to optimize the input weights and hidden biases. The improved ELM could obtain better generalization performance as well as improve the conditioning of the SLFN by decreasing the input-to-output sensitivity of the network. Finally, experiment results on the regression and classification problems verify the improved performance the proposed ELM. [ABSTRACT FROM AUTHOR]

Radial basis function (RBF) neural networks for Multi-Instance Multi-Label (MIML) directly can exploit the connections between instances and labels so that they can preserve useful prior information, but they only adopt Gaussian radial basis function as their RBF whose parameters are difficult to determine. In this paper, parameters can be obtained by multi-objective optimization methods with multi performance measures treated as objectives, specifically, parameter estimation of different RBFs by an improved multi-objective particle swarm optimization (MOPSO) is proposed where Recall rate and Precision rate are chosen to obtain the most desirable Pareto optimal solution set. Furthermore, share-learning factor is proposed to modify the particle velocity in standard MOPSO to improve the global search ability and group cooperative ability. It is experimentally demonstrated that the proposed method can estimate the reliable parameters of different RBFs, and it is also very competitive with the state of art MIML methods. [ABSTRACT FROM AUTHOR]

In real life, object detection is widely applied and plays a significant part in the field of computer vision. However, when detecting small objects, the advanced You Only Look Once v4 (YOLO v4) model often misses or incorrectly detects them due to the limited resolution and unclear features of the small objects, which reduces the detection accuracy. A small object detection algorithm based on YOLO v4 and Multi-scale Contextual information and Soft-CIOU loss function, called MCS-YOLO v4, is proposed in this paper. MCS-YOLO v4 adds a scale detection to the existing three scales to obtain rich location information. To enhance the ability of network to locate and classify the object, MCS-YOLO v4 introduces an expanded field-of-perception block. This block obtains the object contextual features and integrates them with the convolutional features to obtain more robust and discriminative features. To reduce the influence of insignificant information in images on small object, the attention module is introduced in the neck part of YOLO v4. To further improve the detection accuracy of small objects, the Soft-CIOU loss function is proposed. The aspect ratio weight factor is introduced into the weight function of the CIOU (Complete-IOU) loss function, while the Euclidean distance is subjected to the open-root operation, which improves the contribution of small objects to the loss function and enhances the learning ability of the network for small objects. The experimental results on the publicly available small object datasets verify that the proposed model has better detection effect than other detection models in detecting small objects. [ABSTRACT FROM AUTHOR]

For ensemble learning, how to select and combine the candidate classifiers are two key issues which influence the performance of the ensemble system dramatically. Random vector functional link networks (RVFL) without direct input-to-output links is one of suitable base-classifiers for ensemble systems because of its fast learning speed, simple structure and good generalization performance. In this paper, to obtain a more compact ensemble system with improved convergence performance, an improved ensemble of RVFL based on attractive and repulsive particle swarm optimization (ARPSO) with double optimization strategy is proposed. In the proposed method, ARPSO is applied to select and combine the candidate RVFL. As for using ARPSO to select the optimal base RVFL, ARPSO considers both the convergence accuracy on the validation data and the diversity of the candidate ensemble system to build the RVFL ensembles. In the process of combining RVFL, the ensemble weights corresponding to the base RVFL are initialized by the minimum norm least-square method and then further optimized by ARPSO. Finally, a few redundant RVFL is pruned, and thus the more compact ensemble of RVFL is obtained. Moreover, in this paper, theoretical analysis and justification on how to prune the base classifiers on classification problem is presented, and a simple and practically feasible strategy for pruning redundant base classifiers on both classification and regression problems is proposed. Since the double optimization is performed on the basis of the single optimization, the ensemble of RVFL built by the proposed method outperforms that built by some single optimization methods. Experiment results on function approximation and classification problems verify that the proposed method could improve its convergence accuracy as well as reduce the complexity of the ensemble system. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a new approach coupling adaptive particle swarm optimization (APSO) and a priori information for function approximation problem is proposed to obtain better generalization performance and faster convergence rate. It is well known that gradient-based learning algorithms such as backpropagation (BP) algorithm have good ability of local search, whereas PSO has good ability of global search. Therefore, in the new approach, first, APSO encoding the first-order derivative information of the approximated function is applied to train network to near global minima. Second, with the connection weights produced by APSO, the network is trained with a gradient-based algorithm. Due to combining APSO with local search algorithm and considering a priori information, the new approach has better generalization performance and convergence rate than traditional learning ones. Finally, simulation results are given to verify the efficiency and effectiveness of the proposed approach. [Copyright &y& Elsevier]

The diversity of the population is a key factor for particle swarm optimisation (PSO) when dealing with most optimisation problems. The best previously visited positions of each particle are the exemplar in PSO to guide particle swarm to search, and the diversity of the population can be controlled by these best previously visited positions. Base on this idea of to control the diversity of population to improve the performance of PSO, this paper proposes a novel PSO with mutation breeding (MBPSO), which performs a mutation breeding operation periodically, to control the diversity of the population to improve the global optimisation ability. The mutation breeding operation can be divided into two steps: breeding and mutation. The breeding step is to replace all of best previously visited positions of each particle with the global best previously visited position, and the mutation step is to perform a mutation operation for those new generated best previously visited positions. In addition, we adopt a new updating mechanism of the global best previously position to avoid falling into local optimum. The experimental results on a suit of benchmark functions verifies that the proposed PSO is a competitive algorithm when compare with other PSO variants. [ABSTRACT FROM AUTHOR]

Feature selection is an important task to improve the classifier's accuracy and to decrease the problem size. A number of methodologies have been presented for feature selection problems using metaheuristic algorithms. In this paper, an improved self-adaptive inertia weight particle swarm optimisation with local search and combined with C4.5 classifiers for feature selection algorithm is proposed. In this proposed algorithm, the gradient base local search with its capacity of helping to explore the feature space and an improved self-adaptive inertia weight particle swarm optimisation with its ability to converge a best global solution in the search space. Experimental results have verified that the SIW-APSO-LS performed well compared with other state of art feature selection techniques on a suit of 16 standard data sets. [ABSTRACT FROM AUTHOR]

As an evolutionary computing technique, particle swarm optimization (PSO) has good global search ability, but the swarm can easily lose its diversity, leading to premature convergence. To solve this problem, an improved self-inertia weight adaptive particle swarm optimization algorithm with a gradient-based local search strategy (SIW-APSO-LS) is proposed. This new algorithm balances the exploration capabilities of the improved inertia weight adaptive particle swarm optimization and the exploitation of the gradient-based local search strategy. The self-inertia weight adaptive particle swarm optimization (SIW-APSO) is used to search the solution. The SIW-APSO is updated with an evolutionary process in such a way that each particle iteratively improves its velocities and positions. The gradient-based local search focuses on the exploitation ability because it performs an accurate search following SIW-APSO. Experimental results verified that the proposed algorithm performed well compared with other PSO variants on a suite of benchmark optimization functions. [ABSTRACT FROM AUTHOR]

Abstract Random single-hidden layer feedforward neural network (RSLFN) is currently a popular learning algorithm proposed for improving traditional gradient-based model due to its fast learning speed and acceptable performance. For RSLFN, the input weights and/or other parameters are randomly initialized, and the other ones are iteratively or non-iteratively trained. However, the performance of RSLFN is sensitive to the number of hidden neurons and randomly initialized parameters. Numerous methods have been successfully employed to improve the RSLFN from various perspectives. Because of their favourable search ability, metaheuristic optimization approaches gradually attract more and more attentions. Metaheuristic algorithms usually formulate the random parameters of RSLFN into an optimization model, and then provide a near-optimum solution which could be converted into RSLFN with better generalization performance. The hybrid method for optimizing RSLFN therefore shows considerable potential in intelligent computing and artificial intelligence. However, there is no comprehensive survey on RSLFN with metaheuristic in the research area, which ultimately leads to lost opportunities for an advancement. This paper firstly introduces the basic principles of RSLFN along with several metaheuristic algorithms. Secondly, it provides a comprehensive survey of the state-of-the-art contributions in the area. Finally, current challenges are highlighted and promising research directions are also presented. [ABSTRACT FROM AUTHOR]

To obtain predictive genes with lower redundancy and better interpretability, a hybrid gene selection method encoding prior information is proposed in this paper. To begin with, the prior information referred to as gene-to-class sensitivity (GCS) of all genes from microarray data is exploited by a single hidden layered feedforward neural network (SLFN). Then, to select more representative and lower redundant genes, all genes are grouped into some clusters by K-means method, and some low sensitive genes are filtered out according to their GCS values. Finally, a modified binary particle swarm optimization (BPSO) encoding the GCS information is proposed to perform further gene selection from the remainder genes. For considering the GCS information, the proposed method selects those genes highly correlated to sample classes. Thus, the low redundant gene subsets obtained by the proposed method also contribute to improve classification accuracy on microarray data. The experiments results on some open microarray data verify the effectiveness and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

Abstract: Recently Extreme Learning Machine (ELM) for single-hidden-layer feedforward neural networks (SLFN) has been attracting attentions for its faster learning speed and better generalization performance than those of traditional gradient-based learning algorithms. However, ELM may need high number of hidden neurons and lead to ill-condition problem due to the random determination of the input weights and hidden biases. In this paper, a hybrid learning algorithm is proposed to overcome the drawbacks of ELM, which uses an improved particle swarm optimization (PSO) algorithm to select the input weights and hidden biases and Moore–Penrose (MP) generalized inverse to analytically determine the output weights. In order to obtain optimal SLFN, the improved PSO optimizes the input weights and hidden biases according to not only the root mean squared error (RMSE) on validation set but also the norm of the output weights. The proposed algorithm has better generalization performance than traditional ELM and other evolutionary ELMs, and the conditioning of the SLFN trained by the proposed algorithm is also improved. Experiment results have verified the efficiency and effectiveness of the proposed method. [Copyright &y& Elsevier]

Multi-leader comprehensive learning particle swarm optimiser possesses strong exploitation ability, by randomly selecting and assigning best-ranked particles as leaders during optimisation. However, it lacks the ability to preserve diversity by mainly focusing on exploitation, and adopting random selection to choose leaders also hinders its performance. To overcome these deficiencies, an improved multi-leader comprehensive learning particle swarm optimiser is proposed based on Karush-Kuhn-Tucker proximity measure and Gravitational Search Algorithm. Karush-Kuhn-Tucker proximity measure is employed to determine the best-ranked particles' contribution to the swarm's convergence to influence their selection as guides for other particles. Gravitational Search Algorithm is introduced to preserve the algorithm's ability to maintain diversity. To curb premature convergence and particles getting trapped in a local optimum, an adaptive reset velocity strategy is incorporated to activate stagnated particles. Some benchmark test functions are employed to compare the proposed algorithm with seven other peer algorithms. The results verify that our proposed algorithm possesses a better capability to elude local optima with faster convergence than other algorithms. Furthermore, to prove the efficacy of the application of our proposed algorithm in real-life, the algorithms are used to train a Feedforward neural network for epilepsy detection, of which our proposed algorithm outperforms the others. [ABSTRACT FROM AUTHOR]

How to determine the network structure is an open problem in extreme learning machine (ELM). Error minimized extreme learning machine (EM-ELM) is a simple and efficient approach to determine the number of hidden nodes. However, similar to other constructive ELM, EM-ELM lays much emphasis on the convergence accuracy, which may obtain a single-hidden-layer feedforward neural networks (SLFN) with good convergence performance but bad condition. In this paper, an effective approach based on error minimized ELM and particle swarm optimization (PSO) is proposed to adaptively determine the structure of SLFN for regression problem. In the new method, to establish a compact and well-conditioning SLFN, the hidden node optimized by PSO is added to the SLFN one by one. Moreover, not only the regression accuracy but also the condition value of the hidden output matrix of the network is considered in the optimization process. Experiment results on various regression problems verify that the proposed algorithm achieves better generalization performance with fewer hidden nodes than other constructive ELM. [ABSTRACT FROM AUTHOR]