Search

Novelty search is a powerful tool for finding diverse sets of objects in complicated spaces. Recent experiments on simplified versions of novelty search introduce the idea that novelty search happens at the level of the archive space, rather than individual points. The sparseness measure and archive update criterion create a process that is driven by a two measures: (1) spread out to cover the space while trying to remain as efficiently packed as possible, and (2) metrics inspired by k nearest neighbor theory. In this paper, we generalize previous simplifications of novelty search to include traditional population (μ,λ) dynamics for generating new search points, where the population and the archive are updated separately. We provide some theoretical guidance regarding balancing mutation and sparseness criteria and introduce the concept of saturation as a way of talking about fully covered spaces. We show empirically that claims that novelty search is inherently objectiveless are incorrect. We leverage the understanding of novelty search as an optimizer of archive coverage, suggest several ways to improve the search, and demonstrate one simple improvement-generating some new points directly from the archive rather than the parent population., (© 2023 Massachusetts Institute of Technology.)

Malaria is a significant health concern worldwide, particularly in Africa where its prevalence is still alarmingly high. Using artificial intelligence algorithms to diagnose cells with malaria provides great convenience for clinicians. In this paper, a densely connected convolutional dynamic learning network (DCDLN) is proposed for the diagnosis of malaria disease. Specifically, after data processing and partitioning of the dataset, the densely connected block is trained as a feature extractor. To classify the features extracted by the feature extractor, a classifier based on a dynamic learning network is proposed in this paper. Based on experimental results, the proposed DCDLN method demonstrates a diagnostic accuracy rate of 97.23%, surpassing the diagnostic performance than existing advanced methods on an open malaria cell dataset. This accurate diagnostic effect provides convincing evidence for clinicians to make a correct diagnosis. In addition, to validate the superiority and generalization capability of the DCDLN algorithm, we also applied the algorithm to the skin cancer and garbage classification datasets. DCDLN achieved good results on these datasets as well, demonstrating that the DCDLN algorithm possesses superiority and strong generalization performance., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

The Differential Evolution (DE) algorithm is one of the most successful evolutionary computation techniques. However, its structure is not trivially translatable in terms of mathematical transformations that describe its population dynamics. In this work, analytical expressions are developed for the probability of enhancement of individuals after each application of a mutation operator followed by a crossover operation, assuming a population distributed radially around the optimum for the sphere objective function, considering the DE/rand/1/bin and the DE/rand/1/exp algorithm versions. These expressions are validated by numerical experiments. Considering quadratic functions given by f(x)=xTDTDx and populations distributed according to the linear transformation D-1 of a radially distributed population, it is also shown that the expressions still hold in the cases when f(x) is separable (D is diagonal) and when D is any nonsingular matrix and the crossover rate is Cr=1.0. The expressions are employed for the analysis of DE population dynamics. The analysis is extended to more complex situations, reaching rather precise predictions of the effect of problem dimension and of the choice of algorithm parameters., (© 2022 Massachusetts Institute of Technology.)

For multilayer perceptron (MLP), the initial weights will significantly influence its performance. Based on the enhanced fractional derivative extend from convex optimization, this paper proposes a fractional gradient descent (RFGD) algorithm robust to the initial weights of MLP. We analyze the effectiveness of the RFGD algorithm. The convergence of the RFGD algorithm is also analyzed. The computational complexity of the RFGD algorithm is generally larger than that of the gradient descent (GD) algorithm but smaller than that of the Adam, Padam, AdaBelief, and AdaDiff algorithms. Numerical experiments show that the RFGD algorithm has strong robustness to the order of fractional calculus which is the only added parameter compared to the GD algorithm. More importantly, compared to the GD, Adam, Padam, AdaBelief, and AdaDiff algorithms, the experimental results show that the RFGD algorithm has the best robust performance for the initial weights of MLP. Meanwhile, the correctness of the theoretical analysis is verified., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks. This work takes one step on this direction by introducing the complex-reaction network with fully-connected feed-forward architecture. We prove the universal approximation property for complex-reaction networks, and show that a class of radial functions can be approximated by a complex-reaction network using the polynomial number of parameters, whereas real-valued networks need at least exponential parameters to reach the same approximation level. For empirical risk minimization, we study the landscape and convergence of complex gradient descents. Our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks, which may show some insights on finding the optimal solutions more easily for complex-reaction networks., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Sign-based Stochastic Gradient Descents (Sign-based SGDs) use the signs of the stochastic gradients for communication costs reduction. Nevertheless, current convergence results of sign-based SGDs applied to the finite sum optimization are established on the bounded assumption of the gradient, which fails to hold in various cases. This paper presents a convergence framework about sign-based SGDs with the elimination of the bounded gradient assumption. The ergodic convergence rates are provided only with the smooth assumption of the objective functions. The Sign Stochastic Gradient Descent (signSGD) and its two variants, including majority vote and zeroth-order version, are developed for different application settings. Our framework also removes the bounded gradient assumption used in the previous analysis of these three algorithms., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

A norm version of the RMSProp algorithm with penalty (termed RMSPropW) is introduced into the deep learning framework and its convergence is addressed both analytically and numerically. For rigour, we consider the general nonconvex setting and prove the boundedness and convergence of the RMSPropW method in both deterministic and stochastic cases. This equips us with strict upper bounds on both the moving average squared norm of the gradient and the norm of weight parameters throughout the learning process, owing to the penalty term within the proposed cost function. In the deterministic (batch) case, the boundedness of the moving average squared norm of the gradient is employed to prove that the gradient sequence converges to zero when using a fixed step size, while with diminishing stepsizes, the minimum of the gradient sequence converges to zero. In the stochastic case, due to the boundedness of the weight evolution sequence, it is further shown that the weight sequence converges to a stationary point with probability 1. Finally, illustrative simulations are provided to support the theoretical analysis, including a comparison with the standard RMSProp on MNIST, CIFAR-10, and IMDB datasets., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

Checkerboard is a typical instability in finite element (FE) simulations of bone adaptation and topology optimization in general. It consists in a patchwork pattern with elements of alternating stiffness, producing lack of convergence and instabilities in the predicted bone density. Averaging techniques have been proposed to solve this problem. One of the most acknowledged techniques (node based formulation) has severe drawbacks such as: high sensitivity to mesh density and type of element integration (full vs reduced) and, more importantly, oscillatory solutions also leading to lack of convergence. We propose a new solution consisting in a non-local smoothing technique. It defines, as the mechanical stimulus governing bone adaptation in a certain integration point of the mesh, the average of the stimuli obtained in the neighbour integration points. That average is weighted with a decay function of the distance to the centre of the neighbourhood. The new technique has been shown to overcome all the referred problems and perform in a robust way. It was tested on a hollow cylinder, resembling the diaphysis of a long bone, subjected to bending or torsion. Checkerboard instability was eliminated and local convergence of bone adaptation was achieved rapidly, in contrast to the other averaging technique and to the model without control of checkerboard instability. The new algorithm was also tested with good results on the same geometry but in a model containing a void, which produces a stress concentration that usually leads to checkerboard instability, like in other applications such as simulations of bone-implant interfaces., (© 2020 John Wiley & Sons, Ltd.)

Aiming at the problem that many algorithms could not effectively balance the global search ability and local search ability, a new optimization algorithm is proposed. Inspired by Finite Element Analysis (FEA) approach, a relationship of mapping between Finite Element Analysis approach and a population-based optimization algorithm is constructed through comparing the similarities and differences of FEA node and ideal particle. In algorithm framework, the stiffness coefficient corresponds to a user-defined function of the value of an objective function to be optimized, and the node forces among individuals are defined and an attraction-repulsion rule is established among them. The FEA approach that can simulate multi- states of matter is adopted to balance the global search ability and local search ability in the novel optimization algorithm. A theoretical analysis is made for algorithm parallelism. The conditions for convergence are deduced through analyzing the algorithm based on discrete-time linear system theory. In addition, the performance of the algorithm is compared with PSO for five states which include free state, diffusion state, solid state, entirely solid state, synthesis state. The simulation results of six benchmark functions show that the algorithm is effective. The algorithm supplies a new method to solve optimization problem.

Learning in physical neural systems must rely on learning rules that are local in both space and time. Optimal learning in deep neural architectures requires that non-local information be available to the deep synapses. Thus, in general, optimal learning in physical neural systems requires the presence of a deep learning channel to communicate non-local information to deep synapses, in a direction opposite to the forward propagation of the activities. Theoretical arguments suggest that for circular autoencoders, an important class of neural architectures where the output layer is identical to the input layer, alternative algorithms may exist that enable local learning without the need for additional learning channels, by using the forward activation channel as the deep learning channel. Here we systematically identify, classify, and study several such local learning algorithms, based on the general idea of recirculating information from the output layer to the hidden layers. We show through simulations and mathematical derivations that these algorithms are robust and converge to critical points of the global error function. In most cases, we show that these recirculation algorithms are very similar to an adaptive form of random backpropagation, where each hidden layer receives a linearly transformed, slowly-varying, version of the output error., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

Unlike the first and the second generation artificial neural networks, spiking neural networks (SNNs) model the human brain by incorporating not only synaptic state but also a temporal component into their operating model. However, their intrinsic properties require expensive computation during training. This paper presents a novel algorithm to SpikeProp for SNN by introducing smoothing L 1∕2 regularization term into the error function. This algorithm makes the network structure sparse, with some smaller weights that can be eventually removed. Meanwhile, the convergence of this algorithm is proved under some reasonable conditions. The proposed algorithms have been tested for the convergence speed, the convergence rate and the generalization on the classical XOR-problem, Iris problem and Wisconsin Breast Cancer classification., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

The Congruent Matching Cells (CMC) method was invented at the National Institute of Standards and Technology (NIST) for accurate firearm evidence identification and error rate estimation. The CMC method is based on the principle of discretization. The toolmark image of the reference sample is divided into correlation cells. Each cell is registered to the cell-sized area of the compared image that has maximum surface topography similarity. For each resulting cell pair, one parameter quantifies the similarity of the cell surface topography and three parameters quantify the pattern congruency of the registration position and orientation. An identification (declared match) requires a significant number of CMCs, that is, cell pairs that meet both similarity and pattern congruency requirements. The use of cell correlations reduces the effects of "invalid regions" in the compared image pairs and increases the correlation accuracy. The identification accuracy of the CMC method can be further improved by considering a feature named "convergence," that is, the tendency of the x-y registration positions of the correlated cell pairs to converge at the correct registration angle when comparing same-source samples at different relative orientations. In this paper, the difference of the convergence feature between known matching (KM) and known non-matching (KNM) image pairs is characterized, based on which an improved algorithm is developed for breech face image correlations using the CMC method. Its advantage is demonstrated by comparison with three existing CMC algorithms using four datasets. The datasets address three different brands of consecutively manufactured pistol slides, with significant differences in the distribution overlap of cell pair topography similarity for KM and KNM image pairs. For the same CMC threshold values, the convergence algorithm demonstrates noticeably improved results by reducing the number of false-positive or false-negative CMCs in a comparison., (Published by Elsevier B.V.)

Weight elimination offers a simple and efficient improvement of training algorithm of feedforward neural networks. It is a general regularization technique in terms of the flexible scaling parameters. Actually, the weight elimination technique also contains the weight decay regularization for a large scaling parameter. Many applications of this technique and its improvements have been reported. However, there is little research concentrated on its convergence behavior. In this paper, we theoretically analyze the weight elimination for cyclic learning method and determine the conditions for the uniform boundedness of weight sequence, and weak and strong convergence. Based on the assumed network parameters, the optimal choice for the scaling parameter can also be determined. Moreover, two illustrative simulations have been done to support the theoretical explorations as well., (Copyright © 2016 Elsevier Ltd. All rights reserved.)

This paper presents an augmented algorithm for fully complex-valued neural network based on Wirtinger calculus, which simplifies the derivation of the algorithm and eliminates the Schwarz symmetry restriction on the activation functions. A unified mean value theorem is first established for general functions of complex variables, covering the analytic functions, non-analytic functions and real-valued functions. Based on so introduced theorem, convergence results of the augmented algorithm are obtained under mild conditions. Simulations are provided to support the analysis., (Copyright © 2015 Elsevier Ltd. All rights reserved.)

The aim of this paper is to develop a novel method to prune feedforward neural networks by introducing an L1/2 regularization term into the error function. This procedure forces weights to become smaller during the training and can eventually removed after the training. The usual L1/2 regularization term involves absolute values and is not differentiable at the origin, which typically causes oscillation of the gradient of the error function during the training. A key point of this paper is to modify the usual L1/2 regularization term by smoothing it at the origin. This approach offers the following three advantages: First, it removes the oscillation of the gradient value. Secondly, it gives better pruning, namely the final weights to be removed are smaller than those produced through the usual L1/2 regularization. Thirdly, it makes it possible to prove the convergence of the training. Supporting numerical examples are also provided., (Copyright © 2013 Elsevier Ltd. All rights reserved.)

In this paper a new meta-heuristic search method, called Cat Swarm Optimization (CSO) algorithm is applied to determine the best optimal impulse response coefficients of FIR low pass, high pass, band pass and band stop filters, trying to meet the respective ideal frequency response characteristics. CSO is generated by observing the behaviour of cats and composed of two sub-models. In CSO, one can decide how many cats are used in the iteration. Every cat has its' own position composed of M dimensions, velocities for each dimension, a fitness value which represents the accommodation of the cat to the fitness function, and a flag to identify whether the cat is in seeking mode or tracing mode. The final solution would be the best position of one of the cats. CSO keeps the best solution until it reaches the end of the iteration. The results of the proposed CSO based approach have been compared to those of other well-known optimization methods such as Real Coded Genetic Algorithm (RGA), standard Particle Swarm Optimization (PSO) and Differential Evolution (DE). The CSO based results confirm the superiority of the proposed CSO for solving FIR filter design problems. The performances of the CSO based designed FIR filters have proven to be superior as compared to those obtained by RGA, conventional PSO and DE. The simulation results also demonstrate that the CSO is the best optimizer among other relevant techniques, not only in the convergence speed but also in the optimal performances of the designed filters., (Copyright © 2013 ISA. Published by Elsevier Ltd. All rights reserved.)

In this paper, we first develop the modified gradient‐based iterative (MGI) method for the complex conjugate and transpose matrix equations A1XB1+A2X‾B2+A3XTB3+A4XHB4=E$$ {A}_1X{B}_1+{A}_2\overline{X}{B}_2+{A}_3{X}^T{B}_3+{A}_4{X}^H{B}_4=E $$. By adopting the updated technique, we can make full use of the latest information to compute the next result, which leads to a faster convergence rate. In theory, we apply the real representation of a complex matrix and the vec‐operator to prove the convergence properties. Furthermore, we extend the MGI algorithm to solve the generalized complex conjugate and transpose matrix equations. Then, the necessary and sufficient conditions for convergence of the MGI algorithm are presented. Lastly, three numerical examples are introduced to testify the efficiency of our methods. [ABSTRACT FROM AUTHOR]

In this article, a new numerical algorithm for solving a 1-dimensional (1D) and 2-dimensional (2D) time-fractional diffusion equation is proposed. The Sinc-Galerkin scheme is considered for spatial discretization, and a higher-order finite difference formula is considered for temporal discretization. The convergence behavior of the methods is analyzed, and the error bounds are provided. The main objective of this paper is to propose the error bounds for 2D problems by using the Sinc-Galerkin method. The proposed method in terms of convergence is studied by using the characteristics of the Sinc function in detail with optimal rates of exponential convergence for 2D problems. Some numerical experiments validate the theoretical results and present the efficiency of the proposed schemes. [ABSTRACT FROM AUTHOR]

In this investigation, a new algorithm based on the compact difference method is proposed. The purpose of this investigation is to solve the 2D time-fractional integro-differential equation. The Riemann–Liouville derivative was utilized to define the time-fractional derivative. Meanwhile, the weighted and shifted Grünwald difference operator and product trapezoidal formula were utilized to construct a high-order numerical scheme. Also, we analyzed the stability and convergence. The convergence order was O (τ 2 + h x 4 + h y 4) , where τ is the time step size, h x and h y are the spatial step sizes. Furthermore, several examples were provided to verify the correctness of our theoretical reasoning. [ABSTRACT FROM AUTHOR]

In the past decade, multi-UAV systems have gained significant attention and companies from in various fields. The main aims of growing such systems are able to operate in a coordinated way in complicated situations. To encompass and increase the speed of the applications, this research proposed optimization with intelligence. In the existing work, it does not detect the collision effectively. In this work, the Particle Swarm Optimization algorithm based on fast matching square is proposed for flightpath planning in Multi-UAVs is used. A distributed full coverage optimal route design is derived using the particle swarm optimization (PSO) technique, and a trajectory planner is built utilizing a dynamic fitness function. We applied the PSO technique for every UAV individually to achieve active fitness, maximizing the fitness function while reducing the cost function. The novelty of this work is using the dynamic fitness function for PSO while selecting optimal data. Fast matching square FM2 helps to detect the collision and avoid collisions between the paths. The suggested method generates an optimal path for multi-UAVs and updates the maps for swarms of UAVs. According to experimental findings, the suggested PSO-FM2 outperforms the previous FOA in aspects of convergence and accuracy. Finally, the proposed method is compared with existing algorithms such as MSFOA and PSO. [ABSTRACT FROM AUTHOR]

Summary: In the process of evolution, the multi‐target Firefly algorithm has a single learning strategy and is easy to fall into premature convergence, resulting in poor comprehensive performance of the algorithm. To solve this problem, a multi‐objective firefly algorithm (MOFA‐LM) combining logistic mapping and Cauchy mutation is proposed in this article. To improve the uneven distribution of the initial population, the initial population with good ergodicity is generated by logistic mapping. Aiming at the problem that the population is easy to fall into local optimal, Cauchy mutation can generate relatively large disturbance when fireflies are updating their positions, which makes it easier to jump out of the local optimal value with a wide range of individual optimization, effectively improve the optimization accuracy of the algorithm, overcome the premature convergence of the algorithm and maintain the convergence of the population. In the experimental part, the proposed algorithm is compared with some multi‐objective optimization algorithms in 19 benchmark test functions, and the effectiveness of the strategy added by MOFA‐LM is verified. The results show that MOFA‐LM has obvious advantages over other algorithms in improving population convergence and distribution. [ABSTRACT FROM AUTHOR]

We present an altered version of the inertial Krasnosel’skii-Mann algorithm and demonstrate convergence outcomes for mappings that are asymptotically nonexpansive within real, uniformly convex Banach spaces. To achieve our results, we skillfully construct the inequality in equation (6) and apply it accordingly. Our findings support and broadly generalize a number of significant findings from the literature. We demonstrate, as an application, the generation of maximal monotone operators’ zeros via fixed point methods in Hilbert spaces. Additionally, we solve convex minimization issues using our fixed-point techniques. [ABSTRACT FROM AUTHOR]

To improve the convergence of the gradient iterative (GI) algorithm and the Jacobi‐gradient iterative (JGI) algorithm [Bayoumi, Appl Math Inf Sci, 2021], a shift‐splitting Jacobi‐gradient iterative (SSJGI) algorithm for solving the matrix equation A풱−풱‾B=C is presented in this paper, which is based on the splitting of the coefficient matrices. The proposed algorithm converges to the exact solution for any initial value with some conditions. To demonstrate the effectiveness of the SSJGI algorithm and to compare it to the GI algorithm and the JGI algorithm [Bayoumi, Appl Math Inf Sci, 2021], numerical examples are provided. [ABSTRACT FROM AUTHOR]

In this paper, efficient algorithms will be studied for solving nonlinear saddle point problems from the elliptic optimal control problems. Based on the preconditioned inexact Uzawa (PIU) algorithmic framework, an improved preconditioned inexact Uzawa (I-PIU) algorithmic framework will first be constructed for solving a class of block-structured nonlinear systems, which possess a more general form of the nonlinear saddle point systems. When the I-PIU algorithmic framework is used for solving the block-structured nonlinear systems, a detailed theoretical analysis of the global convergence properties is proposed under the case that the nonlinear operator is single-valued. Besides, a special I-PIU method is established based on a new matrix splitting technique at the first iterative step of the I-PIU algorithmic framework, and then we establish the detailed theoretical results of the convergence. Finally, some numerical experiments are given to demonstrate the efficiency of the I-PIU method. [ABSTRACT FROM AUTHOR]

Many-objective evolutionary algorithms often struggle to strike a balance between convergence and diversity when solving many-objective optimization problems. Consequently, an adaptive boundary-based selection many-objective evolutionary algorithm with density estimation, MaOEA/ABS-DE, is proposed. Specifically, the algorithm initially utilizes prior information regarding the adaptive estimation of the shape of the Pareto front to perform coordinate transformation on the population, thus guiding the population towards the approximate boundary of the current Pareto front. Subsequently, a novel approach for estimating the neighboring density of individuals is introduced by utilizing the distribution information of the transformed population. Furthermore, adaptive boundary-based selection and shifted-based density estimation selection strategies are designed for the environmental selection process. The environmental selection process is completed through a two-step selection. In the first step, a candidate pool is formed by selecting solutions with lower neighboring density, which aims to maintain a broad search for the population in the objective space. In the second step, the SDE is introduced to compare the convergence of individuals in the candidate pool, and those with the optimal SDE values are selected. Finally, extensive comparative experiments are conducted on two benchmark test suites, MaF (Many-objective Functions) and WFG (Walking Fish Group), as well as on two practical cases. The MaOEA/ABS-DE is compared with five representative many-objective evolutionary algorithms, including 1by1EA, NSGA-III, MaOEA-CSS, RVEA, and SPEA2 + SDE. Experimental results demonstrate that the proposed algorithm outperforms or at least matches the performance of the five algorithms on 87.04%, 92.59%, 87.04%, 88.89%, and 87.04% of the test instances, respectively. [ABSTRACT FROM AUTHOR]

In this paper, a high-accuracy conservative implicit algorithm for computing the space fractional coupled Schrödinger–Boussinesq system is constructed. Meanwhile, the conservative nature, a priori boundedness, and solvability of the numerical solution are presented. Then, the proposed algorithm is proved to be second-order convergence in temporal and fourth-order spatial convergence using the discrete energy method. Finally, some numerical experiments validate the effectiveness of the conservative algorithm and confirm the accuracy of the theoretical results for different choices of the fractional-order α. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to study multiple structural changes that occur at unknown locations in high-dimensional linear regression. We consider a structural change model where the parameters are subject to shifts at possibly different locations. We propose a penalized least squares approach, combined with a temporal difference penalty term for the difference between the coefficients at successive points for identifying latent change points, as well as a common sparsity penalty to detect important covariates. This procedure automatically estimates the number and the locations of the change-points and the parameters in each corresponding segment. To implement the proposed approach, we devise an alternating direction method of multipliers (ADMM) algorithm. We demonstrate the convergence of the proposed ADMM algorithm in the present setting. We also establish an oracle inequality for the proposed estimator. We carry out simulation studies to evaluate the finite sample performance of the proposed method and illustrate its application on a data set. [ABSTRACT FROM AUTHOR]

In engineering, most sensors can only measure the intensity information of a signal, but without the phase information. The phase retrieval problem is to reconstruct the original signal only from intensity values, which is an ill-posed problem. To address this issue, we introduce sparse prior information to overcome the ill-posed problem and propose the sparse reweighted thresholded Wirtinger flow (SRThWF) algorithm. In the SRThWF algorithm, the original signal is reconstructed from Gaussian random measurements via intensity values. In particular, SRThWF carries on iteratively in two stages: In stage one, it works with initialization by truncated spectral method restricted on support recovery of sparse signal. In the second stage, the initial estimate is refined iteratively via hard-thresholding-based adaptive reweighted gradient descent. Furthermore, the convergence of the proposed algorithm has been proved. Experimental results show that the proposed algorithm is an effective sparse phase retrieval solver in the probability of success even if the sparsity level is unknown. Not only that, the proposed algorithm exhibits a faster convergence rate and shorter running time. In addition, SRThWF is robust to the noise. [ABSTRACT FROM AUTHOR]

Iterative algorithms requiring the computationally expensive in general inversion of linear operators are difficult to implement. This is the reason why hybrid Newton-like algorithms without inverses are developed in this paper to solve Banach space-valued nonlinear equations. The inverses of the linear operator are exchanged by a finite sum of fixed linear operators. Two types of convergence analysis are presented for these algorithms: the semilocal and the local. The Fréchet derivative of the operator on the equation is controlled by a majorant function. The semi-local analysis also relies on majorizing sequences. The celebrated contraction mapping principle is utilized to study the convergence of the Krasnoselskij-like algorithm. The numerical experimentation demonstrates that the new algorithms are essentially as effective but less expensive to implement. Although the new approach is demonstrated for Newton-like algorithms, it can be applied to other single-step, multistep, or multipoint algorithms using inverses of linear operators along the same lines. [ABSTRACT FROM AUTHOR]

The dislocation hyperbolic augmented Lagrangian algorithm (DHALA) is a new approach to the hyperbolic augmented Lagrangian algorithm (HALA). DHALA is designed to solve convex nonlinear programming problems. We guarantee that the sequence generated by DHALA converges towards a Karush-Kuhn-Tucker point. We are going to observe that DHALA has a slight computational advantage in solving the problems over HALA. Finally, we will computationally illustrate our theoretical results. [ABSTRACT FROM AUTHOR]

This work focuses on exploring pointwise error estimate of three-level conservative difference scheme for supergeneralized viscous Burgers' equation. The cut-off function method plays an important role in constructing difference scheme and presenting numerical analysis. We study the conservative invariant of proposed method, which is energy-preserving for all positive integers p and q. Meanwhile, one could apply the discrete energy argument to the rigorous proof that the three-level scheme has unique solution combining the mathematical induction. In addition, we prove the L 2 -norm and L ∞ -norm convergence of proposed scheme in pointwise sense with separate and different ways, which is different from previous work in [1]. Numerical results verify the theoretical conclusions. [ABSTRACT FROM AUTHOR]

In this study, we extended the applicability of a derivative-free algorithm to encompass the solution of operators that may be either differentiable or non-differentiable. Conditions weaker than the ones in earlier studies are employed for the convergence analysis. The earlier results considered assumptions up to the existence of the ninth order derivative of the main operator, even though there are no derivatives in the algorithm, and the Taylor series on the finite Euclidian space restricts the applicability of the algorithm. Moreover, the previous results could not be used for non-differentiable equations, although the algorithm could converge. The new local result used only conditions on the divided difference in the algorithm to show the convergence. Moreover, the more challenging semi-local convergence that had not previously been studied was considered using majorizing sequences. The paper included results on the upper bounds of the error estimates and domains where there was only one solution for the equation. The methodology of this paper is applicable to other algorithms using inverses and in the setting of a Banach space. Numerical examples further validate our approach. [ABSTRACT FROM AUTHOR]

In this paper, we develop a Jacobi-type algorithm for the approximate diagonalization of tensors of order $ d\geq 3 $ d ≥ 3 via tensor trace maximization. For a general tensor, this is an alternating least squares algorithm and the rotation matrices are chosen in each mode one-by-one to maximize the tensor trace. On the other hand, for symmetric tensors, we discuss a structure-preserving variant of this algorithm where in each iteration the same rotation is applied in all modes. We show that both versions of the algorithm converge to the stationary points of the corresponding objective functions. [ABSTRACT FROM AUTHOR]

In this article, an adaptive normalized Sign Regression Least Mean Mixed Norm (ε-NSRLMMN) algorithm-based control technique is developed for the Shunt Active Power Filter (SAPF) to compensate for the grid current harmonics and reactive power under balanced and unbalanced loading conditions. The adaptive ε-NSRLMMN control algorithm improves power quality and makes the source current sinusoidal and power factor (PF) close to unity. It is used for load compensation in a three-phase distribution network. The fundamental active component is extracted from the distorted load current using an adaptive technique. This control algorithm has the advantages of improved convergence, better stability, lower harmonic distortion, less complexity and less steady-state error than the Sign regressor least mean mixed norm (SRLMMN), Least mean square (LMS) and least mean fourth (LMF). Adaptive ε-NSRLMMN is used for learning the fundamental active weight components of the load current. The parameters of the proposed algorithm are trained in real-time. A traditional PI controller has been used to regulate the dc-link voltage of the Voltage Source Converter (VSC). The performance of the proposed algorithm has been justified by simulation and experimental results. [ABSTRACT FROM AUTHOR]

As a universal approximator, the first order Takagi–Sugeno fuzzy system possesses the capability to approximate widespread nonlinear systems through a group of IF THEN fuzzy rules. Although group lasso regularization has the advantage of inducing group sparsity and handling variable selection issues, it can lead to numerical oscillations and theoretical challenges in calculating the gradient at the origin when employed directly during training. The paper addresses the aforementioned obstacle by invoking a smoothing function to approximate group lasso regularization. On this basis, a gradient‐based neuro fuzzy learning algorithm with smoothing group lasso regularization for the first order Takagi–Sugeno fuzzy system is proposed. The convergence of the proposed algorithm is rigorously proved under gentle conditions. In addition, experimental outcomes acquired on two approximations and two classification simulations demonstrate that the proposed algorithm outperforms the algorithm with original group lasso regularization and L2 regularization in terms of error, pruned neurons, and accuracy. This is particularly evident in significant advancements in pruned neurons due to group sparsity. In comparison to the algorithm with L2 regularization, the proposed algorithm exhibits improvements of 6.3, 5.3, and 142.6 in pruned neurons during sin(πx)$(\pi x)$ function, Gabor function, and Sonar benchmark dataset simulations, respectively. [ABSTRACT FROM AUTHOR]

Kinetic Monte Carlo (kMC) modelling is ubiquitous to simulate the time evolution of (bio)chemical processes, specifically if populations are involved. A recurring task is the selection of the smallest control volume that leads to convergence, which means that the model outputs are accurate and sufficiently free from stochastic noise and do not significantly change upon further increasing this volume. Selecting a too high (safe) control volume leads to an excessive simulation time, while many small incremental control volume increases are inefficient. This work therefore presents an automated tool to determine the smallest control volume leading to convergence. The tool is illustrated for (intrinsic) free radical and nitroxide mediated polymerization (FRP/NMP), in which the chain length distribution (CLD) is a crucial output. The algorithm starts with a very low volume to then check if the desired (monomer) conversion can be reached, the number average chain length is accurate, and finally the signal‐to‐noise (SNR) ratio at the CLD level is below a threshold. The execution time of the algorithm is less than twice the time of running the converged simulation directly, hence, saving tremendous time in setting up a kMC simulation and facilitating benchmark studies even beyond polymer reaction engineering applications. [ABSTRACT FROM AUTHOR]

A crucial element in detecting unusual network system behavior is the network intrusion detection system (NIDS), which also helps to stop network attacks from happening. Despite the fact that a great deal of machine learning techniques has been used in intrusion detection, current solutions still struggle to provide accurate classification results. Furthermore, when dealing with imbalanced multi-category traffic data, a single classifier may not be able to produce a superior. Particularly, internet of things (IoT) gadgets is now a commonplace aspect of life. On the other hand, some problems are becoming worse and lack clear remedies. Convergence, communication speed, and security between various IoT devices are among the primary concerns. In order to achieve this goal, an enhanced artificial bee colony technique utilizing binary search equations and neural networks--known as the (BABCN) algorithm for intrusion detection in terms of convergence and communication speed--is presented in this study. The artificial bee is improved by the depthfirst search framework and binary search equations upon which the BABCN method is based. The suggested approach has a good ability to detect intrusions in the network and enhances categorization, according to the findings obtained by using the NSL-KDD dataset. [ABSTRACT FROM AUTHOR]

This paper designs a new algorithm based on scale-3 Haar wavelets to emulate the nonlinear (2+1) Korteweg–de Vries–Rosenau-regularized long-wave model. For the flourish of the algorithm, the primary effort is semi-discretization in temporal variable via finite difference schemes, namely forward and Crank–Nicolson differences, and then truncation error is discussed. After that, full discretization is made via scale-3 Haar wavelets. In the end, the convergence analysis of the scale-3 wavelets is discussed. To check the competence and exactitude of the scheme and to predict the nature of the wave behaviors, one- and two-dimensional numerical problems are taken into account. [ABSTRACT FROM AUTHOR]

In the past few decades, to solve the multi-objective optimization problems, many multi-objective evolutionary algorithms (MOEAs) have been proposed. However, MOEAs have a common difficulty: because the diversity and convergence of solutions are often two conflicting conditions, the balance between the diversity and convergence directly determines the quality of the solutions obtained by the algorithms. Meanwhile, the nondominated sorting method is a costly operation in part Pareto-based MOEAs and needs to be optimized. In this article, we propose a multi-objective evolutionary algorithm framework with convergence and diversity adjusted adaptively. Our contribution is mainly reflected in the following aspects: firstly, we propose a nondominated sorting-based MOEA framework with convergence and diversity adjusted adaptively; secondly, we propose a novel fast nondominated sorting algorithm; thirdly, we propose a convergence improvement strategy and a diversity improvement strategy. In the experiments, we compare our method with several popular MOEAs based on two widely used performance indicators in several multi-objective problem test instances, and the empirical results manifest the proposed method performs the best on most test instances, which further demonstrates that it outperforms all the comparison algorithms. [ABSTRACT FROM AUTHOR]

Relying upon Legendre polynomials, an improved multiwavelet orthonormal basis is constructed in the reproducing kernel space. We study boundary value problems (BVPs) based on the improved wavelet orthonormal basis and construct the ε$$ \varepsilon $$‐best approximate solution. Convergence and the stability are discussed; this method has higher order convergence. The main advantage of the wavelet based our method is its efficiency and simple applicability for a variety of boundary conditions. Some numerical tests even including singular BVPs reveal the stability and efficiency of the new algorithm. [ABSTRACT FROM AUTHOR]

The Artificial Electric Field Algorithm (AEFA) is a recently developed optimization algorithm inspired by the principles of electrostatic force and the law of motion. It operates as a stochastic population-based algorithm and utilizes probabilistic techniques to search for solutions. Due to its stochastic nature, it is crucial to investigate the numerical stability of AEFA. Such a study helps in determining optimal parameter values for the AEFA scheme. In this article, we analyse the stability of AEFA using two different criteria: Von Neumann stability and agent dynamics. Both methods provide sufficient conditions for the stability of AEFA, enabling the adaptation of its parameters. We further examine the trajectory behaviour of agents by evaluating various benchmark functions from the CEC 2021 test suite. Through illustrative examples, we demonstrate the theoretical and experimental studies, highlighting the stable nature of AEFA and validating the proposed stability conditions. [ABSTRACT FROM AUTHOR]

The solution of convection diffusion problem is a complex task, requiring the solving of linear equations which can take up a significant amount of computational time. This issue can be mitigated through the use of iterative methods. We present a class of fast successive permutation iterative algorithms with relaxation factors (ω -SPI) to solve 1D, 2D and 3D convection diffusion equations. The stability, convergence and optimal value of the relaxation factor are proved. In order to test the efficiency and accuracy of our proposed iterative algorithms, we present several numerical experiments. The results demonstrate that the new algorithms are more accurate and faster than traditional methods such as full-implicit, Gauss-Seidel and SOR algorithms. [ABSTRACT FROM AUTHOR]

The K-Means clustering algorithm is a widely used technique for grouping data into clusters, with applications spanning various domains. This study presents a comparative investigation into the optimization of K-Means clustering through the evaluation of different optimization algorithms. The primary focus is on enhancing the convergence speed and computational efficiency of the K-Means algorithm, with implications for diverse real-world scenarios. The research systematically examines a range of optimization techniques, including gradient descent, stochastic gradient descent, and metaheuristic algorithms such as genetic algorithms and simulated annealing. A comprehensive analysis of convergence speed, clustering quality, and computational efficiency is conducted across these algorithms. By assessing their performance on diverse datasets, the study aims to provide insights into the trade-offs between different optimization strategies and their implications for practical clustering tasks. The results reveal distinct convergence patterns, highlighting the advantages and limitations of each optimization algorithm. Gradient-based approaches demonstrate rapid convergence but susceptibility to local optima, while stochastic gradient descent and metaheuristic algorithms exhibit a balance between exploration and exploitation. The findings shed light on the interplay between optimization techniques, convergence speed, and clustering quality, offering valuable guidance for practitioners seeking to optimize K-Means clustering according to specific dataset characteristics and computational requirements. This comparative study contributes to the broader understanding of optimizing K-Means clustering algorithms and aids researchers and practitioners in selecting suitable optimization strategies for efficient and effective data clustering in real-world applications. [ABSTRACT FROM AUTHOR]

A randomized block Kaczmarz method and a randomized extended block Kaczmarz method are proposed for solving the matrix equation A X B = C , where the matrices A and B may be full-rank or rank-deficient. These methods are iterative methods without matrix multiplication, and are especially suitable for solving large-scale matrix equations. It is theoretically proved that these methods converge to the solution or least-square solution of the matrix equation. The numerical results show that these methods are more efficient than the existing algorithms for high-dimensional matrix equations. [ABSTRACT FROM AUTHOR]

To obtain the iterative methods for solving a class of random generalized quasi variational inequalities (RGQVIs) in the Hilbert spaces, the measurability of the projection operators on varying-constraint sets depicted by the mapping from points to random-value sets with closed (convex) values, was proved. Moreover, the random iterative algorithm was proposed for solving RGQVIs, and the convergence of the random sequences generated with the random iterative algorithm was obtained under some suitable conditions of monotony and Lipschitz continuity. Finally, 2 applications were given with depicting results of the random generalized Nash games and random Walrasian equilibrium problems, respectively. [ABSTRACT FROM AUTHOR]

In this paper, by combining Polyak's inertial extrapolation technique for minimization problem with the viscosity approximation for fixed point problem, we develop a new type of numerical solution method for split feasibility problem. Under suitable assumptions, we establish the global convergence of the designed method. The given experimental results applied on the sparse reconstruction problem show that the proposed algorithm is not only robust to different levels of sparsity and amplitude of signals and the noise pixels but also insensitive to the diverse values of scalar weight. Further, the proposed algorithm achieves better restoration performance compared with some other algorithms for image recovery. [ABSTRACT FROM AUTHOR]

In this paper, we apply the Anderson acceleration technique to the existing relaxation fixed‐point iteration for solving the multilinear PageRank. In order to reduce computational cost, we further consider the periodical version of the Anderson acceleration. The convergence of the proposed algorithms is discussed. Numerical experiments on synthetic and real‐world datasets are performed to demonstrate the advantages of the proposed algorithms over the relaxation fixed‐point iteration and the extrapolated shifted fixed‐point method. In particular, we give a strategy for choosing the quasi‐optimal parameters of the associated algorithms when they are applied to solve the test problems with different sizes but the same structure. [ABSTRACT FROM AUTHOR]

We study high-order numerical methods for solving Hamilton-Jacobi equations. Firstly, by introducing new clear concise nonlinear weights and improving their convex combination, we develop WENO schemes of Zhu and Qiu (2017). Secondly, we give an algorithm of constructing a convergent adaptive WENO scheme by applying the simple adaptive step on the proposed WENO scheme, which is based on the introduction of a new singularity indicator. Through detailed numerical experiments on extensive problems including nonconvex ones, the convergence and effectiveness of the adaptive WENO scheme are demonstrated. [ABSTRACT FROM AUTHOR]

The Harris Hawks Optimizer (HHO) is a bio-inspired metaheuristic acknowledged for its effectiveness in addressing mono-objective optimization problems. However, its application has been limited to these specific challenges. To overcome this constraint and to navigate complex multi-objective optimization challenges, a Guided Multi-Objective variant of HHO, termed as Guided Multi-Objective Harris Hawks Optimization (GMOHHO), is introduced in this study. In the developed GMOHHO algorithm, an archival mechanism is integrated. This mechanism is specifically designed to store non-dominated solutions and to enhance their retrievability during the search process. Moreover, a robust multi-leader selection procedure is implemented, facilitating the steering of the primary set of solutions towards potential areas within the search space. Further, the Bi-Goal Evolution (BIGE) framework is utilized. This framework aids in the transformation of a search space with multitudinous objectives into a bi-objective one, thereby augmenting environmental selection. This integration ensures a balanced compromise between the convergence and diversity of solutions. The performance of the proposed GMOHHO algorithm was appraised across a series of test functions. The results consistently displayed its supremacy over the conventional HHO approach as well as other cutting-edge multi-objective optimization techniques. With its noteworthy capability to address a broad range of multi- objective optimization problems, the GMOHHO algorithm delivers high-quality solutions within acceptable computational timeframes. This study, therefore, paves the way for a promising approach to multi-objective optimization, potentially expanding the application sphere of the HHO algorithm. [ABSTRACT FROM AUTHOR]

In this article, we globalize the quasi-Newton algorithm proposed in Macías et al. (Appl Math Comput 441:127678, 2023) introducing an exact line search, we propose a polynomial approximation to the merit function and we deduce a sufficient condition for its minimization interval. We use the exact merit function and its approximation to propose two global quasi-Newton algorithms for solving matrix polynomial equations. For each algorithm, we prove that the exact line search does not affect the convergence of quasi-Newton method. In addition, we present comparative numerical tests of the algorithmic proposals in which we also compare with global Newton's method. [ABSTRACT FROM AUTHOR]