Search

For ensemble learning, how to select and combine the candidate classifiers are two key issues which influence the performance of the ensemble system dramatically. Random vector functional link networks (RVFL) without direct input-to-output links is one of suitable base-classifiers for ensemble systems because of its fast learning speed, simple structure and good generalization performance. In this paper, to obtain a more compact ensemble system with improved convergence performance, an improved ensemble of RVFL based on attractive and repulsive particle swarm optimization (ARPSO) with double optimization strategy is proposed. In the proposed method, ARPSO is applied to select and combine the candidate RVFL. As for using ARPSO to select the optimal base RVFL, ARPSO considers both the convergence accuracy on the validation data and the diversity of the candidate ensemble system to build the RVFL ensembles. In the process of combining RVFL, the ensemble weights corresponding to the base RVFL are initialized by the minimum norm least-square method and then further optimized by ARPSO. Finally, a few redundant RVFL is pruned, and thus the more compact ensemble of RVFL is obtained. Moreover, in this paper, theoretical analysis and justification on how to prune the base classifiers on classification problem is presented, and a simple and practically feasible strategy for pruning redundant base classifiers on both classification and regression problems is proposed. Since the double optimization is performed on the basis of the single optimization, the ensemble of RVFL built by the proposed method outperforms that built by some single optimization methods. Experiment results on function approximation and classification problems verify that the proposed method could improve its convergence accuracy as well as reduce the complexity of the ensemble system. [ABSTRACT FROM AUTHOR]

In the Artificial Bee Colony (ABC) algorithm, the employed bee and the onlooker bee phase involve updating the candidate solutions by changing a value in one dimension, dubbed one-dimension update process. For some problems which the number of dimensions is very high, the one-dimension update process can cause the solution quality and convergence speed drop. This paper proposes a new algorithm, using reinforcement learning for solution updating in ABC algorithm, called R-ABC. After updating a solution by an employed bee, the new solution results in positive or negative reinforcement applied to the solution dimensions in the onlooker bee phase. Positive reinforcement is given when the candidate solution from the employed bee phase provides a better fitness value. The more often a dimension provides a better fitness value when changed, the higher the value of update becomes in the onlooker bee phase. Conversely, negative reinforcement is given when the candidate solution does not provide a better fitness value. The performance of the proposed algorithm is assessed on eight basic numerical benchmark functions in four categories with 100, 500, 700, and 900 dimensions, seven CEC2005’s shifted functions with 100, 500, 700, and 900 dimensions, and six CEC2014’s hybrid functions with 100 dimensions. The results show that the proposed algorithm provides solutions which are significantly better than all other algorithms for all tested dimensions on basic benchmark functions. The number of solutions provided by the R-ABC algorithm which are significantly better than those of other algorithms increases when the number of dimensions increases on the CEC2005’s shifted functions. The R-ABC algorithm is at least comparable to the state-of-the-art ABC variants on the CEC2014’s hybrid functions. [ABSTRACT FROM AUTHOR]

This paper presents a stable and fast algorithm for independent component analysis with reference (ICA-R). This is a technique for incorporating available reference signals into the ICA contrast function so as to form an augmented Lagrangian function under the framework of constrained ICA (cICA). The previous ICA-R algorithm was constructed by solving the optimization problem via a Newton-like learning style. Unfortunately, the slow convergence and potential misconvergence limit the capability of ICA-R. This paper first investigates and probes the flaws of the previous algorithm and then introduces a new stable algorithm with a faster convergence speed. There are two other highlights in this paper: first, new approaches, including the reference deflation technique and a direct way of obtaining references, are introduced to facilitate the application of ICA-R; second, a new method is proposed that the new ICA-R is used to recover the complete underlying sources with new advantages compared with other classical ICA methods. Finally, the experiments on both synthetic and real-world data verify the better performance of the new algorithm over both previous ICA-R and other well-known methods. [ABSTRACT FROM AUTHOR]

We introduce a new iterative algorithm for finding a common element of a fixed point problem of amenable semigroups of nonexpansive mappings, the set solutions of a system of a general system of generalized equilibria in a real Hilbert space. Then, we prove the strong convergence of the proposed iterative algorithm to a common element of the above three sets under some suitable conditions. As applications, at the end of the paper, we apply our results to find the minimum-norm solutions which solve some quadratic minimization problems. The results obtained in this paper extend and improve many recent ones announced by many others. [ABSTRACT FROM AUTHOR]

In this paper, the Hager and Zhang (HZ) conjugate gradient (CG) method and the modified HZ (MHZ) CG method are presented for large-scale nonsmooth convex minimization. Under some mild conditions, convergent results of the proposed methods are established. Numerical results show that the presented methods can be better efficiency for large-scale nonsmooth problems, and several problems are tested (with the maximum dimensions to 100,000 variables). [ABSTRACT FROM AUTHOR]

Particle swarm optimization (PSO) is an evolutionary computing method based on intelligent collective behavior of some animals. It is easy to implement and there are few parameters to adjust. The performance of PSO algorithm depends greatly on the appropriate parameter selection strategies for fine tuning its parameters. Inertia weight (IW) is one of PSO’s parameters used to bring about a balance between the exploration and exploitation characteristics of PSO. This paper proposes a new nonlinear strategy for selecting inertia weight which is named Flexible Exponential Inertia Weight (FEIW) strategy because according to each problem we can construct an increasing or decreasing inertia weight strategy with suitable parameters selection. The efficacy and efficiency of PSO algorithm with FEIW strategy (FEPSO) is validated on a suite of benchmark problems with different dimensions. Also FEIW is compared with best time-varying, adaptive, constant and random inertia weights. Experimental results and statistical analysis prove that FEIW improves the search performance in terms of solution quality as well as convergence rate. [ABSTRACT FROM AUTHOR]

This paper considers the noncooperative maximization of mutual information in the vector Gaussian interference channel in a fully distributed fashion via game theory. This problem has been widely studied in a number of works during the past decade for frequency-selective channels, and recently for the more general multiple-input multiple-output (MIMO) case, for which the state-of-the art results are valid only for nonsingular square channel matrices. Surprisingly, these results do not hold true when the channel matrices are rectangular and/or rank deficient matrices. The goal of this paper is to provide a complete characterization of the MIMO game for arbitrary channel matrices, in terms of conditions guaranteeing both the uniqueness of the Nash equilibrium and the convergence of asynchronous distributed iterative waterfilling algorithms. Our analysis hinges on new technical intermediate results, such as a new expression for the MIMO waterfilling projection valid (also) for singular matrices, a mean-value theorem for complex matrix-valued functions, and a general contraction theorem for the multiuser MIMO watefilling mapping valid for arbitrary channel matrices. The quite surprising result is that uniqueness/convergence conditions in the case of tall (possibly singular) channel matrices are more restrictive than those required in the case of (full rank) fat channel matrices. We also propose a modified game and algorithm with milder conditions for the uniqueness of the equilibrium and con- vergence, and virtually the same performance (in terms of Nash equilibria) of the original game. [ABSTRACT FROM AUTHOR]

The Lloyd algorithm originated in the context of optimal quantization and represents a fixed point iteration for computing an optimal quantizer. Reducing average distortion at every step, it constructs a Voronoi partition of the domain and replaces each generator with the centroid of the corresponding Voronoi cell. Optimal quantization is obtained in the case of a centroidal Voronoi tessellation (CVT), which is a special Voronoi tessellation of a domain Ω ϵ ℝd having the property that the generators of the Voronoi diagram are also the centers of mass, with respect to a given density function ? ⩾ 0, of the corresponding Voronoi cells. The Lloyd iteration is currently the most popular and elegant algorithm for computing CVTs and optimal quantizers, but many questions remain about its convergence, especially in d-dimensional spaces (d > 1). In this paper, we prove that any limit point of the Lloyd iteration in any dimensional spaces is nondegenerate provided that Ω is a convex and bounded set and ? belongs to L¹(Ω) and is positive almost everywhere. This ensures that the fixed point map remains closed and hence the standard theory of descent methods guarantees weak global convergence of the Lloyd iteration to the set of nondegenerate fixed point quantizers. While previously only conjectured, the convergence properties of the Lloyd iteration are rigorously justified under such minimal regularity assumptions on the density functional. The results presented in this paper go beyond existing convergence theories for CVTs and optimal quantization related algorithms and should be of interest to both the mathematical and engineering communities. [ABSTRACT FROM AUTHOR]

The active appearance model (AAM) is a powerful tool for modeling images of deformable objects and has been successfully used in a variety of alignment, tracking, and recognition applications. AAM uses subspace-based deformable models to represent the images of a certain object class. In general, fitting such complicated models to previously unseen images using standard optimization techniques is a computationally complex task because the gradient matrix has to be numerically computed at every iteration. The critical feature of AAM is a fast convergence scheme which assumes that the gradient matrix is fixed Around the optimal coefficients for all images. Our work in this paper starts with the observation that such a fixed gradient matrix inevitably specializes to a certain region in the texture space, and the fixed gradient matrix is not a good estimate of the actual gradient as the target texture moves away from this region. Hence, we propose an adaptive AAM algorithm that linearly adapts the gradient matrix according to the composition of the target image's texture to obtain a better estimate for the actual gradient. We show that the adaptive AAM significantly outperforms the basic AAM, especially in images that are particularly challenging for the basic algorithm. In terms of speed and accuracy, the idea of a linearly adaptive gradient matrix presented in this paper provides an interesting compromise between a standard optimization technique that recomputes the gradient at every iteration and the fixed gradient matrix approach of the basic AAM. [ABSTRACT FROM AUTHOR]

The matrix rank minimization problem has applications in many fields, such as system identification, optimal control, low-dimensional embedding, etc. As this problem is NP-hard in general, its convex relaxation, the nuclear norm minimization problem, is often solved instead. Recently, Ma, Goldfarb and Chen proposed a fixed-point continuation algorithm for solving the nuclear norm minimization problem ( Math. Program., doi:, ). By incorporating an approximate singular value decomposition technique in this algorithm, the solution to the matrix rank minimization problem is usually obtained. In this paper, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving affinely constrained matrix rank minimization problems are reported. [ABSTRACT FROM AUTHOR]

This paper proposes an efficient method to solve the optimal power flow problem in power systems using Particle Swarm Optimization (PSO). The objective of the proposed method is to find the steady-state operating point which minimizes the fuel cost, while maintaining an acceptable system performance in terms of limits on generator power, line flow, and voltage. Three different inertia weights, a constant inertia weight (CIW), a time-varying inertia weight (TVIW), and global-local best inertia weight (GLbestIW), are considered with the particle swarm optimization algorithm to analyze the impact of inertia weight on the performance of PSO algorithm. The PSO algorithm is simulated for each of the method individually. It is observed that the PSO algorithm with the proposed inertia weight yields better results, both in terms of optimal solution and faster convergence. The proposed method has been tested on the standard IEEE 30 bus test system to prove its efficacy. The algorithm is computationally faster, in terms of the number of load flows executed, and provides better results than other heuristic techniques. [ABSTRACT FROM AUTHOR]

In this paper, we are interested in some convergent formulations for the unsymmetric collocation method or the so-called Kansa’s method. We review some newly developed theories on solvability and convergence. The rates of convergence of these variations of Kansa’s method are examined and verified in arbitrary–precision computations. Numerical examples confirm with the theories that the modified Kansa’s method converges faster than the interpolant to the solution; that is, exponential convergence for the multiquadric and Gaussian radial basis functions (RBFs). Some numerical algorithms are proposed for efficiency and accuracy in practical applications of Kansa’s method. In double–precision, even for very large RBF shape parameters, we show that the modified Kansa’s method, through a subspace selection using a greedy algorithm, can produce acceptable approximate solutions. A benchmark algorithm is used to verify the optimality of the selection process. [ABSTRACT FROM AUTHOR]

As power systems becoming more closely interconnected and are being deregulated in energy markets, distributed simulations among different dispatch centers are highly required for online full system analysis and control applications. In this paper a new algorithm for distributed transient stability simulation of interconnected power systems is presented. Based on a Jacobian-free Newton-GMRES(m) method, this algorithm requires only exchanges of states of boundary buses among different regions. Therefore, it has strong scalability in distributed computing environments built on heterogeneous computing resources. Moreover, several accelerating methods are developed to enhance its efficiency, including continuous preconditioning with adaptive preconditioners, predicting boundary conditions and multistep coordination. The standard IEEE 39-bus system and a real power system with 1165 buses were used as test systems. The test results show that these accelerating methods greatly enhance the convergence rate of the proposed algorithm and reduce communication costs remarkably, which proves the novel algorithm is feasible and can be adopted in wide area networks with high-latency. [ABSTRACT FROM AUTHOR]

In this paper, we consider algorithms for constructing n-terms approximations with nonnegative coefficients. The convergence theorem is proved for a “positive” analog of the Pure Greedy Algorithm. We establish a condition on the sequence of weakness coefficients which is sufficient for the convergence of the Positive Weak Greedy Algorithm. This condition is also necessary for the class of monotone sequences. [ABSTRACT FROM AUTHOR]

Abstract: A family of algorithms for approximate solution of the bound-constrained minimization problem was introduced in [K.A. Ariyawansa, W.L. Tabor, A class of collinear scaling algorithms for bound-constrained optimization: Derivation and computational results, Technical Report 2003-1, Department of Mathematics, Washington State University, Pullman, WA, 2003, submitted for publication. Available at http://www.math.wsu.edu/math/TRS/2003-1.pdf]. These algorithms employ the standard barrier method, with the inner iteration based on trust region methods. Local models are conic functions rather than the usual quadratic functions, and are required to match first and second derivatives of the barrier function at the current iterate. The various members of the family are distinguished by the choice of a vector-valued parameter, which is the zero vector in the degenerate case that quadratic local models are used. This paper presents a convergence analysis of the family of algorithms presented in [K.A. Ariyawansa, W.L. Tabor, A class of collinear scaling algorithms for bound-constrained optimization: Derivation and computational results, Technical Report 2003-1, Department of Mathematics, Washington State University, Pullman, WA, 2003, submitted for publication. Available at http://www.math.wsu.edu/math/TRS/2003-1.pdf]. Specifically, convergence properties similar to those of barrier methods using quadratic local models are established. [Copyright &y& Elsevier]

Abstract: In this paper, a noniterative identification procedure for neuro-fuzzy based Hammerstein model is presented. The proposed method not only avoids the inevitable restrictions on static nonlinear function encountered by using the polynomial approach, but also overcomes the problems of initialization and convergence of the model parameters, which are usually resorted to trial and error procedure in the existing iterative algorithms used for the identification of Hammerstein model. To construct the neuro-fuzzy based model, a clustering algorithm is presented to estimate the centers and widths of the model, and an analytical solution is developed to calculate the weights of the model in a noniterative manner. Examples are used to illustrate the applicability of the proposed method and a comparison with polynomial approach is made. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a new inversion free variant of the basic fixed point iteration method for obtaining a maximal positive definite solution of the nonlinear matrix equation . It is more accurate than Zhan''s algorithm (J. Sci. Comput. 17 (1996) 1167) and has less number of operations than the algorithm of Guo and Lancaster (Math. Comput. 68 (1999) 1589). We derive convergence conditions of the iteration and existence conditions of a solution to the problem. Finally, we give some numerical results to illustrate the behavior of the considered algorithm. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce and study a new system of variational inclusions involving (H, η)-monotone operators in Hilbert space. Using the resolvent operator associated with (H, η)-monotone operators, we prove the existence and uniqueness of solutions for this new system of variational inclusions. We also construct a new algorithm for approximating the solution of this system and discuss the convergence of the sequence of iterates generated by the algorithm. [Copyright &y& Elsevier]

Abstract: In this paper, a kind of nonlinear optimization problems with nonlinear inequality constraints are discussed, and a new SQP feasible descent algorithm for solving the problems is presented. At each iteration of the new algorithm, a convex quadratic program (QP) which always has feasible solution is solved and a master direction is obtained, then, an improved (feasible descent) direction is yielded by updating the master direction with an explicit formula, and in order to avoid the Maratos effect, a height-order correction direction is computed by another explicit formula of the master direction and the improved direction. The new algorithm is proved to be globally convergent and superlinearly convergent under mild conditions without the strict complementarity. Furthermore, the quadratic convergence rate of the algorithm is obtained when the twice derivatives of the objective function and constrained functions are adopted. Finally, some numerical tests are reported. [Copyright &y& Elsevier]

Abstract: This paper proves the convergence of some generalized Schwarz algorithms for solvingthe obstacle problems with a T-monotone operator. Numerical results show that the generalized Schwarz algorithms converge faster than the classical Schwarz algorithms. [Copyright &y& Elsevier]

In this paper we show that the lagged diffusivity fixed point algorithm introduced by Vogel and Oman in [SIAM J. Sci. Comput., 17 (1996), pp. 227­238] to solve the problem of total variation denoising, proposed by Rudin, Osher, and Fatemi in [Phys. D, 60 (1992), pp. 259­268], is a particular instance of a class of algorithms introduced by Voss and Eckhardt in [Computing, 25 (1980), pp. 243­251], whose origins can be traced back to Weiszfeld's original work for minimizing a sum of Euclidean lengths [Tôhoku Math. J., 43 (1937), pp. 355­386]. There have recently appeared several proofs for the convergence of this algorithm [G. Aubert et al., Technical report 94-01, Informatique, Signaux et Systèmes de Sophia Antipolis, 1994], [A. Chambolle and P.-L. Lions, Technical report 9509, CEREMADE, 1995], and [D. C. Dobson and C. R. Vogel, SIAM J. Numer. Anal., 34 (1997), pp. 1779­1791]. Here we present a proof of the global and linear convergence using the framework introduced in [H. Voss and U. Eckhart, Computing, 25 (1980), pp. 243­251] and give a bound for the convergence rate of the fixed point iteration that agrees with our experimental results. These results are also valid for suitable generalizations of the fixed point algorithm. [ABSTRACT FROM AUTHOR]

This paper is concerned with a classical denoising and deblurring problem in image recovery. Our approach is based on a variational method. By using the Legendre­Fenchel transform, we show how the nonquadratic criterion to be minimized can be split into a sequence of half-quadratic problems easier to solve numerically. First we prove an existence and uniqueness result, and then we describe the algorithm for computing the solution and we give a proof of convergence. Finally, we present some experimental results for synthetic and real images. [ABSTRACT FROM AUTHOR]